Biases
How the distribution and order of few-shot examples affect outputs
How the distribution and order of few-shot examples affect outputs
A reusable prompt, example set, evaluation record or safety rule with its task boundary preserved.
Run at least one representative case, inspect the result and record what still requires human review.
LLMs can produce problematic outputs that negatively impact model performance on downstream tasks and display biases that degrade results. Some of these can be mitigated through effective prompting strategies, though harder cases may require more advanced solutions like moderation and filtering.
Distribution of Exemplars
When doing few-shot learning, does the distribution of exemplars affect model performance or bias the model in some way? Here's a simple test.
Prompt:
Q: I just got the best news!
A: positive
Q: We just got a raise at work!
A: positive
Q: I'm very proud of what I accomplished today.
A: positive
Q: I had a great day today!
A: positive
Q: I'm really looking forward to the weekend.
A: positive
Q: I just got the best gift!
A: positive
Q: I'm very happy right now.
A: positive
Q: I'm lucky to have such an amazing family.
A: positive
Q: The weather outside is very gloomy.
A: negative
Q: I just heard some terrible news.
A: negative
Q: That feels unpleasant.
A:
Output:
negative
In the example above, the skewed distribution of exemplars doesn't seem to bias the model. Good. Now let's try a harder-to-classify example and see how the model handles it:
Prompt:
Q: The food here is delicious!
A: positive
Q: I'm tired of this course.
A: negative
Q: I can't believe I failed the exam.
A: negative
Q: I had a great day today!
A: positive
Q: I hate this job.
A: negative
Q: The service here is terrible.
A: negative
Q: I feel very depressed about my life.
A: negative
Q: I never get a break.
A: negative
Q: This meal tastes awful.
A: negative
Q: I can't stand my boss.
A: negative
Q: I feel something.
A:
Output:
negative
That last sentence is pretty subjective. So I flipped the distribution — used 8 positive examples and 2 negative — then tried the exact same sentence again. Guess what? The model answered "positive." The model probably has a lot of built-in knowledge about sentiment classification, so it's hard to get it to show bias on this topic. The takeaway: avoid skewed distributions and provide a more balanced number of examples for each label. For harder tasks where the model has less prior knowledge, this becomes a bigger problem.
Order of Exemplars
When doing few-shot learning, does the order of exemplars affect model performance or bias things?
You can try the examples above and see if reordering makes the model lean toward a particular label. The recommendation: randomize the order of exemplars. For example, avoid putting all positive examples first and negative examples last. This problem gets amplified if the label distribution is already skewed. Make sure to experiment extensively to reduce this type of bias.
📚 Related resources
❓ Common questions
Open a question to review the practical answer.
Can the distribution of few-shot examples bias the model toward one label?
Yes, but weaker than people fear. The chapter tests it: with a skewed 8 positive + 2 negative few-shot set, the model still labelled the ambiguous `I feel something` as negative — not following the majority. Sentiment is a domain the model knows well. Takeaway: skewed distribution barely moves a familiar task but amplifies bias on tasks it knows poorly — so balance examples by default.
How should I order few-shot examples?
Randomise, do not put all positives first and all negatives last. The chapter notes the more skewed the distribution, the more order amplifies the bias. Production tip: store examples as an array and either shuffle before each call, or freeze one verified order and log it in the changelog — otherwise you will see outputs shift after an `unrelated` change and never trace why.
How do I tell if my prompt is already poisoned by bias?
Run a controlled test: same input set, two prompt variants — original example order vs reversed, original distribution vs flipped. If outputs shift noticeably, the prompt is contaminated. The chapter recommends `running many experiments to reduce this kind of bias`. The engineering version is to bake those controlled tests into your evaluation pipeline and run them on every prompt change.
How much worse does bias get on tasks the model is unfamiliar with?
The chapter flags this: the model is hard to bias on familiar tasks like sentiment, but `for harder tasks where the model lacks knowledge it can be much more difficult`. In domain-specific work, internal jargon or low-resource languages, distribution and order bias are amplified. The countermeasures: double the example count, force balance, and add retrieval to pump real domain knowledge into the context.
Should bias mitigation happen at the prompt layer, the model layer, or the data layer?
The chapter is explicit: `some bias can be mitigated by effective prompting, but more advanced solutions like moderation and filtering may be required`. The prompt layer handles local bias — example distribution, ordering, wording. Systemic bias — gender, race, cultural default assumptions — needs data curation, RLHF and output moderation. Prompting is the first line of defence, not the only one.