Model Settings

Parameters like temperature, top_p, max length, and stop sequences

Try the "Parameter Lab" at the top of the page to see how temperature changes affect generation results in real time.

Prompts are the "instructions" you give to AI. Parameters are the AI's "personality settings." In production, tweaking these parameters often fixes "unstable output" problems faster than rewriting your prompt.

1. The Creativity Engine: Temperature vs. Top_p

The key insight here: AI doesn't understand word meanings. It's just doing probability prediction.

Temperature (Sampling Temperature)

Under the hood: When picking the next token, the model scales the probability distribution across its entire vocabulary.

Low temperature (T -> 0): "Probability amplifier." High-probability tokens get even higher, low-probability ones basically vanish. The model becomes extremely conservative, always picking the safest word.
High temperature (T -> 1.5+): "Probability equalizer." Lower-probability tokens get a real shot. The model becomes adventurous -- output gets more surprising and creative.

Practical rules:

Need accuracy (writing code, extracting JSON): Lock it at 0.0.
Need diversity (writing fiction, brainstorming names): Try 1.2 or higher.

Top_p (Nucleus Sampling)

Under the hood: Sort tokens by probability, then only sample from the smallest set whose cumulative probability exceeds P.

What it does: Acts as a "noise filter."
Use case: Setting Top_p = 0.1 means the AI won't touch tokens in the bottom 90% of the probability distribution, even in "creative mode."

Parameter tuning rules of thumb:

Don't crank both at the same time.

Adjust Temperature first. If the AI starts producing gibberish (grammar errors, nonsense), lower Top_p to filter out long-tail noise.

2. Presence and Diversity: Penalties

When the AI gets stuck in loops or keeps using the same words, these are your surgical tools:

Presence Penalty: "Topic expander." Once a token has appeared, it gets penalized. This forces the AI to bring up new topics, increasing the "breadth" of the text.
Frequency Penalty: "Vocabulary enricher." The more a token appears, the heavier the penalty. This forces the AI to use synonyms, increasing the "texture" of the text.

3. Length and Boundaries (Constraints)

Context Window -- a core 2026 concept

In 2026, mainstream models (Gemini 3, GPT-5) support million-token or even unlimited context. But you still need to watch out:

Lost in the Middle: Models tend to have the weakest recall for content in the middle of long texts.
Cost control: Longer context means inference cost (and latency) grows exponentially or linearly.
Max Output Tokens: This only limits the reply length -- it doesn't affect how much the model can "see."

Stop Sequences

These aren't just for preventing AI rambling -- they're logic control.

Pro tip: In few-shot prompting, set \n as a stop sequence to force the AI to generate only one line at a time. Great for batch data generation.

The Parameter Mental Model: Dashboard Method

We split tasks into three "quadrants" -- find yours:

Quadrant A: Hard Logic (Code, Logic, Math)

Config: Temp: 0.0, Top_p: 1.0, Penalty: 0.0
Mindset: You're a strict instructor. 1+1 must equal 2. Zero randomness allowed.

Quadrant B: Content Creation (Email, Summary, Translation)

Config: Temp: 0.7, Top_p: 0.9, Penalty: 0.1
Mindset: You're an editor-in-chief. You want smooth prose with professional depth and enough variation to not sound robotic.

Quadrant C: Creative Explosion (Brainstorming, Fiction)

Config: Temp: 1.3, Top_p: 1.0, Penalty: 0.5
Mindset: You're the client saying "show me something I haven't seen before" -- and you're okay with the occasional miss.

Advanced: Scientific Parameter Tuning

Control your variables: Keep the prompt constant, only change parameters.
Fix the Seed: During testing, use a fixed Seed value so you know improvements come from parameter changes, not random luck.
Parallel sampling: At Temperature = 1.0, use n > 1 (generate multiple results at once) and pick the best one.

Next up: Now that you've got the "personality settings" down, we're heading into the real arena -- Core Techniques, where you'll learn how prompt structure directly changes how AI thinks.

📚 Related resources

❓ Common questions

Open a question to review the practical answer.

How big is the gap between temperature 0 and 1.5?

Under the hood it's probability-distribution scaling: at T=0 high-probability tokens get amplified and low-probability ones flattened, so the model picks the safest token every time—mandatory for code or JSON extraction. At T=1.5+ probabilities get equalized, giving long-tail tokens a real shot—reserved for fiction or naming. The chapter's rule: lock at 0.0 for accuracy, push past 1.2 for diversity.

Should I tune temperature and top_p at the same time?

No. The chapter is explicit: 'don't crank both at the same time.' Adjust temperature first; if the model starts producing gibberish (grammar errors, nonsense) at high temperature, lower top_p as a noise filter—e.g. top_p=0.1 cuts the bottom 90% of the distribution. Tweaking both at once makes it impossible to attribute the change.

What's the difference between presence penalty and frequency penalty?

Presence penalty is the 'topic expander': a token gets penalized once it appears at all, regardless of frequency, pushing the model toward new topics and broader text. Frequency penalty is the 'vocabulary enricher': the penalty grows with each repetition, forcing synonym variation. When the model is stuck in a loop, these are your surgical tools.

What is Lost in the Middle, and does it still affect long-context models?

Lost in the Middle is the empirical finding that LLMs recall the start and end of long inputs well but lose information in the middle. It still applies in 2026 even though Gemini 3 and GPT-5 support million-token or unlimited context. Put critical information at the top or bottom of your prompt—never bury it in the middle.

What parameter combos match the three task quadrants?

Quadrant A (hard logic—code, math): Temp 0.0 / Top_p 1.0 / Penalty 0.0. Quadrant B (content—email, summary, translation): Temp 0.7 / Top_p 0.9 / Penalty 0.1. Quadrant C (creative—brainstorming, fiction): Temp 1.3 / Top_p 1.0 / Penalty 0.5. Pair with a fixed seed for controlled-variable testing, otherwise you can't tell parameter wins from random luck.