Skip to main content

Insight · November 18, 2026

What is
temperature in AI?

One dial that controls how random a model's output is. Low for consistent, predictable answers. High for varied, surprising ones. Nothing to do with how smart the model is.

01 · The definition

A dial for randomness, not a measure of quality.

Temperature is a setting that controls how random a language model's output is. It scales the probabilities the model assigns to each possible next word before one is chosen. A low temperature makes the model favor its most likely word, so answers stay consistent. A high temperature flattens those odds, so output grows more varied and creative.

The name comes from physics, where temperature describes how much a system moves at random. A model has no heat inside it. What the setting changes is one step in how the model picks each word: how much weight it gives its top choice against everything else it considered. Turn it down for order. Turn it up for range.

02 · How it works

It reshapes the odds before a word is drawn.

At each step a model does not simply pick a word. It produces a score for every word it knows, then turns those scores into probabilities with a function called softmax. Temperature is applied inside that step. The model divides each score by the temperature before the probabilities are worked out, which stretches or compresses the gap between the leading word and the rest.

Lower the temperature and the distribution grows sharper. A Hugging Face guide to text generation puts the mechanism plainly: the trick is to make the distribution sharper by “increasing the likelihood of high probability words and decreasing the likelihood of low probability words” as you lower the temperature of the softmax. Raise it and the distribution flattens, so words the model rated unlikely get a real chance of being chosen.

Push the temperature toward zero and the behavior has a name. The same guide notes that in its limit, as temperature approaches zero, the sampling “becomes equal to greedy decoding,” where the model takes its single highest scoring word every time. That is why a near zero temperature reads as consistent, and a high one reads as loose.

The one line to keep

“Temperature does not make a model smarter or dumber. It only decides how often it reaches past its first choice.”

03 · What the dial changes

Four things the setting actually moves.

01

The gap between choices

Before the setting is applied, every candidate word carries a score the model assigned. Temperature decides how much that score counts. Low, and the top word wins almost every time. High, and second and third choices start to surface.

02

Consistency across runs

Ask the same question twice at a low temperature and the answers tend to match. Ask at a high one and they drift. This is why a system that needs a stable format or a fixed schema runs cold.

03

Range of vocabulary

A higher value lets the model reach for rarer words and less obvious phrasing. It is what people mean when they say a setting makes a model more creative. The variety is real, and so is the cost of it.

04

Risk of a wrong turn

The same reach that adds variety adds error. A word the model rated unlikely is sometimes unlikely for a good reason. A higher temperature raises both the surprise and the chance the answer goes off.

04 · The other knob

Two controls, and a rule about using both.

Temperature is not the only control over randomness. The other common one is nucleus sampling, set through a parameter called top_p. Where temperature rescales the odds on every word, nucleus sampling trims the list. It keeps the smallest set of words whose combined probability passes a threshold you set, then samples only from those. The Hugging Face guide describes it as choosing “from the smallest possible set of words whose cumulative probability exceeds the probability p,” so the pool grows when the model is unsure and shrinks when it is confident. The idea traces to work by Ari Holtzman and colleagues in 2019.

The two settings interact, and most providers tell you to pick one. OpenAI's API documentation, carried in its official Python library, says temperature runs “between 0 and 2,” that “higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic,” and then adds a plain instruction: “We generally recommend altering this or top_p but not both.” Turning both at once makes the result hard to reason about, because each is reshaping the same distribution in a different way.

There is no single scale across vendors either. OpenAI's temperature runs from 0 to 2. Other providers use their own range, so a value of 1 is not the same setting everywhere. The number means little on its own. What travels is the direction: lower for order, higher for variety.

05 · One example that shows the whole idea

The same prompt, cooled and heated.

Picture one prompt: name a color. At a low temperature the model answers blue almost every time, because blue sits at the top of its list and the low setting makes the top choice win. Raise the temperature and the same prompt starts returning teal, crimson, amber. Nothing about the model changed. The odds it held all along were simply allowed to show.

There is a catch that trips up teams who expect a low number to lock the output. Setting temperature to zero does not guarantee the same answer twice. Anthropic states it directly in its own glossary: “Even with temperature set to 0, the results will not be fully deterministic, and identical inputs might produce different outputs across API calls.” For a reproducible result, temperature is the wrong lever. A fixed random seed, where the provider offers one, is the setting built for that, and even then the providers note determinism is not guaranteed.

temperature 0.2    name a color  →  blue · blue · blue · blue
                   the top choice wins almost every time

temperature 1.2    name a color  →  teal · crimson · amber · blue
                   lower ranked words get a real chance

Same model · same prompt · one setting deciding how far it reaches

06 · The operator view

Most production systems run cold.

For anything a business ships, the default is a low temperature. A support answer, a data extraction, a summary, a classification: these need the same input to yield the same shape of output, and a low setting is how you get there. The reach that a high temperature buys is a liability when the job has one right answer.

The exceptions are real but narrow. Brainstorming, naming, first drafts, and variation on a theme all benefit from a higher value, because the goal is range rather than a single correct line. Even then the output is a starting point a person edits, not a result a system trusts. Temperature sits alongside the other standing controls on a model call, the kind set in the system prompt and exercised through tool calling.

It also interacts with the newer class of reasoning models, which spend extra steps thinking before they answer. For those, the sampling setting matters less than it once did, because the structure of the reasoning carries more of the weight than the single dial does.

07 · Where the model meets the work

A default, set once, applied everywhere.

In practice the temperature on a given call should not be a number a developer types from memory each time. It is a policy decision: which tasks run cold, which run warm, and who is allowed to change that. When one product calls more than one model, each with its own scale, that policy has to hold across all of them at once.

That standardizing layer is where Bttr. works. We build the Token Control Plane, “the middle layer between you and every AI vendor,” where routing and policy for model calls live in one place rather than scattered through the code. A setting like temperature then becomes a rule the system keeps, not a guess repeated across a hundred call sites.

Closing

A model holds more answers than it shows.

Temperature is the dial that decides how many you hear. Keep it low when the work has one right answer. Raise it when you want range. Just never read a warmer setting as a smarter model, because the intelligence was fixed before you turned the dial. All you moved was how far it reaches.

Donny Smith

Written by · November 18, 2026

· Founder & Executive Creative Director, Bttr.

Over the past 15 years, he has led creative teams and contributed to products used by millions of people worldwide, working with companies including Apple, Opendoor, JP Morgan, GE Aerospace, Pepsi, and Alterra Mountain Company.

LinkedIn

Share this perspective

More insights

Adjacent perspectives.

What Is a Multimodal Model

7 min read

What Is a Multimodal Model

A multimodal model is an AI system that takes in and reasons across more than one kind of data, such as text, images, audio, and video, inside a single context. It maps each input into one shared representation, so it can relate a word, a picture, and a sound as parts of the same thing. The distinction that matters is how it is built. When Google launched Gemini in December 2023, it noted that the standard approach trained separate components for each modality and then stitched them together, and said Gemini was instead pre-trained from the start on different modalities. OpenAI made the same move with GPT-4o in May 2024, a single model trained end to end across text, vision, and audio so that every input and output runs through one neural network, which is why it answers audio in about 320 milliseconds rather than the seconds of lag in a stitched voice pipeline. The operator rule follows: a native multimodal model keeps every sense in the same context, while a vision tool bolted onto a text model reasons only over what it already turned into words.

What Is Tool Calling

7 min read

What Is Tool Calling

Tool calling is how a language model acts on the world. You describe a function to the model as a name, a description, and a schema for its inputs. When a request needs it, the model does not answer in prose. It returns a structured call naming the function and its arguments, your code runs that call, and the result goes back so the model can answer from something real. Anthropic puts it plainly: tool use, also called function calling, lets a model call functions you define or that the vendor provides, and the model only decides what to run, never runs it. The names differ across vendors and the substance does not. OpenAI shipped the idea as function calling, Anthropic calls it tool use, and the Claude documentation settles the overlap in one line. This is the base layer under every agent, since each single act an agent takes is a tool call, and the Model Context Protocol standardizes the layer above it so one tool, described once, can be reached by any model that speaks the protocol.

Bttr. Field Brief

The brief Bttr. writes for senior buyers.

Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.