learn me a gpt
Pages in Beginners

19 Nov 2026

Temperature

Temperature divides the logits by a positive number before softmax. Above 1 the distribution flattens. Below 1 it sharpens.

Temperature is a setting that changes how evenly the model's probability is spread across the possible next tokens. Precisely, temperature divides the logits by a positive number before softmax. Above 1 the distribution flattens. Below 1 it sharpens.

Logits are the model's raw scores, one per possible next token. Softmax turns those scores into probabilities that add up to 1. This page uses one small example throughout: after "The cat sat on the", suppose only three tokens are possible, "floor", "sofa", and "mat", with logits 1, 2, and 3. These scores are made up to keep the arithmetic short.

Divide the logits, then softmax

Temperature adds one step before softmax:

  1. Pick a temperature, written T. It must be a number above 0.
  2. Divide every logit by T.
  3. Apply softmax to the results.

Written as a formula, the probabilities are softmax(logits / T). T must be positive. Zero would mean dividing by zero, and a negative number would flip the order of the scores.

Dividing changes the spread because softmax responds to the gaps between scores. With logits 1, 2, and 3, each gap is 1. Dividing by 2 gives 0.5, 1, and 1.5, so each gap shrinks to 0.5, and smaller gaps give probabilities that sit closer together. Dividing by a number below 1 does the opposite: T = 0.5 gives 2, 4, and 6, so the gaps double and the top token takes a larger share. The order never changes. "mat" stays the most likely token at every temperature.

T = 1 leaves the model’s distribution

Dividing by 1 changes nothing, so the logits stay 1, 2, and 3. Softmax then works as usual:

  1. Raise the number e (about 2.718) to the power of each score: 2.7183, 7.3891, and 20.0855.
  2. Add them up: 30.1929.
  3. Divide each one by the total: 0.0900, 0.2447, and 0.6652.

So at T = 1, "floor" gets 0.0900, "sofa" gets 0.2447, and "mat" gets 0.6652. These are the model's own probabilities, unchanged.

T = 2 flattens it

Now divide each logit by 2 first:

  1. The logits 1, 2, and 3 become 0.5, 1, and 1.5.
  2. Raise e to each: 1.6487, 2.7183, and 4.4817.
  3. Add them up: 8.8487.
  4. Divide each one by the total: 0.1863, 0.3072, and 0.5065.

The largest probability fell from 0.6652 to 0.5065. The smallest rose from 0.0900 to 0.1863. The probabilities still add up to 1, and "mat" is still first, but the three bars are closer in height.

0.09000.24470.66520.18630.30720.5065floorsofamatfloorsofamatT = 1T = 2
A larger temperature moves probability toward the smaller scores.
Scores (logits):
T = 1.0

Each score is divided by T, then softmax turns the scores into probabilities that add up to 1. T below 1 makes the top option more likely; T above 1 spreads the chances out.

OptionScoreProbability at T = 1Probability at T = 1.0
110.09000.0900
220.24470.2447
330.66520.6652
Draw picks one option at random, using the probabilities at the chosen T.

The calculation runs in this browser. The numbers are not sent anywhere.

A computer that raises e to a large score can run out of room for the result. The usual fix is to subtract the largest score from every score before raising e. At T = 2 that turns 0.5, 1, and 1.5 into −1, −0.5, and 0. Raising e to those gives 0.3679, 0.6065, and 1, which add up to 1.9744, and dividing gives the same 0.1863, 0.3072, and 0.5065. This version is called a stable softmax.

The code below divides each logit by T, subtracts the largest result, applies softmax, and prints the probabilities for T = 1 and T = 2.

import math

def softmax_with_temperature(logits, T):
    scaled = [x / T for x in logits]
    biggest = max(scaled)
    exps = [math.exp(x - biggest) for x in scaled]
    total = sum(exps)
    return [e / total for e in exps]

logits = [1, 2, 3]
for T in [1, 2]:
    probs = softmax_with_temperature(logits, T)
    print("T =", T, [f"{p:.4f}" for p in probs])
# T = 1 ['0.0900', '0.2447', '0.6652']
# T = 2 ['0.1863', '0.3072', '0.5065']

A draw is one sample

Temperature changes the probabilities. It does not choose a token. Choosing is a separate step called sampling:

  1. Temperature and softmax produce the probabilities.
  2. A random draw picks one token. Each token's chance of being picked equals its probability.
  3. The picked token is appended to the text, and the next step begins.

A spinner is a fair comparison. Picture a wheel split into slices sized 0.1863, 0.3072, and 0.5065 of the circle. One spin lands on one slice. The comparison stops at the wheel: the computer draws a random number between 0 and 1 and checks which slice that number falls in.

One draw at T = 2 might land on "floor", even though "mat" has the largest probability. That single result is not the distribution. Only many draws show it: out of 10,000 draws at T = 2, about 1,863 would be "floor", about 3,072 "sofa", and about 5,065 "mat".

This is also why the same prompt can give different replies. Every new token is a fresh draw from that step's probabilities, and temperature only sets the sizes of the slices.