How GPT models work, one token at a time
This site explains how GPT-style language models work, from the text in a chat window down to the calculation that produces the next token. When a page inspects a real model, that model is GPT-2 small.
- BeginnersThe reading order, in plain language.
- TechnicalThe same system, one calculation at a time.Not written yet
- ToolsSmall calculators for the operations those pages describe.
- ExplorerGPT-2 small, opened on one fixed sentence.Not written yet
The full list includes titles that are not written yet. The header field finds a written page or tool by name.