A prompt is the token sequence already present when generation starts. Put simply, it is all the text the model has been given before it produces its first new token.
The prompt is the starting sequence
The model works out the chances for the next token from the tokens it already has. Those starting tokens are the prompt. The model conditions on the prompt, which means every chance it computes depends on those tokens.
A small example shows the effect. Given "Once upon a", the token "time" gets a high chance. Given "The capital of France is", the token "Paris" does. Change the prompt and the chances change with it.
The prompt is input. It is already split into tokens before any new token is produced.
In a chat product, the prompt is not only your last message. The product packs the whole transcript into one sequence: the system instructions, every earlier message, and your newest one. That packed transcript is the prompt.
A prompt can contain an instruction, source text, and a question
A prompt is one input string, but it often has three parts in it:
- An instruction: a line saying what kind of text should follow.
- Source text: material the answer should draw on, such as a paragraph.
- A question: what is being asked about that material.
Here is a short example with all three:
Answer the question using only the paragraph below.
The library opens at 9 a.m. on weekdays and at 10 a.m. on Saturdays. It is closed on Sundays.
When is the library closed?
The instruction is not a separate command sent to the model. It is text inside the prompt, split into tokens like every other part. It has an effect because the model was trained on many texts where an instruction like this was followed by a matching answer.
The three parts make some continuations more likely than others. They do not guarantee a correct or complete answer.
The model continues that sequence
The output is the continuation: new tokens added after the last token of the prompt. In the library example, the continuation might be "The library is closed on Sundays."
The output is not a separate lookup in a database. There is no table of stored answers that the model searches. "Sundays" comes out because, given those exact prompt tokens, the calculation gives it a high chance. It happens to appear in the paragraph, and the question points at it.
If the paragraph were left out, the model would still continue the text. It would produce a likely-sounding answer about some library, which may not match this one. The prompt is the only information about this library that the model has.
The same words can tokenize in more than one way
Tokens are pieces of text, and they are not always whole words. The same word can be split into different tokens depending on what surrounds it. A word at the start of a line, the same word after a space, and the same word with a capital letter can each become different tokens.
So two prompts that look almost the same to you can be different sequences to the model. The next page shows the split on real text.