learn me a gpt
Pages in Beginners

19 Nov 2026

Using a chatbot

A chatbot sends the conversation so far to a model and appends the model’s new tokens to that conversation.

A chatbot sends the conversation so far to a model and appends the model’s new tokens to that conversation. Each time you press send, the whole conversation goes in, and the reply is added to the end of it.

The model underneath is the same kind described on the previous page. The chatbot is the program that keeps the conversation and passes it along.

The window holds a transcript

A transcript is the written record of the conversation: an ordered list of messages, oldest first. Here is a short one.

  1. user: What is the capital of France?
  2. assistant: Paris.
  3. user: And of Italy?

The third message only makes sense because the first two are still in the list. "And of Italy?" does not say what is being asked. The transcript does.

When the model answers, its reply becomes message 4. The newest reply is not stored somewhere separate. It is more tokens added to the end of the same list.

Roles label the messages

Each message in the transcript carries a label that says who it is from. Chat APIs commonly use three labels. (An API is the way one program sends requests to another.)

  • system: instructions placed at the start by the product, before you type anything. You often cannot see them.
  • user: what you type.
  • assistant: what the model generated earlier in the conversation.

These labels are text in the transcript. They are not three separate models. One model reads all of it. Before sending, the product writes the whole transcript out as one long piece of text, with the labels as markers. A simplified version looks like this:

system: Answer briefly.
user: What is the capital of France?
assistant: Paris.
user: And of Italy?
assistant:

The model still only continues tokens

After the product packs the transcript into one piece of text, the text is split into tokens. The model then runs the same loop as on the previous page, next-token prediction:

  1. Work out a chance for every possible next token.
  2. Choose one token.
  3. Add it to the end.
  4. Repeat until an end token or a length limit is reached.

In the example above, the text ends with "assistant:", so tokens like "Rome" get a high chance. The model is not answering a question in any special mode. It is continuing a piece of text that happens to be shaped like a conversation.

The model itself keeps nothing between replies. The earlier messages reach it because they are part of the transcript that is sent again.

Compare it to passing someone a whole thread of letters each time and asking for the next letter. The comparison stops there: the model does not read like a person. It computes chances for the next token.

A reply is appended, then the loop can run again

One full turn of a conversation goes like this:

  1. You type a message. The product adds it to the transcript as a user message.
  2. The product packs the whole transcript and sends it to the model.
  3. The model generates new tokens, one at a time.
  4. The product adds those tokens to the transcript as an assistant message and shows them to you.
  5. Your next message is added after the assistant tokens, and the whole cycle starts again from step 2.

So the transcript grows with every turn. The model receives a longer piece of text each time, and continues it in the same way.

Transcriptsystemuserassistantassistant (new)Modelnew tokens
The transcript is the model’s input