learn me a gpt
Pages in Tools

19 Nov 2026

Tokenizer

The tool encodes text with the GPT-2 tokenizer and shows the token strings and ids.

This tool splits your text into GPT-2 tokens and shows each token’s text and its id number.

The splitting runs in your browser, and the text is not sent anywhere.

Up to 200 characters. 0 / 200 used.

Tokens:0

Type some text to see its tokens.

The text is split by the GPT-2 tokenizer in this browser and is not sent anywhere.

The Python below shows the same call with the tiktoken library. It is the same GPT-2 mapping from text to ids as the browser tool, not a second algorithm, so it gives the same ids for the same sentence.

import tiktoken

enc = tiktoken.get_encoding("gpt2")
ids = enc.encode("A cat sat on the mat.")
print(ids)              # [32, 3797, 3332, 319, 262, 2603, 13]
print(enc.decode(ids))  # A cat sat on the mat.