Explained · Models
Tokens
Models do not read letters or words. Text is first cut into tokens — common word-pieces from a fixed vocabulary — and every token is billed, counted against the context window, and produced one at a time. A rough rule for English: a token is about four characters, or three-quarters of a word.
Where it breaksThe rule of thumb only holds for ordinary English prose. Code, JSON, long numbers, rare names and most non-English languages fragment into far more tokens per character, so the same content can cost several times more in one language than another. Character-level tasks — counting letters, reversing a word — fail for the same reason: the model never saw the letters.
~4 characters per token, for ordinary English textOpenAI help centre, "What are tokens and how to count them" · 2024-01-01
Why text gets chopped up at all
A model works on numbers, so text has to become numbers first, and there are only bad options at the extremes. Give every word its own number and the vocabulary is unbounded — every typo, name and compound needs an entry, and anything unseen becomes a blank. Use single letters and the vocabulary is tiny but every sentence becomes an enormously long sequence, which is expensive because the work of reading grows faster than the length. Tokenisation splits the difference by building a fixed vocabulary of frequent chunks: whole words where they are common, fragments where they are not. "Unbelievable" might arrive as three pieces, "the" as one. The vocabulary is learned once, before training, by repeatedly merging the most frequent pair of characters in a large sample of text — so the chunks reflect the statistics of whatever that sample happened to contain.
Why this is the unit of money
Everything downstream is denominated in tokens, because the token is the unit of work. The model does one full pass of arithmetic per generated token, the context window is a token count, and providers bill input and output tokens separately — output costing several times more, because those are produced one at a time while input can be read in parallel. That makes token count the only honest measure of what a feature costs. Two prompts that look the same length on screen can differ by a factor of three in tokens if one is prose and the other is JSON, and a summarisation feature that looks cheap per call becomes the largest line on the bill once you multiply by volume. If you want to forecast an AI bill, count tokens per job and multiply — nothing else in the stack is as predictive.
Where it breaks
The four-characters rule is an English prose rule. Code fragments heavily on punctuation and indentation, JSON pays for every brace and quote, long numbers split into two- or three-digit pieces, and languages that were thinly represented when the vocabulary was built can take two or three times as many tokens to say the same thing — a real and rarely discussed cost difference between users in different countries. The other consequence is cognitive rather than financial. Because the model never sees letters, questions about spelling are genuinely hard for it: asking how many times a letter appears in a word is asking it to reason about the inside of a symbol it only ever received whole. Arithmetic on long numbers fails for the same structural reason.
What to do about it
Measure rather than estimate. Every provider ships a tokeniser you can run over a real sample of your traffic, and the answer is often twice what people guessed. Then attack the count where it is largest: trim the boilerplate that rides on every request, prefer terse formats over verbose ones for machine-to-machine work, cap output length explicitly rather than hoping, and cache the fixed prefix so it stops being re-read on every turn. And when a task is really about characters — validating, counting, reversing, formatting — write four lines of ordinary code instead of asking a model to do it. Knowing what a token is mostly buys you the judgement to stop paying for the wrong ones.