Roll the dice: how LLMs pick the next word

An early language model does two things: it turns everything it has read so far into odds over the possible next words, and then it rolls for one. Both animations below use the same fair 100-sided die.

Part 1 · The homework assignment

A word predictor that only ever looks at the last word. “Orange” goes in, and this fixed set of odds comes out — every time.

Part 2 · What makes an LLM different

Same five words, same die. But a real model weighs the whole sentence, so changing one earlier word redraws the odds. Switch the context and watch the slices move.