Inspect every value
Use small vectors and a limited vocabulary so the computation can be followed by hand.
Teaching tool · AI education / Interactive systems · 2026
The Transformer, Drawn
An interactive working drawing that makes the arithmetic of attention visible, one step at a time.
The problem
Connect high-level transformer diagrams with the arithmetic behind attention, masking and next-token probabilities.
The approach
Use a small deterministic simulation and render its intermediate values in ten SVG figures. Let readers change the sentence, select tokens and heads, and adjust temperature while following the computation.
What the project delivers
An interactive explanation with inspectable values, keyboard controls and standalone SVG export. The small model runs in the browser without API keys or a GPU.
Design decisions
Deterministic examples
Seed the simulation so changes can be compared meaningfully.
Reusable explanations
Export figures as SVG for lessons, slides and discussion.