“In '…a 300 L tank with two coils and a 2 kW electric immersion heater,' what actually matters when I ask how long heating will take?”
Attention Heatmap
"How long would the heater take to heat the tank?"
Darker highlight = higher attention weight. Your clicked guesses stay outlined for comparison.
Click any highlighted word to inspect its Query/Key dot product and softmax weight.
Prototype note: the raw relevance scores and Q/K vectors are hand-authored to be plausible, not extracted from a real model — but the softmax and dot-product arithmetic shown is computed live, for real, on those numbers.
Depth ladder
Not every word in a sentence matters equally once you ask a specific question. If you ask 'how long would the heater take to heat the tank?', the words '300', 'kW', 'heater', and 'tank' matter a lot — they're what the answer actually depends on. Words like 'a', 'with', and 'and' barely matter at all. Attention is the mechanism that lets the model weigh each word by how relevant it is to the question being asked right now, so it can lean harder on the words that actually carry the answer. Try it yourself below: click the words you think matter most, then reveal how the model actually weighted them.
Knowledge check
Given a sentence and a question, predict which words the attention heatmap should light up before revealing it.