← Curriculum map
L07 · Phase 3 · 25 min

Transformers

“If the model predicts one token at a time, how does it keep track of relationships across a sentence or document?”

Transformer Pipeline — build the order

One token's journey through a transformer passes through the same stages, in the same order, every time. Click the stages below in what you think is the correct order.

Your order will build here…
Prototype note: this is the pipeline order for one transformer layer, then repeated — real models stack dozens of these layers, each with its own learned weights, before the final prediction.

Depth ladder

A transformer is a pipeline: text goes in one end, gets broken into pieces, mixed with context from every other piece, refined, and repeated — the same handful of stages, over and over, until a prediction comes out the other end. Try ordering the stages yourself below.

Knowledge check

Reorder six shuffled stage labels into the correct transformer pipeline order.