← Curriculum map
L05 · Phase 3 · 30 min

How a Network Learns

“Who chooses the billions of weights inside a model like this?”

Gradient Descent Animation

The curve below is a simplified loss landscape for one weight in the same toy network from Neural Networks. Lower on the curve means fewer mistakes. Press Step to nudge the ball downhill.

minimum
current weight w = 12.50loss = (w − 6)² = 42.25gradient = 2(w − 6) = 13.00

0 steps taken. Each step: w ← w − (learning rate × gradient). At 0.50 exactly, one step lands precisely on the minimum. Push past that — try 1.00 — and it starts bouncing from one side of the minimum to the other instead of settling; push higher still and each bounce gets bigger instead of smaller.

Prototype note: the loss curve here is a hand-picked parabola for one weight, not a real training loss surface — real networks have millions of weights and a correspondingly high-dimensional, much bumpier landscape. The stepping rule (w ← w − lr × gradient) is exactly gradient descent’s real update rule.

Depth ladder

Nobody hand-picks the billions of weights inside a model like this. Instead, the model is shown an example, checked against the correct answer, and nudged — a tiny amount, many millions of times — toward making fewer mistakes. Press Step below and watch one weight get nudged toward a better value.

Knowledge check

Given a loss-landscape picture and a current position, pick the direction that reduces loss.