← Curriculum map
L11 · Phase 4 · 20 min

Where Did the Knowledge Come From?

How did the model know what an immersion heater or a solar coil was, when I never explained it?

True or False?

AI Tool has a searchable file of every document it was trained on, and looks things up in it.

AI Tool's knowledge of immersion heaters comes from statistical patterns learned across huge amounts of text, not a lookup.

If a document wasn't in AI Tool's training data, it's mathematically impossible for the model to say anything relevant about the topic.

Scale: training text vs. stored parameters

Text read during training

Hundreds of billions of words — manuals, forum posts, product descriptions, and far more — read once, during training.

What gets encoded as parameters

A much smaller set of adjustable numbers — statistical structure distilled from all that text, not a copy of it.

Prototype note: the comparison above is illustrative, not a precise measurement — the actual compression ratio between training corpus size and parameter count varies enormously by model and isn’t a single agreed-upon number.

Depth ladder

Long before this conversation started, the model read an enormous amount of text — technical manuals, forum posts, product descriptions, and much more — and that reading shaped it. It knew what an immersion heater was for the same reason a well-read person would: it had encountered the idea, many times, in many contexts, before you ever asked.

Knowledge check

True or false: 'AI Tool has a searchable file of every document it was trained on.' Explain why.