7 stops · Explore at your own pace Sound optional · No AI subscription required
↑ WHAT WE SEE ↓ WHAT WE DON’T
A NEW EXPEDITION / CONCEPT DEMO
01 / ABOVE THE WATERLINE
You met the snowman.
Picture a carefully sculpted snowman sitting on top of an iceberg. This represents an application like ChatGPT, Claude or Gemini.
The application is the part you see: the interface, the conversation, the buttons. Beneath it sits a model. Its behaviour is shaped by training, instructions and the tools the application makes available.
In the AI Iceberg metaphor, the visible ice represents the model. The huge submerged body represents the data used to train it. The surrounding ocean is the wider environment that data comes from.
FIELD EXPERIMENT 01↕
Same question. Different instructions.
“What is an iceberg?”
Scripted examples. Instructions shape responses; they do not guarantee that the model follows them.
The interface is only the tip.
02 / THE TOKEN ROOM
First, break the words apart.
Before an LLM can process text, it must break it into small pieces called tokens—like Scrabble tiles the model can work with.
A token can be a word, part of a word, punctuation or another sequence of characters. Each token is mapped to a numerical identifier. Different tokenizers split the same sentence differently.
FIELD EXPERIMENT 02▤
Open the sentence.
The iceberg is unbelievable.
An illustrative split, not the output of a particular model’s tokenizer.
Next, those identifiers become numerical vectors: learned representations the network can operate on. That is the starting point for embeddings.
Fluent language begins as numbers.
03 / THE ATTENTION CHAMBER
A word needs its neighbours.
A “mole” can be an animal, a unit in chemistry, or a mark on your skin. The letters stay the same. The context changes.
Inside a transformer, attention combines information from different token positions. This helps build representations that depend on the surrounding text, rather than treating every appearance of a word identically.
FIELD EXPERIMENT 03◎
Follow the context.
burrows ← mole → underground
A conceptual illustration, not measured attention weights. Attention is one part of a larger network.
Embeddings, attention and feed-forward layers work together through repeated stages. In a typical text-generating model, each position can use earlier context, but not future text that has yet to be generated.
Context changes the representation.
04 / THE PREDICTION ENGINE
One token. Then another.
A language model produces a distribution over possible next tokens. The system selects a token, adds it to the context, and repeats.
It needn’t always pick the highest-probability option. Sampling settings can make some alternatives more or less likely. That is one reason the same prompt can produce different answers.
FIELD EXPERIMENT 04↻
Try the next step.
Below the surface lies a …
world55%
network30%
mystery15%
Invented probabilities for this demonstration; whole words stand in for tokens.
A plausible continuation is not the same as a verified fact. The process can produce useful explanations, code and reasoning—and confident errors. Understanding the mechanism helps us ask what else is needed to trust an output.
Likely does not mean true.
05 / THE SUBMERGED MASS
What went into the ice?
The bulk of the iceberg, hidden underwater, represents the vast dataset on which the LLM is trained.
Training adjusts the model’s numerical parameters using examples. The trained model is not simply a searchable copy of the dataset, though models can memorise and reproduce some training material.
Books, websites, code and other material can contribute to training. Which sources are included, excluded, repeated or licensed matters. So do the people whose language and experiences are missing.
FIELD EXPERIMENT 05⌕
Inspect a sample.
The ocean’s “sharks” stand for bias, misinformation, privacy violations and other harms. Filtering data can help, but it cannot make these questions disappear.
Data choices become model consequences.
06 / THE HUMAN LAYER
Someone shapes the snowman.
A raw language model does not automatically behave like a helpful assistant. Further training shapes how it responds.
One approach is reinforcement learning from human feedback, or RLHF. People compare responses; their preferences can train a reward model that helps guide further optimisation. Other post-training approaches also exist.
FIELD EXPERIMENT 06✦
Which response would you prefer?
“Can you help me understand this assignment?”
People write, label, filter and evaluate material throughout development. Their working conditions, values and disagreements are part of the system, even when the polished interface hides them.
Human judgement is inside the machine.
07 / THE IMAGE LAB
Not everything starts with words.
Many image-generation systems use diffusion: learning to reverse a process that adds noise to images.
At generation time, a trained system can start with noise and progressively denoise it, guided by a prompt. This is a different process from generating text token by token; not every image model uses the same architecture.
FIELD EXPERIMENT 07▧
Bring the iceberg into view.
A visual analogy using a prepared image, not a real diffusion model.
Different outputs can involve different machinery.
EXPEDITION COMPLETE
The surface looks different now.
The chatbot. The model. The data. The people. You can’t see them all from the text box.
“I’m deliberately avoiding any kind of analogy that represents the AI as magical, mythical, human, or godlike—we’ve seen enough of them.”Leon Furze · The AI Iceberg
Knowing how these systems work is the beginning of critical AI literacy. So is asking who builds them, whose work they use, what they can do—and who gets to decide.
