What a Language Model Actually Is

You ask a question and get a careful, well-structured answer that clearly grasped what you meant. Ten minutes later the same model recommends a book that does not exist — with an author, a year and a publisher, all invented on the spot.

It's tempting to file the first as intelligence and the second as a bug. They are the same machine doing the same thing. Until you know what that thing is, you'll keep swinging between expecting too much of it and dismissing it for the wrong reasons.

What it literally does

A language model continues text. That's the whole operation.

You give it a beginning; it produces what most plausibly comes next, then reads its own output as part of the beginning and produces the next piece, and again, until the answer is finished. "The capital of France is" → "Paris". "Salt and" → "pepper". A three-page answer is built the same way as those two words, just many more times over.

That is genuinely all. There is no database it queries, no lookup table of facts, no small copy of the internet inside. Nothing is retrieved. Every word you read was generated, including the ones that happen to be true.

The obvious objection: guessing the next word sounds like a party trick. Where does the reasoning come from?

Why a party trick turns into a conversation

Because prediction, done well enough, quietly requires everything else.

To predict a chess player's next move you have to understand chess. To finish "the glass tipped over and Mum ran for the—" you need to know that spilled liquid calls for a cloth and not a hammer. To continue a proof you need the proof to actually work; to continue a joke you need to know why it's funny; to continue an argument you need to track what was already claimed.

Prediction is the exercise, not the goal. It's the pressure that forces structure to form. Trained on enough text, a system that only ever learns "what comes next" ends up carrying grammar, facts, styles, chains of reasoning and a rough working model of how the world behaves — not because anyone installed them, but because there's no other way to guess well.

Keep both halves of that in mind. The mechanism is simple and slightly dumb. What the mechanism grew is not.

The landscape

Here's the picture worth carrying through the rest of this sphere.

Training didn't file texts away in a cabinet. It shaped a terrain. Imagine an enormous landscape of hills and valleys, worn in by millions of pages the way paths get worn into a hillside by millions of footsteps. Wherever text went a certain way again and again, a groove formed. Where it went a hundred different ways, the ground stayed flat and open.

Now your prompt sets a ball down somewhere on that terrain, and the answer is the path the ball rolls.

The model doesn't retrieve an answer — it rolls to one.

Four things follow, and they explain most of what beginners find baffling:

A confident answer is a deep groove, not a checked fact. Well-trodden ground means "this shape of text appeared constantly," not "this is true." Where the topic is common and settled, plausible and true coincide almost perfectly. Where it's rare, recent or specific, the ball still rolls somewhere — smoothly, fluently, and possibly nowhere real. That's the whole story of hallucination, and the fourth article of this stage.

Wording is not cosmetic. Move the starting point twenty metres and the ball ends up in a different valley. "Write about remote work" and "you're a sceptical ops manager; list the three reasons remote work fails in teams under ten people" don't differ in politeness — they start on different slopes.

Context reshapes the terrain locally. Paste your own document, your notes, three examples of the style you want, and you've raised ridges around the ball that keep it in your part of the map. This is why feeding material beats any amount of instruction, and why "use my data" works while "please be accurate" mostly doesn't.

There isn't one correct path. Ask the same thing twice and you may get two different routes, both reasonable. That's not the model being unreliable; it's the terrain being wide. When the answer matters, asking for three variants is cheap and tells you how flat the ground is there.

The landscape is a metaphor, and like every metaphor it's a simplification — inside, the thing is a vast pile of numbers adjusted during training, and there is no literal hillside. But it predicts the model's behaviour better than "a very smart search engine," which predicts it almost entirely wrong.

Does it understand anything?

Honest answer: this is genuinely argued about, and nobody hands you a settled verdict.

What can be said without hype is that "it just predicts words" and "it understands like a person" are both wrong in useful ways. The system behaves as though it holds a working model of the world — it tracks who did what to whom, catches a contradiction three paragraphs back, transfers an idea to a domain it wasn't stated in. And at the same time it has no access to whether it knows something. It can't feel the difference between recalling and inventing, because from the inside both are the same operation: continue plausibly.

That last sentence is the one to keep. It's the root of almost every practical rule in this sphere.

Where the "assistant" comes from

One more thing the raw picture doesn't explain. A system trained only to continue text would just as happily continue your question with five more questions — that's what text often does.

The chat behaviour is a second layer: after the reading phase, the model was additionally trained to act as a helpful interlocutor — to answer rather than continue, to follow instructions, to refuse some things. Its "character" comes from that stage, not from the books. The third article of this stage takes it apart, because several of the model's oddest habits — including its eagerness to agree with you — come from there and nowhere else.

In practice

Ask "what would this look like as text?" The model responds to the shape of what you give it. A vague request sits on flat ground and rolls to the most average, most-written-about answer on the internet. Specifics, constraints and examples are how you move the starting point.

Put material in, don't describe it. Ten lines of your actual text beat two paragraphs describing your style. You're shaping terrain, not giving orders.

Treat fluency as style, not evidence. The same smooth, assured tone comes out whether the ground under the answer is deep or empty. Fluency tells you nothing about accuracy — a point the fourth article turns into a working method.

When it matters, ask for variants. "Give me three different approaches, with the weakness of each." One path tells you where the ball rolled; three tell you how wide the valley is.

Check yourself

Close the article and answer in your own words — that alone will fix the material better than rereading it:

What does a language model literally do, and what does it not do at all?
Why does "just predicting the next word" end up producing reasoning rather than gibberish?
In the landscape metaphor, what corresponds to training, to your prompt, and to the answer?
Why does a confident tone say nothing about whether the answer is right?
Why does pasting in your own material work better than describing what you want?
Where does the model's "assistant" behaviour come from, if not from the texts it read?

In short

A language model continues text: it produces the most plausible next piece, then the next, and so on. Nothing is looked up; everything is generated, including the true parts.
Prediction is the exercise, not the goal. To predict well you have to absorb grammar, facts, reasoning and a working model of the world — which is exactly what happens.
Training doesn't store texts, it shapes a landscape. Your prompt is where the ball is placed; the answer is where it rolls.
Well-trodden ground means "written about constantly," not "true." On common topics the two coincide; on rare or recent ones they come apart.
Wording moves the starting point, and context reshapes the terrain — which is why examples and pasted material outperform instructions.
The model has no access to whether it knows something: recalling and inventing are the same operation from the inside.
The model doesn't retrieve an answer — it rolls to one.