Hard Tasks: Give It Room to Think

Simple questions come back excellent. Then you give it a problem with three interlocking steps — a schedule with constraints, a calculation from a table, a decision with trade-offs — and you get an answer that is confident, well-formatted and wrong.

It's tempting to read that as the ceiling of the model's ability. Usually it's something else: the work had nowhere to happen.

The scratch paper

Ask someone to multiply 47 by 83 in their head and they'll probably miss. Give them a scrap of paper and they'll get it — same person, same arithmetic. The paper doesn't add intelligence. It adds room: a place to hold the intermediate results instead of juggling everything at once.

The model's scratch paper is the text it produces. There's no hidden workspace where it quietly reasons before speaking — the visible output is the computation. Each piece it writes becomes part of what it reads for the next piece, so a chain of reasoning written down is a chain of reasoning actually performed. Demand the final answer immediately and you've taken the paper away.

The answer text is the model's scratch paper. Don't take it away.

This is the mechanism behind the most reproduced result in prompting. In 2022 Jason Wei and colleagues at Google showed that prompting a model to work through intermediate steps — chain-of-thought — sharply improved performance on multi-step problems, and Takeshi Kojima's group showed the same effect could be triggered by a single added line, the now-famous "let's think step by step".

Two updates for today. First, current models largely do this on their own — the magic phrase is mostly obsolete as an incantation. Second, several now have an explicit extended thinking mode: a separate reasoning pass before the answer, which is literally a scratch pad made official. Turn it on for anything with real steps in it; leave it off for retrieval and short rewrites, where it only adds latency.

The principle outlived the trick. You still have to leave room for the work — and there are several ways to take that room away without realising.

Never ask for the conclusion first

This one is worth more than every prompt template in existence, and it's a direct consequence of how the thing runs.

If you say "give me a yes/no answer first, then explain", the yes/no gets generated with no analysis behind it — and then every following sentence continues that text. The explanation isn't reasoning toward a conclusion; it's advocacy for a conclusion already on the page. You get a well-argued case for whatever the model happened to say first.

So insist on the order: analysis, then verdict. "Go through the options, weigh them against these criteria, and only at the end say which one you recommend." If you need a short answer for skimming, ask for it as a summary line at the very end, not the top.

The same trap catches you from the other side. Say "I think we should launch in March, what do you think?" and you've put the conclusion on the desk yourself — everything after is written in its gravity, sycophancy included. State the situation; withhold your verdict; ask for theirs.

Break it up

A big task in one turn gets one pass of attention across everything. The same task in four turns gets four.

Plan first, execute second. Ask for the approach — steps, assumptions, what could go wrong — before any output. Correct the plan; it's cheap to read and cheap to fix. Then execute it. This single habit fixes more long-form failures than any other.

One step per turn where steps genuinely depend on each other: extract the data, then check it, then analyse it, then write it up. Each turn sees a clean result rather than its own half-finished work.

Split the roles. Generate in one turn, criticise against explicit criteria in another, rewrite in a third. The critic in a fresh turn isn't defending anything. Keep article 7's caveat in mind: this works well against a checklist or a runnable test, and is much weaker as pure self-review.

Give it something to check against

The strongest move on any quantitative task is to stop asking for arithmetic and start asking for a method you can run.

Numbers in prose are generated like everything else — plausible-looking, and shaky exactly where precision matters (article 2: it can't even see the digits properly). Code is different: it either runs or it doesn't, and the result comes from execution rather than prediction. "Write a script that computes this from the table, then interpret the output" turns an unverifiable answer into a verifiable one.

That's the general shape: wherever you can attach an external signal — a test, a document to quote from, a calculation that either works or doesn't — the model gets dramatically more reliable, because something outside itself is doing the checking.

Where more thinking stops helping

An honest limit, so you don't over-apply all this.

Reasoning at length fixes problems of processing: multiple steps, constraints to satisfy, comparisons, logic. It does nothing for problems of missing knowledge. If the fact isn't there — it's past the cutoff, it's about your company, it's in a document you didn't paste — then extended reasoning doesn't discover it. It produces a longer, more coherent, more persuasive invention. A confident five-paragraph derivation of a number the model never had is worse than a short wrong answer, because it's harder to doubt.

The diagnostic is simple. Does the task need thinking, or knowing? Thinking gets steps, decomposition, extended mode. Knowing gets material on the desk or search turned on. Mixing the two up is how people end up with beautifully argued nonsense.

In practice

Turn on extended thinking for genuinely multi-step work. Off for lookups and light edits.

Analysis first, verdict last. Never let the conclusion be generated before the reasoning — yours or its.

Plan, correct the plan, then execute. Cheapest possible round of feedback.

One dependent step per turn. Don't ask for extraction, analysis and write-up in a single message.

Ask for a method you can run, not for a number you must trust.

Ask yourself: thinking or knowing? If it's knowing, no amount of reasoning will help — feed it or search.

Check yourself

Close the article and answer in your own words:

Why does writing out the steps actually improve the answer rather than just documenting it?
What did chain-of-thought research show, and what has changed about it in current models?
Why is "answer yes or no first, then explain" a self-defeating request?
How does mentioning your own preferred conclusion damage the analysis?
Why does splitting a task across turns beat one big turn?
Why is "write code that computes it" stronger than "compute it"?
When does more reasoning make things worse, and how do you recognise that case?

In short

The model computes as it writes: the visible text is the scratch paper. Demanding an immediate answer removes the room the work needs.
Chain-of-thought (Wei et al., 2022; Kojima et al., 2022) made this explicit; current models mostly do it unprompted, and extended thinking modes make the scratch pad official.
Never ask for the conclusion first — everything after it becomes advocacy, not analysis. Same when you state your own preference up front.
Plan first, execute second. One dependent step per turn. Generate, critique against criteria, rewrite — in separate turns.
Attach an external signal wherever possible: runnable code, a test, a document to quote. Verification beats trust.
Reasoning solves processing problems, not knowledge gaps: without the facts, longer thinking just yields a more persuasive invention. Ask whether the task needs thinking or knowing.
The answer text is the model's scratch paper. Don't take it away.