The Map: Where It's Strong, Where It Isn't

Ask ten people whether AI is good at their job and you'll get contradictory answers from people doing the same work. One automated half their week; the other tried, got burned, and stopped.

They're not disagreeing about the model. They're standing at different points on a very uneven edge — and knowing the shape of that edge is the difference between the two outcomes.

Read everything, seen nothing

Here's the profile in one line.

It has read everything and seen nothing.

Imagine a colleague who has read essentially all the text ever written — every textbook, every manual, every forum argument, every style of writing — and who has never once been in a room. They have no idea what your product looks like, who your customer is, what was decided at Tuesday's meeting, what your last release broke, or how your boss reacts to bad news. They answer instantly, in perfect prose, and they never say "I don't know."

Everything in the map below follows from that sentence. Where the work needs breadth of reading, form, or transformation, this colleague is extraordinary. Where it needs seeing — your specifics, the current state of the world, what actually happened — the same colleague confidently improvises.

The jagged frontier

In 2023, Fabrizio Dell'Acqua and colleagues at Harvard ran the field experiment on this with hundreds of Boston Consulting Group consultants. On tasks inside the model's capability, consultants using GPT-4 completed significantly more work, faster, at higher quality. On a task designed to fall just outside it — where the answer required combining data in a way that looked similar but wasn't — those using the model were substantially more likely to get it wrong than those working without it.

The authors named the shape: a jagged technological frontier. Not a smooth line with "easy" on one side and "hard" on the other, but a ragged edge where two tasks that feel equally difficult to you sit on opposite sides. The model breezes through a task you find hard and fails one you find trivial, with no change in tone to warn you.

That's why "is AI good at X?" is the wrong question. The useful question is always about a specific task, and the map is what lets you answer it in five seconds.

Where it's strong

Note the pattern in the why — it's the same one each time.

Transforming text you supplied. Translating, rewriting, changing format, summarising, extracting structure, turning notes into prose or prose into a table. Everything needed is already on the desk; the job is transformation, not knowledge. This is the single most reliable category of AI work, and it's underused because it's unglamorous.

Explaining what's been explained a thousand times. Deeply worn terrain — and unlike a textbook, it adapts to you: your level, your analogy, your specific confusion, as many times as you want.

The first draft. A weak draft beats a blank page, because editing is a different, easier cognitive mode than starting. The value isn't the text; it's escaping the empty screen.

Breadth and coverage. Twenty options, a checklist, "what am I forgetting", "how do people usually solve this". Where the value is in not missing something rather than in depth, having read everything is exactly the right qualification.

Formal, patterned work. Boilerplate code, regular expressions, SQL, standard documents, structured emails. The form is heavily worn and the result is usually verifiable — the two conditions that matter most.

Language itself. Register, tone, translation, the difference between "we regret" and "we're sorry". This is the closest thing it has to a native skill.

Where it's weak

Anything recent, and anything yours. Past the cutoff, inside your company, about your customers, about specific non-famous people. Not on the desk means invented.

Precision with numbers. It can't see digits properly and it computes in prose. Any arithmetic that matters goes through code.

Narrow expertise. The thinner the terrain, the more the confident output is a reconstruction of what such an answer usually looks like. Specialists notice this immediately; non-specialists cannot.

Taste calibrated to your audience. Its default output is the median — competent, unobjectionable, and therefore unremarkable. Median is fine for a summary and fatal for anything meant to stand out. You supply the deviation; it can't know which deviation is right for your readers.

Continuity. It doesn't remember. Everything that persists across sessions persists because you or a tool put it back.

Consequences. It has no stake in the outcome, no access to what happens after, and no capacity to be accountable. That's not a technical limitation to be fixed later — it's the whole reason a person stays in the loop.

The dangerous quadrant

The map's most valuable region is not "weak". It's weak while looking strong: thin terrain plus a highly patterned output format.

Citations and references. Statistics and precise figures. Legal and medical specifics. API parameters and library versions. "Best practice" in a niche field. Assessments of your particular situation. In every one of these, the shape of the answer is extremely well worn — the model knows exactly what a citation, a dosage table, a legal argument or a config block looks like — while the content may be nearly untrodden ground.

That's the combination that produced the fabricated court cases in article 4. Confidence is highest exactly where reliability is lowest, because the form is what confidence is made of.

Three questions before you delegate

The map compresses to a five-second check:

1. Is the material on the desk, or does this need knowledge from outside? On the desk → strong. From memory → suspicious. From memory about something recent or specific → don't.
2. Can the result be checked by something outside the model? Code that runs, a document to quote, a number you can recompute → strong. Nothing to check against → your judgement is the only check.
3. What does an error cost? Cheap → don't verify, just iterate. Expensive → verify every specific, every time.

And one rule that sits above all three, which the rest of this stage builds on: don't delegate what you couldn't evaluate yourself. Not "couldn't produce" — plenty of useful work is worth delegating precisely because producing it is slow. But if you can't tell a good result from a bad one in this domain, you haven't automated anything. You've just moved the risk somewhere you can't see it.

In practice

Ask about the task, not about "AI". "Is it good at this specific thing?" has an answer; the general question doesn't.

Prefer transformation to recall. Whenever you can put the source on the desk instead of asking it to remember, do.

Treat the median as a starting point, never as the target. Its instinct is the average of everything written; your value is the deviation.

Be most suspicious where the format is most polished. Citations, figures, specs, legal and medical detail.

Run the three questions before handing something over. On the desk? Checkable? Cost of error?

Check yourself

Close the article and answer in your own words:

1. What does "read everything and seen nothing" predict about strengths and weaknesses?
2. What is the jagged frontier, and what did the BCG experiment find on the task that lay outside it?
3. Why is transforming supplied text the most reliable category of work?
4. Why is "its default answer is the median" a weakness rather than a neutral fact?
5. What makes the dangerous quadrant dangerous — name the two conditions that combine there.
6. What are the three questions to run before delegating, and what's the rule above all three?

In short

• The model has read everything and seen nothing: strong on breadth, form and transformation; improvising on your specifics, the present, and consequences.
• The capability edge is jagged, not linear (Dell'Acqua et al., 2023): consultants gained substantially inside the frontier and did measurably worse than unaided colleagues on a task just outside it.
• Strong: transforming supplied text, explaining well-covered material, first drafts, coverage and checklists, patterned and verifiable work, language itself.
• Weak: anything recent or yours, precise numbers, narrow expertise, taste for your audience, continuity, and anything requiring accountability.
• The dangerous quadrant is thin knowledge plus polished format — citations, statistics, legal and medical detail, API specifics. Confidence peaks exactly where reliability bottoms out.
• Before delegating: is it on the desk, can it be checked externally, what does an error cost. And never delegate what you couldn't evaluate.
• It has read everything and seen nothing.