Nate B. Jones wrote something last week that I've been chewing on.

He described research (from Epoch and METR) where the same Claude model rebuilt a roughly 16,000-line scientific toolkit from its behavior alone, without seeing the code. Then, asked to customize a smaller library, it produced something unusable.

Same model. Wildly different results.

His explanation: AI improves fastest where results are cheap to check, like proofs and tests. It improves slowest where feedback is slow, like judgment. He calls it narrow superintelligence. I think of it as smarter and spikier.

This lined up with two conversations from my own week.

A Stanford researcher who studies AI in investing told me senior investors use AI least. Part of the reason: early hallucinations made them distrust it. One bad answer, and the tool goes in the drawer.

And with a team at a firm that invests in outside funds, I'd described an AI-built model as about 90 to 95% reliable. Fine for a low-stakes request. Never for a number that matters.

Put those together and a useful question falls out.

Does this task have an answer key?

A reconciliation has one. A calendar invite has one. A formula has one. You can check the output and know in a minute if it's right.

A judgment call has no answer key. Reasonable people would argue about it.

Headline breakthroughs don't mean the tool is reliable on your Tuesday tasks. One bad hallucination doesn't mean it's useless either. Sorting by answer key gives skeptical senior people a defensible place to start, and that's usually the group I most want to get moving.

Here's the exercise:

1. List ten tasks you or your team did this week. Real ones, from the calendar.

2. Mark each one. "Answer key" if there's a right answer you could check. "Judgment" if reasonable people would argue.

3. Hand AI the answer-key tasks first. Check the first three outputs against the key. That's how trust gets built, one verified answer at a time.

4. Use AI differently on judgment tasks. Let it lay out the ground, and keep the call yourself. Try: "Lay out the options and the strongest case for each. Don't recommend."

I'll admit I find this freeing. I spent a long time trying to decide whether AI was "good enough." That question has no answer. "Good enough for which task?" does.

How many of your ten came back as answer key? My guess is more than you'd expect.

Alex