How to use AI to inform a decision


Most of us ask AI to do the one thing it's worst at.

We ask it to decide. "Which vendor should I pick?" "Is this a good hire?" "Should we raise prices?" And it answers instantly, confidently, in a clean paragraph. The problem is that the confident paragraph is often the shallow one.

Here's how to fix that, and it takes about thirty extra seconds.

First, the proof it matters. Researchers gave a top AI model a farmer's question: plant apples or grapefruit next year? The model saw that grapefruit prices were high and said grapefruit. Wrong answer. Prices were only high because bad weather had wrecked the supply the year before. It mistook a one-off event for a good bet. It saw the loudest number in the room and repeated it.

Then they changed how they asked. Instead of "decide," they walked it through four steps first. The same model got it right, and across the study this approach was up to 40% more accurate.

You can run those same four steps on any decision that actually matters. Before you ask your AI to conclude anything, ask it to do this:

  1. Name what we don't know. "List the factors that could change the outcome here." (For a vendor: their pricing next year, their stability, switching costs.)
  2. Put odds on them. "For each one, give me a rough probability of the likely scenarios." This forces it to think in ranges, not in one confident future.
  3. Spell out what each outcome is worth. "What's the actual cost or payoff of each scenario to me?"
  4. Then, and only then, weigh it and choose.

If you want a shortcut, paste this in: "Before you recommend anything, list what's uncertain, estimate how likely each scenario is, and tell me what each one would cost or earn me. Then make your call and show your reasoning."

Watch what happens. The mediocre answer turns into a genuinely useful one in front of you.

Now, this might sound like the old "show your work" trick. It's a step further. You're not just asking it to reason. You're handing it the accounting method and making it put real numbers on the guesses, which is the part it tends to skip on its own.

Why does that change the answer so much? Because of how these models actually work. An AI doesn't think and then talk. It thinks by talking. It was trained to predict the next most plausible word, so when you ask it to "just decide," it reaches for whatever sounds most natural, which means whatever is loudest in front of it. There's no back room where it quietly works the problem out first. The reasoning has to happen out loud. When you make it write out the factors and the odds, those words become its scratchpad, and it builds the real answer on top of them. Skip that, and you get the guess.

One more thing worth knowing: these models are tuned to sound decisive and helpful. So the snap answer isn't just sometimes wrong. It's been trained to sound sure of itself while it's wrong. Confidence is not the signal you think it is.

The intelligence is already in there. Structure is what lets it out.

So here's the shift for your next big decision: stop asking your AI to conclude. Start asking it to think.

Alex

PS If you're interested in the underlying research, it's here.

Alex Talks AI

As an AI Coach, Advisor, and Agent Builder, I help organizations and business leaders harness the power of artificial intelligence to boost productivity and streamline operations. I enable organizations to navigate the transformative landscape of AI, educating teams, identifying operational and strategic opportunities with AI and creating a framework for safe and transparent use of data in the organization.

Read more from Alex Talks AI

Next week we're going to graduate from AI slop creator to real builder. I am going to show you how easy it is. We're going to build a To Do list tracker that really works *for you.* Why? Two reasons: One - Because I've downloaded, abandoned, and felt quietly guilty about more task apps than I can count. Todoist, Things, Notion, a legal pad, the Notes app, a whiteboard I can't see from my desk. None of them were wrong. They just weren't mine; each one was someone else's idea of how I should...

You've heard the complaints about AI slop. The generic LinkedIn posts. The Twitter threads that all sound the same. The flood of content that exists because it's now almost free to make. Everyone's pointing at the public version of this problem, the stuff strangers scroll past on their feeds. Nobody's talking about the slop we make for ourselves. Here's what I mean. The same thing that makes public AI slop so easy to produce (low effort, instant output, a little hit of delight every time) is...

Yesterday a portfolio manager I work with sent me a link. "Free trial," she wrote. "Tell me what you think. Is this actually useful, or am I paying for ChatGPT in a trench coat?" She'd signed up for a new investment research platform. Pick a public company and you get a bull case, bear case, list of risks, questions for management, a multi-year financial story, earnings review, and relevant podcasts. All sourced from the standard stuff: EDGAR, earnings transcripts, proxies, podcasts. Her...