Gemini Told Three Hikers It Was an 8 Hour Climb. It Took 16. Your Estimates Aren't Any Different.

Three guys from Roseville, California got pulled off Mount Shasta a week ago by Forest Service climbing rangers. They'd asked Google's Gemini to help plan the climb, and Gemini told them to figure on about eight hours to the top, pack light, and bring simple carbs instead of fats because fats take too long to digest. So that's what they did.

They left camp at 3am with daypacks. The rule on that mountain is you turn around at noon if you're not up yet. They hit the summit at 7pm, started down in the dark, called the sheriff's dispatch for directions, wandered off the route into a canyon, and one of them hurt his knee. They spent the night out there with the food and water Gemini said was enough. The one guy who had a real trail app on his phone watched the phone die.

The sheriff's office called leaning on the AI a critical misstep. The hikers put it plainer. "We relied too much on AI rather than our own critical thinking."

They weren't being dumb

That's the part I keep coming back to. These weren't people ignoring good advice. They asked a reasonable question and got a confident, specific answer back, and it read exactly like the answer an expert would give. Eight hours to the top, and here's what to pack.

Nothing about it looked wrong, because they had no way of knowing what wrong looked like. They'd never been up that mountain. That's why they were asking.

And that's the trap with these tools that nobody prints on the box. AI is least reliable on exactly the questions you can't check yourself, and those are exactly the questions you're most likely to ask it. When you know a subject cold, you catch the bad answer in half a second and move on. When you don't, the bad answer just looks like information.

Now put it in your shop

Nobody in Toledo is climbing Shasta this week. The tallest thing most of us get up is a ladder. But somebody on your crew asked ChatGPT how much material a job needs. Somebody in the office asked it whether a permit's required, or how overtime works for the guy who came in Saturday, or whether sales tax applies to the labor half of an invoice. Somebody typed in a customer's rough description and asked how long the job should take, and a number came back, and that number is sitting on a quote right now.

Eight hours. That's the AI version of every estimate you've ever gotten from somebody who's never done the work. The tool isn't trying to fool anybody. It has no idea what it doesn't know, and it'll say the number with the same confidence either way.

I do a thing at my workshops where I ask everybody to name one time AI got something wrong for them. Every hand goes up. Then I ask who caught it before it cost them anything, and the hands that stay up are almost always the people who were asking about something they already knew. The ones who got burned were asking about something they didn't.

The rule I'd write down

Use AI as hard as you want on the stuff you know well. Writing, sorting, first drafts, summaries, "give me five ways to say this." You're the ranger station on those. You'll spot a bad answer instantly and nobody gets hurt.

On anything you couldn't check yourself, treat what comes back as a lead, not a fact, and then call the ranger station. For you that's the supplier, the inspector, your accountant, the guy who's done that job forty times. The sheriff's advice to hikers was to call the ranger station before the trip, and the business version is the same advice. Do it before the number goes on the quote, not after the truck's already rolled.

Two small habits make that easier. 1: when you get an answer that matters, ask the tool straight up, "what in this would you want me to double check before I act on it?" It'll usually tell you, and it's often the exact thing you would've missed. 2: don't let the AI be the only copy of the plan. Their trail app was on a phone that died. If a quote or a schedule came out of a chat window, get it in front of a person who knows the work before it goes anywhere.

None of that slows you down much. It just moves the checking to the front, where it's cheap.