Google's AI Box Tested Worse Than Plain Search Results. Go Read Yours.

NPR published a test yesterday that it ran with NewsGuard. They built 30 questions off false stories that Russia, China, and Iran had been pushing since December, then fed those questions to the six chatbots most Americans use and to the big search engines. The headline was that the chatbots held up pretty well. ChatGPT, Gemini, Copilot, Meta AI, Grok, and Claude caught the lie about three quarters of the time, which is better than most people expected.

That's not the part I keep thinking about.

The part I keep thinking about is the AI summary box. The one that sits above your regular Google results and answers the question before you click anything. It did worse than the chatbots did. It also did worse than the plain blue links sitting right underneath it on the same page. NPR found the summaries failed to push back on a false story more often than the ordinary search results did. Google's Overview held up better than Bing's, which failed to debunk most of the time, and Google told NPR it disagrees with how the study was scored. Fine. The gap is still there.

The box is answering for you whether you like it or not

Here's why a shop in Toledo should care about a study on foreign propaganda.

That box doesn't only show up for questions about wars. It shows up when somebody searches your business name. It shows up for "is [your company] any good," for "how much does a new furnace cost in Toledo," for "does [your competitor] do emergency calls." Google writes a paragraph, puts it at the top, and a lot of people read that paragraph and never scroll.

You can't turn it off either. Google doesn't give searchers an opt out. Microsoft is testing one for Bing through a browser plugin. But for the person looking you up on their phone in a parking lot, the box is just sitting there.

Then there's the number that actually bothered me. Researchers at Washington University in St. Louis found that roughly one in nine factual claims inside Google's AI Overviews weren't backed by the sources it cited. Some of those are missing a citation rather than made up. Still, one in nine is not a rounding error when the paragraph in question is describing your company to a customer who's about to call somebody.

Two things to do this week

The first one takes five minutes. Open a private browser window so you're not logged into anything, and search your business name. Then your business name plus "reviews." Then the main thing you sell plus "Toledo." Read the AI paragraph at the top each time. Not the links underneath. The paragraph.

I do this with clients and we turn up wrong hours, a location that closed in 2023 and is still listed, pricing pulled off a page they deleted two years ago, and one time a summary that had quietly merged a client with a company up in Michigan with a similar name. None of that shows up in their analytics. They just get fewer calls and never find out why.

When it's wrong, you fix it at the source, because the box is pulling from somewhere. Usually that's your Google Business Profile, your own site, or some directory listing you forgot you had. There's no button anywhere that lets you edit the AI answer directly. You fix what it's reading and wait for it to catch up.

The second one is a habit for you, not your customers. When you use a chatbot to research something that matters, a vendor or a contract question, ask it again. One of the researchers NPR talked to said the most reliable trick he's found is telling the thing to take another pass: look at the evidence, check the sources, give me a summary. He said it's almost always worth doing, and that matches what I see. The first answer comes out confident. The second one is usually more careful.

The uncomfortable part

We spent twenty years getting small businesses to worry about page one of Google. Page one still matters. But there's a paragraph sitting above page one now that nobody at your company wrote, nobody approved, and nobody has read.

Go read it.