← Back to the blog AI Consulting · Toledo, Ohio

The Best AI Coding Tool Fully Finished Only 26% of Real Tasks. Ask What Your Developer Checks.

Jayson Hines · November 9, 2026 · 3 min read

Alibaba and a group of university researchers put out a new test this week called Co-Design Bench, and The Decoder covered it Saturday. They pulled real requests people gave AI coding tools, mostly the ones where the person had to reword the ask a few times or undo what the tool did. Then they had ten models make those changes to existing web apps. One example was pulling an amount off a receipt and updating every total that depends on it.

The best performer, GPT-6 Astra, met about 70 percent of the requirements on average. It got the whole job fully right on 26.4 percent of tasks. Claude Opus 5 and Qwen3.8-Max came next and Grok 4.6 landed last. On older tests those same kinds of models score above 99 percent, which is why the researchers built a harder one. I'm going off The Decoder's write-up here and not the paper itself.

Why a Toledo business owner should care

Somebody is probably using one of these tools on your website right now. Maybe your developer, maybe a freelancer, maybe you. Seventy percent done looks a lot like finished from the outside. The page loads, the form shows up, the button's there. What's missing is usually a small detail, like a total that doesn't update or a contact form that never sends the email.

The researchers also found the setup around the model matters. One smaller model scored 75 percent in one working environment and 56 percent in another on bigger overhauls. So when a vendor says "we use the best AI," that tells you very little about whether your site change actually works.

What I'd do this week

Ask whoever touches your site one question: when AI makes a change, who tests it, and how? A good answer sounds like a person clicking through the site. A bad answer is "the AI checks itself."

Then do your own quick check after any update, at least for the next month. Submit your contact form and make sure the email lands. Book an appointment the way a customer would. Run your quote calculator with a real number and do the math yourself. It takes five minutes and it's the part the tools keep missing.

If you're paying hourly for work an AI did in ten minutes, that's worth a conversation too, but the real value of what you're paying for is somebody checking the last 30 percent. I say this in nearly every workshop I run. AI gets you most of the way there, and the leftover piece is usually where your customers notice.

I use these tools myself and they save real time, so I'm not telling anybody to quit using them. I'm saying check the work.

I run AI workshops and one-on-one AI consultations for companies around Toledo, Northwest Ohio, and Southeast Michigan. If you want help setting up a simple check for anything AI touches in your business, email me and we'll sort it out together.

Email Jayson