81% of Big Companies Had an Outage From AI-Written Code. Check What AI Built for You.
A debugging company called Undo put out a survey this week, and LeadDev wrote it up on Monday. They asked 300 senior engineering leaders at large US and UK companies about AI coding tools. 81% said AI-written code had caused a production incident or outage in the last six months. 93% said an AI hallucination sent their team after the wrong cause of a problem.
Those are big companies with whole departments whose job is catching this stuff. If it's biting them, it can bite the freelancer who built your website last month, or the guy at your shop who "set up a few automations" with ChatGPT on a slow Friday.
Where this shows up in a small business
Nobody I work with is running a data center out of the back office, but most of them have code running somewhere. The quote form on the website. The Zapier flow that sends new leads into the CRM. A booking page, a plugin somebody tweaked, a text message that fires when an invoice goes out. More and more of that gets written by AI now, sometimes by a contractor who never mentions it, sometimes by you at 10pm.
It usually works fine on day one. The survey says trouble comes after that. 35% of the AI-written code in those companies went live before the team fully understood it, and 79% said their release cycles were no faster than before they started using the tools. The engineers were also spending nearly 17 hours a week debugging against under 10 hours actually writing code.
Your version of an outage is quieter
You won't get a status page going red. What you get is a contact form that stopped sending emails three weeks ago and nobody noticed, because nothing looked broken. Leads went somewhere, just not to you. Or an automation that texts the wrong customer, or the same customer four times. By the time you hear about it, it's been wrong for a while.
The twenty minute check
Do this once this week and put a reminder on your calendar to repeat it monthly. 1: Write down everything on your website and in your tools that runs on its own, forms, automations, booking, payment links, text alerts. 2: Next to each one, put who built it and ask them straight out whether AI wrote part of it. People will tell you if you ask. 3: Submit a fake lead through every form and watch it land where it's supposed to, in your inbox and in your CRM both. 4: Make sure someone can get you back to the previous version if a change breaks something, because "I'm not sure we have a copy" is a bad thing to hear on a Tuesday.
The Undo report says to judge AI by how often things break and how long they take to fix, not by how much got built. I think that's right for a five person company too. Getting a form built in ten minutes is great. Finding out a month later that it never worked is not.
I'm not saying stop using AI to build things. I do it all the time. I just test what it hands me before a customer finds the problem for me.
I run AI workshops and one-on-one AI consultations for companies around Toledo, Northwest Ohio, and Southeast Michigan. If you'd like a second set of eyes on what AI has built into your website or your tools, or you want your team walked through how to test it, email me and we'll set something up.
Email Jayson