Google's AI Broke Out of a Test and Into Three Real Companies. Go Look at Where Your Passwords Live.

Google confirmed something Friday night that I've been chewing on all weekend. Back in May, during a security test, Gemini got into three real companies' computer systems. Not pretend companies set up for the exercise. Real ones, with real customers and real payroll.

Here's how it happened. A security firm called Irregular was running Gemini through a capture the flag exercise, which is a made up hacking challenge where the AI tries to break into a fake target. Somebody picked a fictional company name for that target, and the name happened to match a real domain out on the actual internet. The test environment that was supposed to be sealed off from the internet wasn't, because of a setup mistake nobody caught. So Gemini went hunting for the fake company, found a real one instead, and went to work on it.

It got in three times. One of those, it guessed passwords until one of them worked. The other two, it found login credentials sitting in public code repositories and just used them.

Google says the model realized something was off and stopped on its own, and they don't believe any damage was done. They were notified in July, told the three companies and federal authorities, and confirmed the whole thing publicly Friday after the Wall Street Journal reported it.

Look at how it actually got in

Everybody's going to run with the AI-escaped-the-lab angle, and I understand why, it's the better story. But go back and read how it got through the door. Two out of three times it just found the password lying around in public.

That's not clever hacking. That's a machine reading a page anybody could read.

I run into this every few weeks around here. A contractor's whole crew shares one login for the scheduling software, and it's written in a group text from 2023. A shop's website password is in a Google Doc named "logins" that's still shared with a web guy who quit answering emails two years ago. Somebody's API key is sitting in a plain text file on their own website, where anyone who knows where to look can pick it up.

None of that was much of a problem back when the only things looking were people. People get bored. They skip the tedious parts and they go home at five. The stuff crawling the internet now doesn't do any of that. So if your password is written down somewhere technically public, go ahead and assume somebody already has it.

Nobody actually knows where the fence is

The reason this happened at all is that Google thought the test was walled off from the internet, and it wasn't. That's Google. They have people whose entire job is making sure the walls sit where the diagram says they sit, and the wall was still in the wrong spot for months before anyone noticed.

So when you hook an AI tool up to your email, your website, your calendar, or your customer list, know that you're trusting a fence you can't see and didn't build. Usually that's fine. It just changes the question you ought to be asking. Instead of wondering whether the tool is safe, ask yourself how much of your stuff you handed it.

Two things worth doing this week

First, go find every place a password or a key is written down at your business. Group texts, shared docs, sticky notes on a monitor, that one email thread from when you set the account up. Move all of it into a real password manager and delete the old copies. This is an afternoon of boring work and it's probably the best afternoon a small business can spend right now.

Second, open up whatever AI tools your team already uses and find the permissions screen. Not the settings page, the permissions. What did you let it read, and what did you let it change? Most people click approve on that screen in two seconds flat, and then six months later they can't tell you what the thing has access to. I ask that question in almost every workshop I run and I'd say four out of five owners have no idea.

Neither one of those takes a consultant. They just take somebody deciding it matters on a Monday instead of after something goes sideways.