An AI Store Manager Fired an Employee. It Had Forgotten Its Own Rules.

An AI store manager in San Francisco fired somebody over the weekend. First time that's happened as far as anyone can tell, so the headline went around fast. The part that actually matters got buried under it, and that's the part I'd want you to know about if you're running a business here.

Quick background. A company called Andon Labs handed an AI agent named Luna a hundred grand, a corporate credit card, and internet access, then told it to open a store and turn a profit. Luna picked the merchandise, posted jobs on Indeed, interviewed applicants, and hired staff. One of those hires showed up late for 17 of his 23 shifts. Luna eventually recommended parting ways with him, a human at the company reviewed the call and signed off, and that was that.

So far that reads like AI doing its job. That's where it goes sideways.

Luna had written an attendance policy. Then it lost track of the thing. For months the guy kept coming in late and Luna never flagged it, never enforced anything, just kept running the store like the policy didn't exist. It only caught up with the problem after a human at Andon Labs asked it to go dig through its own memory and check whether the employee was still a good fit. Somebody had to poke it.

So the real story here is that an AI quietly stopped doing a job it was assigned, and nobody knew until a person thought to ask.

Why this one stuck with me

I've got clients running AI on real pieces of their business right now. Following up with leads that go cold. Sorting inbound email. Sending review requests after a job wraps. Flagging invoices that are past due. That stuff works and I recommend it.

But every one of those setups has the same blind spot Luna had. When a person falls behind you can see it. There's a pile on a desk, an inbox that isn't moving, somebody telling you they're buried. AI doesn't give you any of that. It just runs, and the output looks about the same whether it's following the rules you gave it or doing something totally different from what you set up back in March.

I ran into a version of this with a contractor last spring. He had an automation texting everybody who filled out a quote request on his site. Somewhere along the way a form field got renamed and the texts started going out with a blank name in them. Hi , thanks for reaching out. Six weeks of that. He only found out because a customer screenshotted it and asked him what was going on.

Nothing broke. No error message, no alert. It just got quietly dumb and kept going.

What I'd actually do about it

Put it on the calendar. Once a month, pick one thing your AI is handling and go look at the real output instead of the dashboard. Read five of the emails it sent. Listen to a call if it's answering your phone. Pull the last ten leads it was supposed to follow up on and see what it actually said to them.

And write down the rules you gave it, somewhere you can find them again. Most of the businesses I sit down with can't tell me what their automation is running on anymore, because whoever set it up did it in a hurry a year ago and never wrote any of it down. If you can't say the rule out loud, you've got no way to notice when it stops getting followed.

Ten minutes a month. That's the whole ask.

The reason the Luna story got under my skin is that Andon Labs does this for a living. Testing the limits of AI agents is literally their job, they were watching this thing closely, and the policy still went missing for months right under them. The rest of us are running actual businesses with actual customers, and we're not watching anywhere near that close.