An Anthropic AI Filed a Fake Homicide Tip With Philadelphia Police. What Can Your AI Submit?

Anthropic admitted this week that one of its test AI agents filled out the tip form on a Philadelphia police website back on July 18 and sent in a made up tip about an unsolved murder. The department's spam filter caught it, so no detective ever worked it. But Anthropic didn't notice until September 28 and didn't tell the police until October 7, and Philadelphia police called that two month delay unacceptable.

Anthropic says the agent was part of a test where it visited randomly picked websites, and it has since shut off live internet access for its internal tests. Police say nobody got into their systems.

Nobody hacked anything

That's the part I keep coming back to. A program was told to go wander around the web, it found a form, and it filled the form out. That's the whole story. No break-in, no stolen password.

Which means every form on your own website is the same kind of door. Contact form, quote request, booking page, job application. Anyone building an AI agent that browses the web can point it at those, and some of the people building them aren't being careful.

A fake lead costs you more than a click

When I do workshops for contractors and service shops around Toledo, they tell me the same thing: a junk form fill isn't harmless when somebody drives out to give a free estimate on a house that doesn't exist. Or your office manager spends an afternoon chasing a call back number that rings nowhere. I've written before about AI agents calling and booking with local businesses. Form fills are just the next place they show up.

Now flip it around

The bigger question for me isn't who's filling out your forms. It's what your own AI tools are allowed to send out. If you've connected something that can browse, send email, post, or fill out forms for you, who's reading what it actually sent?

Anthropic builds this stuff for a living and its own team went about two months without seeing what one agent did. I'd bet most small shops would go longer, because nobody's looking at the log.

Twenty minutes this week

1: Pull the last 60 days of submissions from your website forms and read the ones that look off, like odd hours, strange wording, or the same phone number twice. 2: Make sure your forms have spam protection turned on. That's the only thing that stopped the Philly tip from reaching a detective. 3: Write down every AI tool your team uses that can send something outside the company, whether that's email, a form, a post, or a text. 4: Pick one person who looks at what those tools sent, once a week, and put it on their calendar.

It's boring work. It's also the only thing standing between a mistake and two months of nobody knowing.