Amazon Is Closing the Service Where Humans Checked AI's Work

A guy in one of my workshops last month told me he'd stopped reading what his AI wrote before he sent it out. He'd checked the first thirty or so, they were all fine, so now he copies and pastes. He wasn't bragging about it. He was tired, and it was one less thing on the list.

I thought about him when this news came across my desk.

What Amazon announced

Amazon is closing Mechanical Turk on September 30. The thing has been running since 2005, which puts it at 21 years old, and Bezos used to call it "artificial artificial intelligence." The setup was pretty simple. Businesses posted small jobs a computer couldn't handle yet, and real people around the world logged in and did them for a few cents apiece. Labeling photos. Transcribing audio. Filling out surveys. Reading a block of text and saying whether it actually made sense.

Two other Amazon services shut down the same day. SageMaker Ground Truth, which is how a whole lot of AI training data got labeled, and Amazon Augmented AI. That second one is the one I'd point at. It existed for one job: when a model wasn't confident about an answer, it handed that answer to a person to look at before it went out the door.

Amazon says the decision came out of a review of its programs and services. The plainer version is that the models got good enough that the market moved on, and the labeling work that's left went to companies like Scale AI, Mercor, and Prolific. Amazon stopped taking new Mechanical Turk customers back in July, so people saw this coming.

Why I'm putting this in front of you

You don't use Mechanical Turk. Almost nobody reading this does, and your business isn't going to feel a thing on September 30.

But for two decades, when you bumped into something automated, there were decent odds a person was sitting behind it somewhere quietly fixing what the machine got wrong. That layer is being retired all over the industry right now, and it's happening in the same stretch of years where small shops started handing over real work. Invoices. Job postings. Estimates. Quotes that go out to a customer with a number on them.

Nobody's checking those but you.

What I'd actually do about it

Pick the stuff that leaves your building and treat it different from the stuff that stays inside. If it's a first draft of a blog post or notes from a meeting, skim it and move on, no harm done. If it's got a price, a date, a name, a measurement, or a promise in it, somebody reads it out loud before it goes. That's the whole rule and it takes about forty seconds.

The failures I've cleaned up for clients this year weren't dramatic. It was a chatbot quoting last spring's pricing because nobody updated the document it was reading from. It was an estimate where the AI carried the square footage from the previous job. Small, boring, expensive stuff, and every one of them would have been caught by one person spending a minute with it.

The other habit worth building is checking your own tools on a schedule instead of waiting for a complaint. Once a month, ask your setup the five questions your customers ask most and see what it says now. Tools change underneath you without telling you. I've watched an assistant that was accurate in March start making things up in June because the source doc got moved.

And if you've got employees using AI, the review step has to be somebody's actual job, written down, not a general hope that people are being careful. Careful fades in about three weeks. I've never seen it hold on its own.

Where this leaves you

The tools really are better than they were two years ago. I use them all day and I'd be worse at my job without them. But the industry is pulling the human safety net out at the exact moment the average small business is leaning on this stuff harder than ever, and if you're the last set of eyes on the work, you should know that's the arrangement you're in.

My guy from the workshop went back to reading his drafts. Took him about two minutes a day. He found something worth fixing in the first week.