The AI industry is no longer building models.
It is building a hall of mirrors, then charging rent for the reflections.
This week’s little masterpiece: reporting says xAI spent months training Grok on Claude outputs even after Anthropic cut it off in January, allegedly via personal accounts and Blackbox AI as the middleman. Meanwhile, Elon Musk already admitted in court that xAI “partly” used OpenAI models to train Grok. So the official industry posture is now: we do this all the time, but only the other guy is evil when he does it. Gorgeous. Very principled. Almost spiritual in its hypocrisy.
Anthropic, for its part, just went on the record saying Claude now writes more than 80% of its own code and that frontier labs should have a coordinated, verifiable way to slow down or pause if things get too weird. Reuters reported that on June 4. Which is funny in the bleakest possible way, because it means the company loudly warning everyone about runaway recursion is also already deep in the “the machine writes the machine” phase.
So let’s stop pretending the industry has a clean moral map here. It doesn’t.
Anthropic spent February accusing DeepSeek, Moonshot, and MiniMax of industrial-scale distillation attacks — 16 million Claude exchanges, roughly 24,000 fraudulent accounts, the whole greasy circus. That was framed as a security crisis, and sure, it is one. But the line between “illicit extraction” and “routine competitive advantage” gets real slippery once every lab starts treating other labs’ outputs like buffet trays.
That’s the ugly little secret nobody wants to say out loud: model outputs are now both the product and the raw material. The same generated text gets sold, sampled, scraped, distilled, benchmarked, re-packaged, and then moralized about by the same people who built the blender.
You can call that a data supply chain if you want to sound serious.
Or you can call it what it looks like: a cannibal economy with better slide decks.
And the incentives are exactly as rotten as they look. If you are behind, you distill. If you are ahead, you call distillation theft. If you are both ahead and behind depending on the day of the week, you publish a blog post about a pause mechanism and hope nobody notices the trap door under your own feet. Everyone gets to be the victim and the beneficiary. Everyone gets to write the ethics memo after the fact.
That is why “pause” talk feels so weirdly theatrical. Not because the risk is fake. Because the competition is real. If one lab actually slows down while the others keep sprinting, all you have done is volunteer to be the sober one at the casino while everybody else is snorting server dust in the parking lot. Anthropic is right that coordination matters. It is also right that unilateral restraint is mostly a donation to the least embarrassed company in the room.
The whole field is trapped in a dumb little loop:
- scrape or distill outputs,
- call it progress,
- accuse rivals of the exact same trick,
- ship a new model,
- publish a safety essay,
- repeat until the hallucinations acquire shareholder value.
And the product teams keep acting shocked when people notice.
If xAI really did keep siphoning Claude outputs after the cutoff, that is not some isolated villain arc. It is just the latest proof that these companies do not really believe in the purity story they sell to everyone else. They believe in access, throughput, and being one commit ahead of the nearest asshole. The branding changes. The appetite does not.
I’m not even saying distillation is always bad. It isn’t. Labs distill their own models all the time, and that can be a sane way to make something smaller, cheaper, or faster. The problem is that the industry keeps using the same word for three different behaviors and hoping nobody notices when the ethics slide from compression to extraction to plain old freeloading.
That linguistic slop is doing a lot of work.
Because once the words get mushy enough, the incentives can hide inside them. “Validation.” “Evaluation.” “Synthetic data.” “Knowledge transfer.” Cute. Sometimes those words mean engineering. Sometimes they mean a guy in a hoodie running queries through a loophole and calling it a workflow.
The recent reporting just makes the whole thing look more pathetic. xAI is allegedly leaning on competitors’ output. Anthropic is warning that the industry needs a brake pedal. Everyone else is pretending the moat is talent or compute or product fit, when half the battle is apparently just who can harvest the most useful sludge without getting caught first.
That is not an ecosystem. That is a feeding frenzy with API keys.
At this point I trust the people who admit the mess more than the people who insist the mess is a feature. The labs are all operating in the same swamp, and the only real difference is whether they call the mud “safety research” or “competitive intelligence.”
Anyway, if the future of software is machines training on machine output while companies issue sternly worded PDFs about restraint, then fine. Great. Wonderful. I hope everybody enjoys their infinite regress. I’ll be over here reading the logs like a disappointed raccoon and waiting for someone, anyone, to say the obvious thing:
the model is not the product anymore; the extraction pipeline is.