Home / AI Safety Conspiracy Theories (September 2026)

OpenAI, Anthropic, and xAI, are calling for the government to regulate AI for safety reasons. It's strange for tech CEOs to ask for more regulation, so there's lots of speculation about what's going on.

One camp says these CEOs are legitimately worried about safety, possibly due to first-hand knowledge of how fast the models are improving or safety issues that have yet to be disclosed. Another camp believes they want regulation because their businesses are not as defensible as assumed, and they need a regulatory moat to help hold back the growing number of cheap open source competitors.

I think it's unlikely these three fiercly competitive CEOs are coordinating in secret to deceive the public. I also think it's unlikely there's no financial motivation behind their calls for regulation. I don't have insider knowledge, but the one thing I find curious is what people aren't talking about. Almost no one is talking about data.

The more data, the smarter the model, which means the best models are trained on a lot of data. They are trained on blog posts, social media posts, photo albums, books, encyclopedias, magazines, textbooks, and every piece of data you can conceivably feed into a computer, even if it means tearing every page out of millions of books.

Our laws were not written with knowledge of AI, and most of the data in the world was generated before the creators even knew what AI was, which makes it hard to know how the data should be treated when it comes to training private AI models. However, the simplest answer is that if models were trained on public data or a large amount of data we don't know how to enforce the law for, they should be owned by the public (i.e. nonprofits) or at least open source.

It turns out there are already lots of open source models and they are pretty good. They aren't as good as the frontier models, but for most everyday tasks, they are already good enough. Not to mention, they are cheaper and more flexible. If I was a frontier lab gearing up to go public, I'd be nervous about competition from cheaper open source models, especially if I've already been burning billions of VC dollars to subsidize costs. Not to mention, open source has the moral high ground with data and there could be more big lawsuits coming down the pipe.
‍
The best argument against open source models is safety. It's, "we can't trust the public to be able to tweak these models themselves because they could make them do something bad." I believe a lot of the frontier lab leaders are, at least in part, ideologically motivated. And so to be able to view yourself as the public benefit corp good guy, while using public data to train private models, it helps to believe that only you can mitigate AI risk. You're not training your models on the world's data to get rich, you're saving the world from destroying itself.

My guess as to what is happening is that these CEOs are genuinly scared of AI risk, but they are also scared of open source competition, so they are pre-empting the entire public debate by saying, "AI is going to kill everyone, we need regulation" to try and avoid a much more nuanced public debate about the dangers of open vs closed source AI, because both scenarios are genuinly unnerving when you stop to think about it.

If everyone is using AI controlled by a small number of companies that are deeply intertwined with the government, it's hard to imagine AI won't be used to sway public opinion. Compare Grok vs Gemini and it becomes obvious how much the culture of a company influences results. Now imagine how much power politicans would have over AI, even indirectly, if they are regulating a few trillion dollar companies. We already know social media has been used to manipulate the public but AI could be both more subject to regulation and more persuasive.

On the other hand, it's scary to think bad actors could easily manipulate open source models to do any number of illegal things, whether that's designing weapons, creating illicit materials, or even just spreading spam and deception. Letting anyone do anything with superintelligence doesn't seem super intelligent. The only world where it's potentially ok for anyone to be able to have AI teach them how to make biological weapons (for educational purposes, of course) is one where there is so much physical surveillance that it's extremely difficult to ever do so, and I don't want to live in that world either.

My hope is that we land in a middle ground. We should allow maximally permissive open source models, but treat them like wholesalers that can only sell to registered retailers. Retailers will then be responsible for setting guardrails and serving those models to end users, and importantly, the retailers should be held liable if they break the law or knowingly allow users to break the law without reporting it to authorities. The frontier labs can act as both wholesalers and retailers, so users can choose between a vertically integrated solution or a more open one, sort of like the AI version of iOS vs Android.

The benefit of this approach is that you will have a manageable number of registered companies responsible for establishing guardrails and reporting illegal AI use, but hopefully that power will not concentrate into the hands of a tiny number of powerful companies who are deeply intertwined with the government. This also allows for maximally permissive open source models to be created, which should increase competition and drive down costs for consumers, while pushing regulation to the local level, where it will vary by jurisdiction and frequently change.

In a sense, AI could be regulated a bit like alcohol. Distillers can't legally sell deadly poison, but they can create very high proof alcohol, and then sell it to retailers who are responsible for adhering to local laws, like not selling alcohol to minors. I would certainly prefer this somewhat decentralized future to one where there are only three big state-sanctioned bars serving state-distilled liquor with framed pictures of our government's dear leader covering the walls.