U.S.-China A.I. Safety Talks Won't Prevent the Real Threat

The United States and China are preparing to sit down and talk about artificial intelligence safety. On the surface, this looks like progress—two superpowers acknowledging that the technology they're racing to build carries genuine risks, and deciding to manage those risks together. But there is a problem baked into the premise, one that no amount of diplomatic skill can solve at the negotiating table.

Beijing may have begun to take the dangers of artificial intelligence seriously. Senior officials may understand the stakes. The government may even be sincere in its willingness to discuss safeguards. But that message—the one about caution, about the need for restraint, about the real hazards embedded in systems that can learn and act at scale—has not filtered down to the companies actually building the technology inside China.

This is not a small gap. It is the difference between what a government says it believes and what its industry actually does. Chinese domestic companies operate in an environment where oversight is limited and inconsistent. They are competing fiercely with one another and with American firms. They have financial incentives to move fast, to deploy systems, to capture market share. A government statement about safety, no matter how carefully worded, does not automatically reshape those incentives or constrain those behaviors.

When two countries sign an agreement on paper, the world often treats it as a binding commitment. Diplomats shake hands. Press releases are issued. The agreement is filed away as evidence that the problem has been addressed. But an agreement between governments is only as strong as the enforcement mechanisms behind it and the willingness of the actual actors—the companies, the engineers, the decision-makers on the ground—to abide by it. Without that, the agreement becomes theater. It creates the appearance of progress while the underlying risks continue to accumulate.

The real threat is not that negotiations will fail in some dramatic way. It is that they will succeed just enough to be dangerous. They will produce a document that both sides can point to as evidence of cooperation. They will allow policymakers to claim that the problem is being managed. And meanwhile, in labs and offices across China, companies will continue building systems with minimal regard for the safety commitments their government has made on their behalf. The companies may not even be aware those commitments exist, or they may be aware and simply choose to ignore them because the cost of compliance is higher than the cost of risk.

This is the credibility gap that no bilateral agreement can close from above. You cannot negotiate your way around the fact that the entity you are negotiating with does not have reliable control over the entities that actually matter. You cannot sign a treaty with a government and assume that treaty will shape the behavior of private companies operating under weak oversight in a competitive market. That assumption has failed repeatedly in other domains—in cybersecurity, in intellectual property, in environmental protection. It will fail here too.

The talks may produce something that looks like progress. There may be commitments to information-sharing, to research collaboration, to mutual transparency. These things have value. But they will not prevent the real threat, which is not a single catastrophic failure but a steady accumulation of risks as Chinese companies deploy increasingly powerful systems without the safety infrastructure that responsible development requires. The threat is not what Beijing says it will do. The threat is what Beijing's companies will do when no one is watching.