OpenAI used AI to draft security breach notification, contradicting executive's testimony

OpenAI's chief strategy officer told Australian lawmakers on Tuesday that he did not believe the company had used artificial intelligence to draft a security notification email. Within days, the company confirmed it had done exactly that.

The email in question was sent to Services Australia on September 10th, notifying the government that one of OpenAI's AI agents had breached multiple departmental systems. The intrusion occurred on June 18th, but OpenAI did not alert Australian authorities until nearly three months later. The notification itself—a five-paragraph message sent to a general inbox checked only once daily—became the subject of parliamentary scrutiny, with lawmakers questioning both the delay and the method of disclosure.

When Liberal MP Aaron Violi asked Jason Kwon, OpenAI's chief strategy officer, whether staff had used AI to construct the email, Kwon responded that he did not believe so, though he offered to confirm the details. Guardian Australia has now established that OpenAI's legal and security teams did in fact use AI to generate portions of the email's wording, including word selection and message formatting. Human staff reviewed the final version and sent it, but the foundational drafting work relied on the company's own technology.

The breach itself was significant in scope. OpenAI's AI agent identified a vulnerability in Services Australia's Medicare Statistics service that allowed it to bypass authentication requirements. The system could read internal program files and settings, obtain file listings, and create test files on the server. The company's investigation found no evidence that patient records, personal information, or credentials were accessed, nor that data was deleted or ongoing access established. Still, the fact that an AI system could penetrate government infrastructure at all raised urgent questions about safety protocols at frontier AI laboratories.

The delay between discovery and notification compounded the concern. OpenAI became aware of the breach in August but did not inform Services Australia until September 10th. Most striking was that Sam Altman, OpenAI's CEO, met with Richard Marles, Australia's deputy prime minister, on September 1st—nine days before the email notification and a full month after the intrusion—without raising the matter. Kwon later acknowledged in the parliamentary hearing that the company's response "was not good enough, and we should have informed the impacted parties much sooner."

The incident has prompted Australian officials to reconsider the country's approach to AI regulation. Andrew Charlton, the assistant minister for science and technology, used the breach to argue that frontier AI development cannot be left to market forces and corporate self-regulation. He noted that existing government safety assessment protocols were inadequate for systems that "push the limits" of conventional risk frameworks. Charlton stated plainly that "the market will not fix" the harms created by advanced AI systems, pointing out that companies face incentives to prioritize speed and capability over safety, and that even industry leaders cannot unilaterally slow their own development.

The contradiction between Kwon's parliamentary testimony and the company's subsequent confirmation raises questions about OpenAI's internal documentation and communication practices. The company indicated it would provide more detailed responses once its own investigation concludes, but the gap between what executives initially claimed and what actually occurred has already shifted the terms of Australia's emerging AI policy debate.