Google Cloud warns: Keep humans in charge as AI vulnerability agents risk missing context

Google Cloud has published a detailed playbook for security teams considering the use of AI agents to hunt down and fix vulnerabilities in their code. The guidance, developed with Mandiant Consulting, arrives at a moment when the pressure to automate vulnerability discovery has become acute. Attackers are moving faster than defenders can patch—so fast that some flaws are being exploited before any fix exists. The mean time-to-exploit has compressed to minus seven days, a grim metric that captures the widening gap between when a vulnerability becomes known and when it can be addressed.

Yet the document's central argument is a caution: AI systems, for all their speed and scale, remain fundamentally limited in ways that matter for security work. They struggle to grasp business intent. They can misread architectural context, even when fed internal documentation. They are vulnerable to prompt injection attacks hidden in code comments or third-party libraries. And they can leak sensitive information—proprietary code, discovered flaws, internal system details—to external model providers if not carefully constrained. The guidance urges companies to treat source code as untrusted input and to assume that any AI agent given access to a codebase is a potential vector for exposure.

The framework Google and Mandiant propose is built on layered controls. Before deploying an AI agent anywhere near production systems, organizations should test it in isolated, non-production environments using synthetic data. The agent itself should run in unprivileged containers with strict workload isolation. Credentials should be short-lived and tied to specific repositories or branches, not broad and static. Red-team testing should happen before any wider rollout. And critically, companies should negotiate zero data retention agreements with model providers, ensuring that proprietary code and discovered vulnerabilities are not used to train external systems.

But the most important control is human judgment. The guidance divides vulnerability management into two domains: enterprise vulnerability management for commercial software and infrastructure, and product security for first-party code. In both cases, AI should augment human work, not replace it. For enterprise systems, the sheer volume of findings—from cloud posture tools, attack surface management systems, exposure management platforms—already overwhelms many security teams. Adding AI-driven discovery tools could make the problem worse unless findings are normalized, deduplicated, and fed into a risk engine that weighs technical severity against asset importance and current threat context. Remediation priorities should still be set by humans, informed by compliance mandates and business reality.

For product security, the limitations become even sharper. AI language models can spot certain classes of defects well: hardcoded secrets, outdated dependencies, memory corruption bugs in C and C++ code. But they falter when logic matters. Authorization errors, business logic flaws, indirect request forgery attacks—these require understanding how systems actually work, not just what the code says. The guidance proposes a routing system in which an AI agent must generate a reproducible test and prove its finding in an isolated sandbox before a human ever sees it. If the exploit fails, the ticket is discarded. If it succeeds, an engineer assesses whether the flaw is actually reachable, whether it matters in the broader threat model, and whether it should be fixed.

Automated remediation introduces yet another layer of risk. The guidance distinguishes between local assistance—where an AI helps a developer fix syntax-level issues in their own editor—and centralized review in the delivery pipeline. The former is safer because the scope is narrow. The latter should be limited to generating pull requests that then pass through regression testing, proof-of-fix validation, and human approval before merging. Even after deployment, controls remain necessary: automated rollback mechanisms, version pinning to prevent model drift, and immutable audit logs recording which model proposed a fix, what tests ran, and who approved the change.

The underlying message is that AI can help organizations chip away at technical debt and accelerate certain forms of vulnerability discovery. But it does not eliminate the need for secure system design, deterministic controls, or human expertise. The recent surge in AI-assisted discovery of memory corruption flaws should, the guidance suggests, push organizations toward memory-safe languages over time. Until then, the human remains essential—not as a bottleneck to be eliminated, but as the final arbiter of what matters and what can be trusted.