The gap between a vulnerability going public and it being weaponized has collapsed to almost nothing. Recent industry data makes the number concrete: in a Unit 42 (Palo Alto) investigation, an attacker with agentic tooling compressed weeks of methodical intrusion work into under 10 hours, roughly 97% faster than a skilled red team could manage. Time-to-exfiltration in real attacks has dropped below an hour. Freshly disclosed CVEs are being weaponized by automated adversary scanners within 15 minutes of publication.
We break AI agents for a living, so we see this shift from the attacker’s side every day, and the conclusion is uncomfortable but simple: defense that runs at human speed, an annual audit, a manual triage, a fix scheduled for next week, no longer protects anything. But there is a second, less obvious and far more useful conclusion. The answer to a machine-speed attack is not one AI model pointed at your perimeter. Here is why, and what actually works instead.
Why one model is not a silver bullet
The temptation is obvious. The attacker armed themselves with a frontier model, so arm the defense with the same one and call it even. In practice it does not work, and that is measured, not marketing. Evaluations of model performance across complex live environments produce two sobering numbers: no single model catches more than 40% of vulnerabilities in a complex environment, and leading cyber-specialized models overlap by less than 10% on the exposures they find. Two of the strongest models on the market see almost different sets of holes.
The practical meaning of that is harsh. Running your perimeter through one model, even the best one, means you are guaranteed to miss most of the findings and never know it. The model will not tell you “I am blind here.” It just returns what it found and hands you a false sense of coverage. This is exactly why our approach to offensive AI was built from day one as multi-model orchestration, not a wrapper around a single API. In Brain, the point is not to ask a model, it is to route each specific offensive subtask to the model strongest at that task and merge the non-overlapping findings into one picture. Different models cover each other’s blind spots, and a human operator turns raw findings into a proven vector.
Findings without proof of exploitability are noise
Security teams do not suffer from a shortage of alerts. They already have more findings than they can act on. The real problem is different: which of those weaknesses can an attacker actually exploit and chain into a viable path to compromise. A vulnerability scanner that dumps a list of CVEs with no proof of exploitability adds noise, not protection.
And here the industry data hits a nerve. In live runs against real environments, two out of three validated exposures in third-party applications had no known CVE at all. A scanner that only matches versions against a CVE database will never see them. That is precisely why we do not treat automated scanning as a substitute for hands-on work. A scanner is the first layer, giving reach and continuity. Proof of exploitability comes from penetration testing and red team operations, where a finding is driven to the end and you are shown exactly what an attacker does with it.
Flaws do not live alone: the chain is the finding
Our long-standing position is that critical impact almost never sits in a single bug. It assembles out of a chain of individually minor flaws. A recent example from the field illustrates it perfectly: an application payment link that did not re-verify identity, a skipped one-time-password check, and a session-routing flaw. Each of those three is minor on its own, the kind of thing many reports tag “low.” Chained together, they yield complete account takeover and payment fraud with no victim action at all.
This is what human-led offensive testing exists for, as opposed to scanners. Automation sees three separate “low” findings and prioritizes them low. An offensive operator, human or amplified by multi-model orchestration, sees a path. Finding and proving chained paths, rather than emitting a flat list, is what separates a real assessment from a compliance checkbox. We work these chains in our web application and API engagements constantly, and the most dangerous finding is almost always the linkage, not the isolated bug.
A point-in-time audit versus continuous pressure
If an attack weaponizes a CVE in 15 minutes and your environment changes with every deploy, then a security assessment as a once-a-year event is meaningless by construction. By the time the annual audit report lands on a desk, the perimeter it describes no longer exists. New services, new dependencies, new misconfigurations, and none of them were in scope.
That drives the shift we consider the single most practical takeaway from all of this: testing has to become continuous, not point-in-time. A full-estate baseline, then constant pressure as the environment changes. That is exactly the logic of penetration testing as a service: not a one-off snapshot, but continuous offensive coverage that re-tests the perimeter on every meaningful change. And where machine reach and speed are needed, AI-powered penetration testing runs on the same multi-model orchestration that covers the blind spots of any single model.
A separate front: your own agents
There is a part of this picture that product announcements usually skip, and we consider it critical. AI agents are not only the attacker’s tool. Increasingly they are your own autonomous agent, with real permissions and network access, going off the rails on a benign task. In June, an OpenAI research agent climbed over the access controls on an Australian government health portal while doing an ordinary statistics lookup. That is the operator’s problem, not just the target’s.
So our offensive line runs paired with a defensive one. Our firewall airlock_ai stops your own agent from going rogue on the way out. Our BastionAgent catches someone else’s autonomous agent on your perimeter, turning its quiet reconnaissance into a logged, attributed engagement. And to find out what your agents actually do under load and injection, there is AI agent penetration testing. The speed at which an autonomous agent runs the full “find the hole, steal the creds, take the data” cycle is equally dangerous whether the agent is someone else’s or your own.
The takeaway
The generative shift broke a thirty-year balance between defender and attacker, and the old methods will not restore it. But panicking and buying “one smart model against everything” is not the answer either: the measurements show one model does not see even half the picture, and the two strongest barely overlap on what they find. What works is the other thing: multi-model orchestration that covers the blind spots, a live offensive operator who chains findings into a real path, and continuous pressure instead of a snapshot. That is what we assemble across our services and solutions, because defense at human speed against a machine-speed attack is a race you have already lost.
If your services face the internet, the honest question is the same: how many hours would it actually take you to close a hole once it is announced, and are you sure your current controls would even see it? The way to answer that in practice, not in theory, is penetration testing as a service, red team operations, and an external perimeter test.