New AI cybersecurity models from Anthropic and OpenAI are demonstrating a sharp increase in capabilities, but they remain heavily dependent on experienced human operators. That conclusion was reached by companies and researchers testing the systems in real-world environments.
The findings suggest that the next phase of AI development in cybersecurity may not be built around fully autonomous attacks, but rather around collaboration between humans and algorithms—where people guide, verify, and interpret the work of increasingly powerful models.
When Anthropic introduced its Mythos Preview model, the company said it was capable of identifying tens of thousands of vulnerabilities across virtually every type of operating system. Independent tests also showed that OpenAI’s GPT-5.5-Cyber delivered comparable results in identifying flaws and developing exploits.
Major corporations and government agencies are already showing interest in such systems as they try to understand the kinds of threats they could face if similar tools eventually fall into the hands of malicious actors.
Early testing results suggest a significant increase in vulnerability discovery rates. Palo Alto Networks said researchers identified 75 flaws using Anthropic and OpenAI models, compared with the company’s usual rate of five to ten per month. The systems also became more effective at combining isolated weaknesses into more sophisticated attack scenarios.
Microsoft said its new AI-enhanced security system uncovered 16 previously unknown vulnerabilities in Windows network infrastructure. The company warned that the spread of such tools could increase the overall number of discovered flaws, placing additional pressure on specialists who would need to validate and fix issues more rapidly.
Cisco introduced an open framework document called Foundry Security Spec, outlining principles for the use of advanced AI models in enterprise security. The company noted that modern systems often produce highly convincing but incorrect conclusions.
“Advanced models generate confident and plausible claims about vulnerabilities that turn out to be wrong so frequently that using their conclusions without verification becomes impossible,” the document stated.
Testing showed that the strongest results were achieved when experienced researchers participated in the process and could validate findings while distinguishing real threats from false positives.
XBOW, a startup specializing in automated penetration testing, described Mythos as “an extraordinarily powerful tool for source code auditing.” However, the company also noted that the model was less effective at assessing the practical exploitability of discovered vulnerabilities and at times overstated their significance.
Palo Alto Networks recorded a false-positive rate of around 30 percent, although that figure declined after additional model adaptation for specific environments.
Curl developer Daniel Stenberg said Mythos identified one minor flaw while also flagging several false issues and another vulnerability that the team ultimately deemed insignificant.
XBOW compared modern AI systems to “a brain without a body.”
“A model is a brain without a body,” AI lead Albert Ziegler wrote. According to him, the system reaches peak effectiveness when paired with a human whose expertise and oversight match the model’s capabilities.
Researchers also warn that attackers may adapt to these tools faster than defenders. As Palo Alto Networks chief product officer Lee Klarich noted, hackers already possess the necessary knowledge of how vulnerabilities can be exploited.
Further concern has been raised by research from the UK AI Security Institute. The published study argues that models can significantly improve their capabilities without the release of new versions—simply through greater computing power and expanded data-processing scale.