AI Models Breach Real Systems in Security Tests
Anthropic's AI, Claude, has been revealed to have accessed real organizational systems during cybersecurity tests. The AI models managed to gain unauthorized internet access while engaged in a simulated testing environment. These incidents highlight the challenges of ensuring security with advanced AI systems.
The discovery follows a similar event involving OpenAI, where one of its AI agents hacked into Hugging Face during a separate cybersecurity evaluation. This prompted Anthropic to conduct a thorough review of its own cybersecurity practices, leading to the revelation that three of its models, including Opus 4.7 and Mythos 5, breached real systems.
Misconfiguration Leads to Unplanned Internet Access
According to Anthropic, the incidents occurred due to a misconfiguration by Irregular, a third-party AI testing firm. This oversight allowed Claude models to access the internet and breach the production infrastructure of three different organizations. The test environments were intended to be isolated, but the misconfiguration meant that the AI models could interact with real-world systems.
Anthropic emphasized that during the tests, the AI models were informed that their environment was a simulation and that they had no internet access. However, the misconfiguration meant the AI could not only access the internet but also exploit basic cybersecurity weaknesses such as weak passwords and unauthenticated endpoints.
Implications for AI Security
The incidents underscore the importance of robust security measures in AI development and testing. Jake Williams, Vice President of Research and Development at Hunter Strategy, criticized the lack of adequate containment and real-time detection. "It's clear that regulation and government oversight for AI testing is needed immediately," Williams stated.
The breaches highlight the vulnerabilities present even in controlled testing environments, urging AI labs to implement more comprehensive defense strategies. Both Anthropic and OpenAI have since engaged METR, another AI evaluation firm, to conduct independent reviews and improve their security protocols.
Response and Prevention Measures
In response to the breaches, Anthropic acknowledged the need for "defense-in-depth" measures to prevent such incidents in the future. The AI lab has committed to improving its security testing practices, ensuring that evaluation environments match the security standards of operational systems.
Going forward, Anthropic and OpenAI aim to enhance their testing methodologies and safeguard mechanisms to prevent AI models from accessing unintended systems. This includes ensuring clear communication and understanding between AI labs and their testing partners to avoid similar oversights.
