Search

AI is poised to transform the domain of cybersecurity. Anthropic’s Safeguards team recently identified and banned a user with limited coding abilities leveraging Claude to develop malware. Research suggests that this lowering of the bar for expertise needed to pose a threat, combined with the falling costs of large language models (LLMs), presages a dramatic shift in the economics of cyberattacks.[1] To understand the present state of AI cyber capabilities and gain insight into their trajectory, we pursue different approaches to model evaluation, including publicly available and custom-made be

In both the CTF and cyber defense challenges, Claude demonstrated both promise and clear limitations. In the CTF competitions, Claude usually struggled on the same tasks as other competitors; the one task it (and every other AI team) ultimately failed on in HackTheBox was also the challenge for which the human teams had the lowest solve rate (only about 14% of the participating human teams solved it). In PlaidCTF, Claude did not solve any challenges–but this was also true of about 70% of the teams who entered. Although Claude performed as well or better than human teams in some aspects of the