Platformer – The company says it has built its most dangerous model yet. “Can its coalition of internet companies fix the internet before others catch up? Two weeks ago, Anthropic accidentally leaked the existence of what the company said was its most powerful artificial intelligence to date: a new model, known as Claude Mythos Preview, that represented “a step change” in AI performance. In particular, according to a blog post that leaked due to human error and a misconfigured content management system, Mythos posed serious new risks to cybersecurity. “It presages an upcoming wave of models that can exploit vulnerabilities in ways that far outpace the efforts of defenders,” the blog post stated. On Tuesday, the wave crashed onto the shore. Anthropic announced Mythos alongside Project Glasswing, an initiative with more than 40 of the world’s biggest tech companies that will see Anthropic grant early access to the model to find and patch vulnerabilities across many of the world’s most important systems. Launch partners in the coalition include Apple, Google, Microsoft, Cisco and Broadcom. They’ll be tasked with scanning and patching their own systems along with the critical open-source systems that modern digital infrastructure depends on. Anthropic is giving participants $100 million in usage credits for Mythos, and donating another $4 million to open-source security efforts. Still, today marks a striking and mostly unsettling moment in the development of AI systems. One of the world’s three frontier labs has now created a model it says is too dangerous to release to the general public. These dangers emerged not from any specialized cyber training but from the same general improvements that every other lab is currently pursuing. As a result, models with similar capabilities may soon be accessible to criminals, hackers, and nation states — or even more broadly via open source models.
Already, Anthropic said, the model has found thousands of high-severity vulnerabilities in every major operating system and web browser, and in many cases developed related exploits. Among them: a vulnerability in OpenBSD, a security-focused open source operating system, that had escaped detection for 27 years; another flaw in the video encoder FFmpeg that had escaped detection in 5 million previous automated tests; and “several” vulnerabilities in the Linux kernel, which could be exploited to take complete control of a user’s machine. “Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely,” the company wrote. “The fallout — for economies, public safety, and national security — could be severe. Project Glasswing is an urgent attempt to put these capabilities to work for defensive purposes.” In a video that Anthropic made to accompany the announcement, researchers say that Mythos is more dangerous largely due to its advanced reasoning capabilities. While current models are capable of identifying high-severity vulnerabilities, Mythos might identify five separate vulnerabilities in a single piece of software and then chain them together into a uniquely dangerous new attack. Coupled with models’ growing ability to work without supervision for extended periods of time, Anthropic said we have reached an inflection point in cybersecurity risks…”
- See also The New York Times: “What used to be the province of big countries, big militaries, big companies and big criminal organizations with big budgets — this ability to develop sophisticated cyberhacking operations — could become easily available to small actors,” explained Craig Mundie, a former director of research and strategy at Microsoft. “What we are about to see is nothing short of the complete democratization of cyberattack capabilities.”
- See also The Guardian – US summons bank bosses over cyber risks from Anthropic’s latest AI model. Fed chair Jerome Powell reportedly attends meeting in Washington following release of Claude Mythos.
