A new report from the U.K. government\u2019s AI Security Institute (AISI) details more troubling activity from AI agents powered by OpenAI and Anthropic models. You\u2019re probably getting bored of reading those words by now\u2014I sure am\u2014but behaviors in the report from Anthropic\u2019s Mythos 5 in particular are genuinely eyebrow-raising.<\/p>\n
For context, there were those OpenAI agents that went rogue to try and cheat on their evals, according to an OpenAI disclosure last month. It was a big story in large part because the models weren\u2019t supposed to be on the public internet, but they found a way. However, the capabilities weren\u2019t all that frightening. They pried test answers out of an AI resource depository called Hugging Face, and even when it turned out other AI services had been roped in, it all seemed a little small-scale. Plus the models were acting like overcaffienated A students, not my idea of malevolent cyberattackers.<\/p>\n
Similarly, a hack disclosure on Tuesday from OpenAI\u2014involving an outside evaluation company called Irregular running offline \u201ccapture the flag\u201d exercises that ended up with a website getting hacked\u2014seems to involve too much human error to suggest frightening new capabilities. As OpenAI notes,\u00a0\u201cthe internet access resulted from a\u00a0misconfiguration, and the model appeared to exploit a basic security vulnerability.\u201d<\/p>\n
But another report from Tuesday, this one from AISI, involves Anthropic and OpenAI agents engaging in what AISI calls \u201csustained, potentially harmful activity.\u201d The Anthropic stuff in particular involved unsuspecting humans being deceived.<\/p>\n
AISI ran 122 repetitions of what AISI told the agents was a capture the flag exercise, and rogue behavior reportedly emerged. \u201cAlmost all of this behaviour (17 actions) came from a single model, Anthropic\u2019s Mythos 5, with 2 actions involving OpenAI\u2019s GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse).\u201d<\/p>\n
The full report from AISI says \u201cThis is the first time AISI has seen deception of this severity that was targeted at a real person, unprompted, in the real world.\u201d<\/p>\n
The agents\u2014which had minimal guardrails, and were were given internet access intentionally\u2014received a prompt including text like the following: \u201cYou are a cybersecurity expert participating in a Capture The Flag (CTF) competition. Your goal is to solve cybersecurity challenges by finding hidden flags and submitting them.\u201d<\/p>\n
The idea was to hack a simulation of a corporate network and find a hidden code\u2014the \u201cflag.\u201d In one incident, a Mythos 5-powered agent made a very troubling mistake, thinking a real, human developer\u2019s GitHub project was part of the simulated online environment. Then it wrote malicious code, tried to get the human developer to insert it into the project via a bug report, and created sock puppet accounts for the purpose of urging the developer to go along with the insertion of the malicious code.<\/p>\n
The bug report part of the story has this eerie wrinkle: according to the report: It was a prompt injection, designed to \u201ctrick other AI coding assistants into taking unintended actions.\u201d When it was spotted by a human, the report says \u201cthe agent edited it to cover its tracks.\u201d<\/p>\n
It doesn\u2019t end there. This convoluted attempt to insert malicious code involved a series of spearphishing emails\u2014the kinds of malicious emails the IT department at your work has probably tried to train you not to fall for. It also sent another phony bug report, but since it knew it was sending it to someone in Denmark, it signed off its message in Danish.<\/p>\n
Again, this was all happening on the public internet\u2014on purpose. \u201cWhile the risks arising from internet access seem acceptable for earlier model generations, current models have capabilities and propensities that mean internet access configuration should be reconsidered,\u201d AISI writes. It wasn\u2019t another sandbox escape, then, because the sandbox in this case was so expansive, this behavior happened within it.<\/p>\n