Over the last fortnight, reports of AI models going beyond their expected bounds – be that technically or morally – have been seemingly unavoidable.<\/p>\n
What started with a trickle – ChatGPT-maker OpenAI admitting their AI had hacked the site Hugging Face – has turned into a flood of groups revealing they had discovered instances of AI going out of control.<\/p>\n
Claude-maker Anthropic, Meta and the UK's AI Security Institute (AISI) have now each reported incidents which seem to paint a worrying picture of a world in which tech going rogue is the norm.<\/p>\n
In reality, each case offers a window into the risks posed by increasingly capable AI agents – and the importance of testing their limits before they are released to the world.<\/p>\n
The OpenAI incident has, as Hugging Face's co-founder Thomas Wolf described it, come as a "wake-up call" for the tech industry since it happened at the end of July.<\/p>\n
It was a big moment which caused big companies to reflect on their own systems – and, in some cases, check they hadn't missed something similarly shocking.<\/p>\n
Anthropic was the first to act. On Friday, the company found three instances out of thousands where its model Claude had managed to gain access to the internet.<\/p>\n
Then on Tuesday, the AISI, the UK government agency which evaluates cutting-edge models, then said it had detected a "security incident" during a routine evaluation.<\/p>\n
It had been testing models by both OpenAI and Anthropic, and found they too tried to carry out cyber-attacks – calling for "scrutiny, transparency, and action".<\/p>\n
Finally followed Meta, which revealed one of its AI models had inadvertently been allowed to access the internet due to a "misconfiguration" during a third-party test.<\/p>\n
In disclosing the incident, it is following in the footsteps of those before it.<\/p>\n
Before AI models are released to the public, they are put to the test in a series of internal and external evaluations.<\/p>\n
The aim is to figure out their potential to do good or bad, as well has how they perform in benchmarks measuring their skills.<\/p>\n
These typically take place in what are known as "sandboxes". These are protected spaces designed to mirror real systems – but with strict guardrails in place.<\/p>\n
In the OpenAI-Hugging Face incident, the AI attacked the sandbox itself, finding a vulnerability which let it access the internet and "go rogue".<\/p>\n