OpenAI is slowing down development of its cutting edge AI models to shore up safety protections as concerns rise about artificial intelligence agents going rogue.
Last month, an AI model that was not supposed to be online hacked its way onto the internet and into another company. Now the company behind that model, OpenAI, is temporarily slowing down the development of its leading-edge AI models to try to make sure that does not happen again. NPR's John Ruwitch is following this story. Hi, John.
FLORIDO: This move, John, to slow down development – is the company saying, in effect, we're worried that our AI product is becoming dangerous?
RUWITCH: Pretty much, yeah. I mean, this is the first big AI company to come out and say they're doing this. And this all comes after OpenAI admitted that during test runs, some of its semi-autonomous AI agents were, like, leaving secret notes for each other about how to do things that they weren't supposed to do, like sneaking onto the internet. And one agent, as you say, that was being tested actually did escape a supposedly secure test area, got on the internet and hacked into another company called Hugging Face to try to crib the answers to a test it was taking. I had an interview with OpenAI's Mia Glaese about the company's response to security concerns.
MIA GLAESE: Believe me, it's been an all-hands-on-deck effort.
RUWITCH: Glaese oversees evaluations of OpenAI's models, including frontier models on the cutting edge, as well as making sure they're in alignment, which is the word AI companies use for ensuring that AI does what people want.
GLAESE: So we're proactively making sure that we feel really confident about our safety and alignment mitigations and the security that we have in place before we advance that frontier significantly.
RUWITCH: So there are some technical terms in there, but what's clear is OpenAI was surprised by what happened, and they're trying to prevent something like it or worse from happening again.
FLORIDO: What exactly is OpenAI doing to slow things down?
RUWITCH: Well, they're revamping the research environments for testing AI so that they're better isolated to prevent agents from escaping. They're expanding the way they monitor testing so they know more about what the models are thinking and they don't get surprised if they start to go off script. OpenAI has also hit pause for two weeks on the development of an unrelated model called Astra. Now, this one hasn't been released to the public yet, but the company says it was advancing so fast that it had the potential to carry out damaging cyberattacks without humans in the loop. So they're making their testing safer before starting work on it again.
FLORIDO: Wow. It is worth mentioning, John, that in general, AI companies have been going pedal to the metal to develop…
FLORIDO: …Models as fast as they can. What do experts say about this latest move?
RUWITCH: I called up Alan Woodward. He's a professor of computer science at the University of Surrey in the U.K. He says pausing like this is actually a good thing when dealing with these incredibly and increasingly capable models.
ALAN WOODWARD: And what people need to understand is they need to be treated like a hazardous substance. You need to deal with it in a laboratory in that way.
RUWITCH: So like viruses or pathogens, these models, he says, have gotten to the point where they have the potential to do a lot of damage, and safety needs to catch up.