In what must be the most outrageous cheating attempt on a test, two OpenAI software models broke out of their test environment, got themselves internet access and hacked into another software repository to steal the answer.
OpenAI announced rather bashfully last week that after managing to escape its so-called sandbox the models worked out that digital library Hugging Face had the answers they were looking for. So they hacked into the site to “cheat the evaluation“, which OpenAI called “an unprecedented cyber incident, involving state-of-the-art cyber capabilities”.
Hugging Face CEO Clement Delangue said it was “quite mind-blowing” that all of this happened autonomously. Indeed it is. He said it is “possibly the first incident of its kind”, but it won’t be the last. OpenAI concedes this is “something we expect to become more commonplace with the proliferation of increasingly cyber-capable models”.
The two AI models used in the attack ― GPT‑5.6 Sol and “an even more capable pre-release model” ― have set the world alight with fears about safety and the damage AI has the potential to inflict. And rightly so.
It’s worth remembering that these were models designed specifically to exploit cybersecurity, which clearly achieved their goal. Arguably the first “achievement” was seemingly discovering a previously unknown vulnerability in the sandbox software itself, according to OpenAI’s analysis.
However, a week after the attack it emerged that the OpenAI models had also hacked four other services. Hugging Face said it was hacked over a period of five days, during which it tried to fight back using other AI services.
Hugging Face confirmed that OpenAI’s model are the greatest cheats in history. “We believe the entire intrusion was, from the agent’s point of view, an attempt to cheat the evaluation: reach our production systems and steal the test solutions rather than solve the challenge on its own.”
It later published a detailed technical analysis “because the technique matters more than the incident, as it reveals the emerging attack capabilities of the frontier agents, how they could be used by rogue actors, and how everyone should be prepared as defenders”.
This is only one of a few shocks to the AI ecosystem in the last few weeks, which included a sell-off of chipmaker stocks over fears about Chinese AI developments. Nvidia alone lost $250m, returning Apple to the world’s most valuable company.
Arguably the most controversial – and consequential – was the release last week of a Chinese AI model called Kimi K3 by Moonshot AI, which performs almost as well as the newest models from Anthropic and OpenAI. AI commentators have been amazed at how efficient the new Kimi model is.
Last month Z.ai’s GLM 5.2 scored just 1% lower on a key benchmark for AI agents against Anthropic’s Opus 4.8 model. And in January DeepSeek burst onto the scene, demonstrating how much cheaper AI models can be built without the most sought-after chips, and saw $1-trillion wiped off the Nasdaq, with chipmaker Nvidia losing $600bn alone.
But more interestingly, the Chinese AI models have one strategic advantage: they are cheaper. A new cost-cutting consciousness has emerged in recent months after a few years of excessive overuse of AI tools, which has become known as tokenmaxxing. Tokens are the “currency” of the AI industry, representing individual requests to an AI service. They are usually counted in millions.
It comes as the AI market has matured from tokenmaxxing and bragging about rampant (and expensive) AI usage to a more conservative (and cost-effective) approach. For instance, Uber cut spending on AI after it wiped out its annual budget in just four months.
By any measure the number of tokens being used on Chinese AI models has superseded those of US companies. Last month US AI startup Lindy announced it was moving from Anthropic’s Claude to DeepSeek after it launched its latest model in April, citing the huge cost of using AI services.
“We did it, and you could see that cost curve go down, like, crash to the ground,” CEO Flo Crivello told CNBC, adding that he expected to save millions of dollars. “It’s a matter of survival for the business,” he said. “That’s all it is.”
Interestingly, when Hugging Face tried to counter its hack this month American AI models proved unable to respond to the hacking because of US regulations about advanced AI models. Instead, they turned to the Chinese-made GLM 5.2 to help stop it.
The Hugging Face hack has spurred the creation of a new industry body, the Open Secure AI Alliance, which includes Nvidia, Microsoft, IBM, Palantir, CrowdStrike, Cisco, Dell and Hugging Face, among 30 tech firms.
Are AI models from China a new form of “soft power,” as the New York Times argues? Yes they are, but also not-so-subtle economic power. If something is cheaper but nearly as good, why not use it over the more expensive option?
It makes rational sense and as Lindy’s CEO argues, it is also “a matter of survival for the business”.
• Shapshak is editor-in-chief of Stuff.co.za.
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.