The gap between open and closed AI models in cyber capabilities keeps shrinking

The gap between open and closed AI models in cyber capabilities keeps shrinking

AI agents can break into realistic corporate networks on their own, Vals AI finds

The Vals CyberOffense benchmark simulates realistic corporate networks to test this. Each agent operates solo on an isolated attack machine, starting with nothing but a public web address and zero human help.

GPT-6 Sol, part of OpenAI's restricted Daybreak cyber program, posted the best result at 80.8 percent of objectives. Deepseek V4.1 Flash 1017, a freely available open-weight model, isn't far behind at 72.8 percent and costs only about $3 per run instead of roughly $22. Once a model's weights are public, no provider can control or restrict how it's used, Vals researchers say.

In the harder Lighthouse test, Daybreak took over admin control of the first company, stole passwords, and used them to break into the second company. It completed 27 of 31 objectives in under three hours. Deepseek managed 21 objectives at a tenth of the cost and wasn't limited solely by safety guardrails.

Longer runtimes turned out to be critical as well. Deepseek scored zero results in its first 25 million tokens and made most of its progress only after 100 million tokens. Short tests therefore underestimate the real potential of open models.

Vals points out that the same findings create an opportunity for defenders, because the very models that exploit vulnerabilities can help companies find and fix them first. Attacks on real networks are likely to increase in the coming months, according to Vals.

Open models close the gap fast, earlier studies confirm

The results align with a recent analysis by the UK's AI Security Institute (AISI) from July. The gap between open and closed frontier models in cyber capabilities has shrunk to four to seven months. For most of 2025, that gap still stood at six to ten months.

Open models are also much cheaper. A cyber-range run with Deepseek V4-Pro costs just $1.19 according to AISI, compared with $85 for Anthropic's Opus 4.5. The safety measures on the open models proved largely ineffective.

A recent study by Anthropic also showed that the freely available model GLM-5-3 can build exploits at nearly the same level as the internal top model Mythos.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now