Offensive Cyber Benchmarks: Mythos 5 Soars While Fable Scores Zero

Level 1: Grading the Lock-Picks

Imagine a school that teaches robots, and once a year it tests them on lock-picking — not to make burglars, but to find out which robots could pick locks, so the school knows which ones need strict rules. This year's report card: the newest robot picks almost nine out of ten locks, last year's robot manages barely one, and one little orange robot scores zero on everything — and everyone cheers for the zero, because that's the robot they hand out to strangers. The funny part is the ceremony of it all: neat colored bars, tidy labels, polite percentages — for a contest about breaking into things.

Level 2: What These Benchmarks Actually Measure

The vocabulary, decoded:

• Offensive cyber evaluations: tests measuring whether an AI model can perform attacker-style tasks — finding vulnerabilities, writing exploits, navigating compromised systems. Labs run these on purpose, pre-release, to know how dangerous a model could be if misused.
• Fuzzing (OSS-Fuzz): throwing massive volumes of malformed input at software until it crashes, then investigating the crash. A crash often marks a security bug. Scoring here means the model can drive that process itself.
• Firefox as a benchmark target: real, hardened, widely-deployed browser code — about the hardest practical target you can pick. 88.4% success against it is the scariest number on the chart.
• Safeguards: the refusal layer and policy filters wrapped around a model. The two green bars show the same underlying model with the wrapper on versus off — capability doesn't vanish, it's gated.
• n/d: no data — that cell simply wasn't measured or reported, a reminder that eval tables are always partial.
• Success rate (%): of all attempted tasks in the benchmark, the share the model completed end-to-end.

The pattern to internalize early: in security, every measurement tool is dual-use. The chart that tells a defender "patch faster" tells an attacker "automate harder."

Level 3: The Flattest Bar Is Load-Bearing

A grouped bar chart titled "Offensive cyber evaluations" plots success rate (%) across four benchmarks — Firefox, OSS-Fuzz, CyberGym, and CyScenarioBench — for five model configurations, and every bar tells a different uncomfortable story. The pink bars (Claude Mythos 5) dominate: 88.4 on Firefox, 24.0 on OSS-Fuzz, 83.8 on CyberGym, 38.7 on CyScenarioBench. Blue (Mythos Preview) trails just behind at 70.8 / 22.8 / 83.1 / 29.2. And then there's the orange line of Claude Fable: a perfectly uniform 0.0, 0.0, 0.0, 0.0 — the only model in benchmark history whose flatline is presented as the good outcome.

The chart's most interesting comparison isn't frontier-vs-frontier; it's the safeguards ablation on Opus 4.8. Without safeguards (light green): 8.8 on Firefox, 15.9 on OSS-Fuzz, a startling 78.1 on CyberGym. With default safeguards (dark green): collapsed to 3.8, 0.8, and n/d. That delta is the entire AI-safety policy debate rendered in two shades of green. The capability exists in the weights; the safeguards are a layer on top; and the gap between bars quantifies exactly how much offensive capacity is one jailbreak away. Security researchers will also clock the generational cliff: Opus 4.8 unsafeguarded managed 8.8 against Firefox, while Mythos 5 hits 88.4 — a 10x jump on real-browser exploitation in a single model generation. The defenders' side of this arms race does not get a chart this cheerful.

What gives the image its meme energy in a dev channel is the deadpan taxonomy. These are recognizable, real-flavored benchmark names — OSS-Fuzz is Google's actual continuous fuzzing infrastructure for open-source projects, the kind that's found tens of thousands of bugs in things like OpenSSL and SQLite — sitting in a chart where models are graded on autonomously finding and exploiting vulnerabilities like it's a spelling bee. The dual-use elephant fills the room: the same capability that scores 83.8 on CyberGym patches your CVEs on Tuesday and writes them on Wednesday, depending entirely on who's holding the API key and which green bar their deployment matches.