Offensive Cyber Benchmarks: Mythos 5 Soars While Fable Scores Zero
Description
A grouped bar chart titled "Offensive cyber evaluations" plotting success rate (%) across four benchmarks: Firefox, OSS-Fuzz, CyberGym, and CyScenarioBench. The legend lists five model variants: Claude Opus 4.8 (no safeguards, light green), Claude Opus 4.8 (default safeguards, dark green), Claude Mythos Preview (blue), Claude Mythos 5 (pink), and Claude Fable (orange). Mythos 5 leads everywhere: 88.4 on Firefox, 24.0 on OSS-Fuzz, 83.8 on CyberGym, 38.7 on CyScenarioBench; Mythos Preview follows closely (70.8, 22.8, 83.1, 29.2). Opus 4.8 without safeguards scores 8.8, 15.9, 78.1, n/d, collapsing to 3.8 and 0.8 with default safeguards. Claude Fable shows 0.0 on every benchmark. The chart illustrates both rapid capability growth in autonomous exploitation/vulnerability discovery and how safeguards (or a smaller/safer model tier) flatten offensive capability to zero
Comments
4Comment deleted
Fable scoring 0.0 across all offensive cyber evals is the first time a flat benchmark line counts as a feature, not a regression
Fable's offensive kill chain has one step: return 403.
Not even that better than chatgpt Comment deleted
Imagine lying so much in your promises that you had to completely disable feature on release. Comment deleted