Long-Context Retrieval MRCR Benchmark Shows Opus 4.6 Crushing Sonnet 4.5

Description

A bar chart titled 'Long-context retrieval' with subtitle 'MRCR v2 (8-needle)' on a white background. The Y-axis shows 'MEAN MATCH RATIO (%)' from 0 to 100. Four bars are displayed: Opus 4.6 at 256k context with 93.0% (tall salmon/coral bar), Opus 4.6 at 1M context with 76.0% (lighter salmon bar), Sonnet 4.5 at 256k with 10.8% (small dark blue bar), and Sonnet 4.5 at 1M with 18.5% (slightly larger light blue bar). The chart dramatically illustrates that Opus 4.6 massively outperforms Sonnet 4.5 on long-context retrieval tasks - 93% vs 10.8% at 256k tokens is a staggering gap. Interestingly, Sonnet 4.5 performs slightly better at 1M than 256k context

Common Tags

Llm BenchmarkingTools PerformanceTesting DataVisualization AIIndustryTrends MachineLearning

Comments

Anonymous ★ Top Pick Sonnet 4.5 at 256k context: 10.8% retrieval rate. At that point you're not doing needle-in-a-haystack, you're just losing needles

Anonymous ★ Top Pick

Sonnet 4.5 at 256k context: 10.8% retrieval rate. At that point you're not doing needle-in-a-haystack, you're just losing needles