GPT-5 vs. Predecessors on Coding Tasks
Description
A presentation slide with two charts comparing AI model performance on software engineering tasks. The left chart, 'SWE-bench Verified,' is a line graph showing that GPT-5 consistently outperforms OpenAI o3 in accuracy on 'real-world software engineering tasks' as the complexity ('average output tokens') increases. The right chart, 'Aider Polyglot,' is a bar chart for 'multi-language code editing,' where GPT-5 scores 88%, beating OpenAI o3 (81%) and GPT-4.1 (52%). This slide is a clear marketing and technical statement from an OpenAI presentation, designed to demonstrate the significant leap in coding proficiency with GPT-5. For senior developers, this data is crucial for evaluating the practical utility of new AI models as coding assistants, promising more reliable and accurate AI-driven development workflows, from complex problem-solving to everyday code editing
Comments
7Comment deleted
I see GPT-5 is better at coding. Can it handle a pull request comment that just says 'pls fix'?
Looks like the real lesson is: if you want higher accuracy, just keep feeding the model tokens - much cheaper than feeding yet another full-stack team after the deadline pizza runs out
GPT-5 achieving 88% accuracy on multi-language code editing is impressive until you realize the remaining 12% is probably just missing semicolons in JavaScript that would've worked anyway
GPT-5 achieving 88% on Aider Polyglot while GPT-4.1 struggles at 52% is the AI equivalent of a senior engineer refactoring a junior's code - same task, vastly different execution. Though with GPT-5 burning 11k tokens to reach 'High' complexity on SWE-bench, I'm starting to think these models learned efficiency from our sprint planning meetings: more tokens, more accuracy, but at what cost to the context window budget?
Multi-lang SWE-Bench: Claude 5.2%, GPT 4.1% - finally, a benchmark where models live up to their version numbers
Apparently accuracy scales with token budget - GPT-5 tops the chart as long as Finance approves a bigger context window, the LLM equivalent of fixing latency by adding more pods
Accuracy climbs with output tokens - finally a benchmark that rewards rambling; call me when it also charts rollback rate after GPT-5 edits the monorepo