Optimized? Benchmark shows the 'faster' version is actually 48 % slower

Description

The image is a screenshot of a tweet. The tweet header shows a blurred profile picture, the name “Kevin Gosse”, and handle “@KooKiz”. The tweet text reads: “My typical workflow as a performance enthusiast: 1. Write the naive solution to a problem 2. Optimize it 3. Benchmark both solutions 4. Pretend it never happened”. Beneath the tweet, a BenchmarkDotNet console output is visible on a black terminal: “BenchmarkDotNet v0.13.7, Windows 10 (10.0.19045.3570/22H2/2022Update) Intel Core i7-5820K CPU 3.30GHz (Broadwell), 1 CPU, 12 logical and 6 physical cores .NET SDK 8.0.100-rc.2.23502.2 [Host] : .NET 8.0.0 (8.0.23.47906), X64 RyuJIT AVX2 Job-YIBPFD : .NET 8.0.0 (8.0.23.47906), X64 RyuJIT AVX2”. A results table follows showing two rows: “Normal 9.454 s Ratio 1.00” and “Optimized 13.979 s Ratio 1.48”. The punchline is that the so-called optimized method is 1.48× slower, illustrating the classic performance-tuning pitfall familiar to seasoned engineers who micro-benchmark their code only to discover a regression

Common Tags

PerformanceOptimization BenchmarkingTools PrematureOptimization DeveloperHumor RelatableDevExperience PerformanceTradeoffs

Comments

Anonymous ★ Top Pick Nothing like a BenchmarkDotNet run to convert your "low-level cache-friendly wizardry" into a certified latency multiplier

Anonymous ★ Top Pick

Nothing like a BenchmarkDotNet run to convert your "low-level cache-friendly wizardry" into a certified latency multiplier
Anonymous

After 20 years of optimizing code, you learn the most important performance metric isn't nanoseconds saved - it's hours of debugging time you'll never get back when your clever bit-twiddling makes the branch predictor cry and the CPU pipeline stall harder than a junior's first production deployment
Anonymous

Ah yes, the classic performance engineer's dilemma: spending three days micro-optimizing an algorithm only to discover your 'clever' solution allocates 3x more memory and runs 48% slower. At that point, you have two choices: admit defeat in the pull request, or quietly git reset --hard and pretend you were 'just exploring the solution space.' The real optimization was the friends we made along the way... and by friends, I mean the CPU cycles we wasted proving that sometimes the compiler knows better than we do
Anonymous

Pro tip: with InvocationCount=1, the only thing you benchmarked is RyuJIT’s warmup; your “optimization” just turned a cache-friendly loop into branch‑mispredict theater
Anonymous

53x speedup and zero Gen2 collections with AVX2, but naive ships to prod - simplicity's the real optimizer
Anonymous

Peak performance engineering: hand‑tune a hot path, trigger a JIT deopt and worse cache locality, BenchmarkDotNet reports +48% slowdown, and the PR title quietly becomes “minor cleanup.”
@purplesyringa 2y

HEY THAT'S LITERALLY ME
HD2 2y

i7 is answer
1. @sylfn 2y
  
  i3wm
  
  @purplesyringa 2y
  
  yu...
  
  HD2 2y
  
  Especially
@hannybu 2y

Replace with origin?
@deerspangle 2y

Compilers are pretty good at optimising shit themselves
1. @purplesyringa 2y
  
  atm i'm stuck with clang being unable to keep 14 integral variables in registers in a tight loop without spilling them to stack
  
  @ColonelPhantom 2y
  
  what platform, amd64?
  
  @purplesyringa 2y
  
  x64
  
  @ColonelPhantom 2y
  
  so amd64 yes
  
  @purplesyringa 2y
  
  ah sorry i thought you said arm64
  
  @ColonelPhantom 2y
  
  hm so 14 is definitely pushing it given the are 16, would fomit-frame-pointer help? if clang/llvm supports that at least
  
  @purplesyringa 2y
  
  it uses rbp as a gpr so i think it's already on
2. Yuri 2y
  
  Tod Howard, is that you? 🤣
3. @SamsonovAnton 2y
  
  "Compiler will optimize!" and 1001 other fairy tales.
  
  @RiedleroD 2y
  
  "I'm able to optimize this better than the compiler!" and 1001 other fairy tales.
  
  @RiedleroD 2y
  
  I mean @purplesyringa might have a different opinion, but imo writing ASM to speedup your code is a waste of time
  
  @purplesyringa 2y
  
  it's usually a waste of time, but when the compiler emits obvious garbage and i can speed up the code by 2x, why wouldn't i do it
  
  @SamsonovAnton 2y
  
  Don't confuse the two things. You are referring to people trying to outsmart the compiler, or even do the compiler's job completely, writing in assembly. This is indeed challenging for most people and hardly needed for most applications, not to say it is near impossible for explicitly parallel architectures with static scheduling. I was referring to people screaming "Compiler will optimize! No thinking required!", thus relying on compiler so blindly that they not only ever check what compiler does and what it does not do, but never scratch their head regarding whether particular constructs may theoretically be optimized or not. I mean that people proclaiming "Premature optimization is the root of all evil" tend to write poor code like: for(int i=0; i<strlen(s); ++i) s[i] = tolower(s[i]); without realizing that even if the compiler is smart enough to move strlen of of loop, it won't do it in all situations like debug builds, not to mention that this turns into 2 passes at best rather than single one.
  
  @deerspangle 2y
  
  And don't confuse me with the folks writing absolute garbage like that. I don't think those folks even consider what the compiler will do
4. @q_rsqrt 2y
  
  You have to be pretty good to allow compilers optimise shit
@Vlasoov 2y

At least he optimized amount of objects in first two generations 😁😁