Skip to content
DevMeme
6556 of 7590
Node.js Event Loop Utilization Drops 80% to 20% by Removing Logs
Performance Post #7184 · source on Telegram

Node.js Event Loop Utilization Drops 80% to 20% by Removing Logs

Description

A tweet by Eric Allam (@maverickdotdev) from trigger.dev stating: 'Node.js event loop utilization for one our services went from about 80% to 20% just by removing logs' with a facepalm emoji. Below is a Grafana-style monitoring chart showing 'Node.js - eventloop utilization' over time. The graph clearly shows utilization around 0.95-1.0 (near 100%) that dramatically drops to approximately 0.25 (25%) at a specific point. The data is from trigger-replication-prod service on AWS ECS (arn:aws:ecs:us-east-1). The chart covers roughly 16:30 to 17:25 timeframe, with the dramatic drop occurring around 17:00

Categories

Performance Observability Monitoring Backend

Common Tags

Nodejs PerformanceOptimization Logging Monitoring GrafanaDashboards PerformanceTuning Observability AWS Backend DeveloperHumor

Comments

30
Anonymous ★ Top Pick console.log('why is the event loop blocked?') was literally the answer to its own question
  1. Anonymous ★ Top Pick

    console.log('why is the event loop blocked?') was literally the answer to its own question

  2. Anonymous

    The fastest code is the code that doesn't run. The second fastest is the code that doesn't log every single action to stdout in a blocking, single-threaded event loop

  3. Anonymous

    Turns out our hottest code path wasn’t business logic at all - it was printf debugging in production

  4. Anonymous

    The classic senior engineer moment: spending weeks optimizing database queries and implementing caching layers, only to discover your junior's console.log('HERE!!!') in a hot path was the actual bottleneck. Sometimes the most sophisticated monitoring systems just reveal the most embarrassing truths

  5. Anonymous

    Ah yes, the classic 'console.log() as a service' architecture pattern. Nothing says 'I care about observability' quite like synchronously blocking your event loop to write to stdout that nobody's reading anyway. At least now they have empirical proof that their logs were more expensive than their actual business logic - a 4x performance improvement just by admitting that maybe, just maybe, logging every single event in a high-throughput Node.js service wasn't the architectural flex they thought it was. The real kicker? Somewhere, a senior architect is updating their resume to include 'Achieved 300% performance improvement through strategic I/O reduction' while conveniently omitting the part where they were the one who approved logging everything in the first place

  6. Anonymous

    Profiler optional: in Node, grep -v 'console.log | commit' is the real hot path optimizer

  7. Anonymous

    When deleting console.log buys you 60% headroom, you didn’t have a Node service - you had a log shipper with a side hustle as an API

  8. Anonymous

    If your hottest code path is console.log(), you didn’t build a microservice - you built a diary. Deleting prose dropped ELU by 60%; turns out the event loop hates storytelling

  9. @Aqualon 10mo

    logs are always really heavy, aren’t they?

    1. @chupasaurus 10mo

      Only for Node😂

    2. @pulsar_sp 10mo

      same thought still, 3/4 of the load looks hilarious

    3. @dsmagikswsa 10mo

      shouldn't it an async process?

  10. @draemort 10mo

    I hope they don't mean console.log calls

    1. @pnlt_s 10mo

      You're saying it like there's a library dedicated just for logging

      1. @azizhakberdiev 10mo

        it's node

  11. @mrYakov 10mo

    probably not because of logs itself, but because of custom formatting of nested structures

  12. @casKd_dev 10mo

    good logging is cheap

  13. @Valithor 10mo

    Async or not doesn't really matter

  14. @azizhakberdiev 10mo

    there are libraries for importing libraries

    1. @TheFloofyFloof 10mo

      C has those too

  15. @digital_insanity 10mo

    But... don't you need logs?

    1. @azizhakberdiev 10mo

      there are different ways of doing that

  16. @digital_insanity 10mo

    I mean To utilize event loop isn't necessarily bad thing, is it? JS boys, correct me if I'm wrong

    1. @ZgGPuo8dZef58K6hxxGVj3Z2 10mo

      The event queue is the main thread. If it is 100% utilized on the core it runs on then even if you get more requests you wont be able to utilize the other cores even if you have like 255 more not being used TLDR: major bottleneck

      1. @digital_insanity 10mo

        But...wait Isn't that a work of scheduler to ensure that tasks are dispatched to multiple threads? Sorry if noob question, I'm from .NET background

        1. _ 10mo

          There's some tasks you can only do from the main thread (typically UI stuff). Then it's the programmer's responsibility to make task that can only run on the main thread as small as possible, while allowing other work to run on other threads

          1. _ 10mo

            That's for .NET / C# / Java and the like actually. If we're talking about JavaScript, in fact it's single-theaded and more or less will always be

        2. @ZgGPuo8dZef58K6hxxGVj3Z2 10mo

          An event queue is always single threaded and JavaScript isnt even multithreaded it can only handle async stuff but async isnt multithread its still even queue based

  17. @deadgnom32 10mo

    yes

  18. @hillary314 10mo

    You can use web workers to simulate multithreading but it's overkill for logs, doing logging async is asinine imho, they must have used something like colored logs or some other bs

More like this

More like this in Performance →
Performance Nodejs PerformanceOptimization
Stuffing Kimi K3 2.8T Into a 3090

Stuffing Kimi K3 2.8T Into a 3090

AI ML
Rust vs HTML5 Memory Safety Comparison Table

Rust vs HTML5 Memory Safety Comparison Table

Languages
Opus 5 Peaks at Medium, Then Overthinks

Opus 5 Peaks at Medium, Then Overthinks

AI ML
Use J and K for navigation