Brain Won't Let You Sleep: What If Monitoring Is Down Too?

Description

A four-panel meme using the 'Brain at bedtime' template. Top-left panel: an anxious pink cartoon brain with wide eyes asks 'WERE THERE ANY MONITORING NOTIFICATIONS'. Top-right: a person lying in bed peacefully responds 'NO'. Bottom-left: the brain, now with an even more alarmed expression, asks 'WHAT IF MONITORING IS DOWN TOO?'. Bottom-right: the person is now wide awake in the dark with huge terrified eyes, clearly unable to sleep. The 'imgflip.com' watermark is visible in the bottom left. This meme captures the existential dread of on-call engineers who realize that the absence of alerts might not mean everything is fine -- it could mean the alerting system itself has failed, which is arguably worse because you have zero observability into your systems

Common Tags

Monitoring OncallLife Observability AlertFatigue SystemReliability Devops ProductionIncidents

Comments

Anonymous ★ Top Pick Quis custodiet ipsos monitores? The answer is another monitoring system that nobody set up alerts for

Anonymous ★ Top Pick

Quis custodiet ipsos monitores? The answer is another monitoring system that nobody set up alerts for
@hoodiepaws 5mo

been there
@victoriaSome 5mo

Just don't release on Friday!
1. @SamsonovAnton 5mo
  
  I'll be careful, I promise! ©
2. @realVitShadyTV 5mo
  
  I’ll gona release right now and start me weekend
@SheepGod 5mo

Ah working on email issue monitoring tool rn
@D13410N3 5mo

Literally me
@NaNmber 5mo

well, it's 5 am on Saturday , time to merge I guess
1. @NaNmber 5mo
  
  ah shit, here we go again...
  
  @NaNmber 5mo
  
  ok gg no way I'm fixing it now 🙂