Skip to content
DevMeme
3962 of 7590
OnCall ProductionIssues Post #4312 · source on Telegram

When your only backup lives on the server that just crashed

Description

Two-panel meme using a scene from a noir-style movie set on a ferry deck. Panel 1 shows two men in trench coats and fedoras facing the sea; white text at the top reads "Server has crashed", and centered text below them asks "Where is backup?". Panel 2 zooms in on one man’s skeptical face; the overlaid text answers "On the server". The joke highlights a disastrous backup strategy where the backup is stored on the same machine that failed, a common operations anti-pattern that leads to extended downtime and frantic on-call firefighting

Comments

11
Anonymous ★ Top Pick Ops said we had a 3-2-1 backup strategy; nobody asked if the “2” referred to two folders on the same ext4 partition
  1. Anonymous ★ Top Pick

    Ops said we had a 3-2-1 backup strategy; nobody asked if the “2” referred to two folders on the same ext4 partition

  2. Anonymous

    The same engineer who convinced management that RAID 1 counts as a backup strategy just discovered why "geographically distributed" doesn't mean different racks in the same data center

  3. Anonymous

    Schrödinger's backup: every backup is simultaneously valid and useless until you attempt a restore - colocating it with prod just collapses the wavefunction early

  4. Anonymous

    Ah yes, the classic 'backup stored on the same RAID array' architecture - because why have a disaster recovery plan when you can have a disaster *discovery* plan? This is the infrastructure equivalent of keeping your spare tire inside the car that's currently on fire. Any architect who's been through a 3am data center outage knows that moment of existential dread when you realize your backup strategy was 'hope and pray' disguised as 'rsync to /mnt/backup'. The real kicker? This usually gets discovered during the RCA when someone asks 'but we had backups, right?' and you watch the color drain from the senior engineer's face as they realize the S3 bucket was in the same region that just went down

  5. Anonymous

    HA cluster of one: resilient until the kernel panics

  6. Anonymous

    Nothing says enterprise maturity like a DR runbook that points to /mnt/backup on the box that just kernel-panicked - RPO: yesterday, RTO: never

  7. Anonymous

    We nailed 3‑2‑1 backups: 3 copies, 2 directories, 1 power supply

  8. @cptnBoku 4y

    Lmao

  9. @cptnBoku 4y

    AWS down But can't get notification because it was hosted on AWS

    1. @TERASKULL 4y

      wanted to check status of server, but status page is hosted on the server

  10. @ZgGPuo8dZef58K6hxxGVj3Z2 4y

    Its normal

Use J and K for navigation