The Three Circles of Testing Hell: SWE, ML Engineering, and ML Research

Level 1: Oil Change? What’s That?

Imagine three friends with three very different approaches to taking care of their cars:

• Friend A treats their car like a baby. They do all the check-ups: regular oil changes, tire rotations, brake inspections – the works. Before a road trip, they’ll check every light and fluid. As a result, their car runs perfectly with no warning lights on; everything is smooth and reliable.

• Friend B is a bit more casual (maybe too casual). Their car has the “check engine” light on and maybe the tires are a little flat. The engine’s making a weird noise, and there’s some smoke coming out from under the hood. It’s like one thing after another is wrong. But when you ask about it, they just shrug and say with a straight face, “Eh, it’s fine. It still runs, doesn’t it?” They’re basically ignoring the problems and hoping for the best, driving around with issues that really should be fixed.

• Friend C is on another planet when it comes to car care. They’ve never even thought about maintenance. Oil change? Tire pressure? What are those? The car is just a thing to get from A to B until it breaks. So when Friend C hears that Friend A and B do things like oil changes or check their engines, they respond in genuine surprise: “Wait, you guys are actually doing maintenance?!” They had no idea that was something people normally do. They’ve been driving their car without any check-ups at all.

Now, the funny part of this scenario comes from the contrast:

• Friend A’s situation is ideal (like a perfect, well-tested code with all green lights).
• Friend B’s situation is problematic (lots of red flags, literally the car is kind of on fire) but they pretend it’s normal.
• Friend C’s attitude is complete obliviousness (they don’t even know what a check-up is, so they’re astonished others bother with it).

We find it humorous because each friend thinks their approach is normal, but seeing them side by side, it’s clear how absurd it is to ignore maintenance or not even know about it. It’s just like the meme: one world where everything is carefully checked, one where problems are ignored, and one where checks aren’t even on the radar. The joke is a reminder that taking care of things (whether cars or code) makes a big difference – and it’s poking fun at those who say “This is fine” in a burning car or are shocked that anyone else changes the oil!

Level 2: Testing Culture Shock

Let’s break down what’s going on in simpler terms. This meme compares three coding worlds – each with a different attitude towards testing and quality checks:

• Normal Software Engineering: Think of a typical app or library maintained by professional software developers. When they write code, they also write tests (small programs to check that the code works correctly). These tests are run automatically by something called a CI pipeline (Continuous Integration pipeline) whenever they make changes. In the first panel, we see a screenshot (likely from GitHub) saying “All checks have passed – 58 successful checks.” Those “checks” are automated steps in the pipeline. For example:

  • Unit tests (pytest, etc.): These are programs that test individual components. If a function or module is broken, a unit test will fail.
  • Doctests: These run examples in the documentation to ensure the docs’ code examples actually produce the output they claim. It’s a way to keep documentation truthful.
  • Multiple environments (Ubuntu 20.04 with Python 3.7, Ubuntu 18.04 with Python 3.6, etc.): They test the code on different operating systems or Python versions to make sure it works everywhere (“works on my machine” isn’t enough – they verify it works on other machines too!). So they might run the whole test suite on Ubuntu 18.04, 20.04, on Python 3.6, 3.7, etc.
  • Build Docker: Likely they attempt to build a Docker image of the project to ensure the software can containerize properly. Docker images are often used to deploy applications, so building them early catches any issues with that process.
  • Check Code Format (isort): Linting and formatting checks. isort is a tool that automatically sorts import statements in Python. There are other linters like flake8 or formatters like Black. These checks enforce a coding style and catch simple errors (unused variables, etc.). In that first panel, “Check valid import formatting with isort” passed in 10 seconds – a quick style check.
  • Docs check (sphinx): Sphinx is a documentation generator. This step ensures that the documentation can be built without errors (like broken links or syntax issues in doc files).
  • Install pkg: This likely means they test that the package can be installed cleanly in a fresh environment (for example, using pip). It’s a sanity check that they haven’t forgotten to list a dependency or broken the installation process.

  So, in normal software, every push or pull request triggers these checks. A green checkmark (✅) means success, a red X means failure. Green across the board (like in the meme) is what you want before merging code or making a release. It’s a badge of honor for developers to say “All tests pass.” If any test fails, developers usually stop and fix it. Often, teams configure their systems so that you literally cannot merge code until all checks are green. This culture ensures high CodeQuality and reliability – things rarely break unexpectedly because they’ve been thoroughly tested every step of the way.

• ML Engineering: This is a newer field where people are writing code to train and deploy machine learning models (like training a neural network to recognize images, then serving it in a web service). They do use CI/CD and tests, but the meme humorously shows that it’s often a mess. In the second panel, we see the “This is fine” dog sitting calmly while the room (the CI pipeline) is on fire (failing). The labels around the dog are:

  • tests failing – meaning some unit tests or integration tests did not pass (red X).
  • lint failing – code style checks failed, maybe someone didn’t run the linter or there’s a stylistic issue not fixed.
  • TPU tests failing – TPUs are Tensor Processing Units, special hardware by Google for accelerating ML. If a test specifically aimed at checking the code on a TPU is failing, perhaps some model doesn’t run on that hardware correctly.
  • coverage 73.00% – Test coverage is a metric telling what percentage of the code is covered by tests. 73% means roughly 27% of the code isn’t even tested by any test case. In many software teams, 73% would be considered low (they might expect, say, >90%). Here it’s shown in red as if this is below some required threshold.
  • build failing – maybe the Docker build or packaging step didn’t succeed (couldn’t build the project, or a deployment step failed).
  • docs failing – generating or checking documentation had errors (maybe the documentation has issues or wasn’t updated).

  Essentially, almost every check is failing, yet the dog says “This is fine.” This implies that the ML engineer is accepting a situation that a normal engineer would consider disastrous. This is funny because it’s a contrast in standards. It’s like showing someone driving a car with the engine light on, flat tires, smoke from the hood, saying “no problem, all good!”

  Why would ML engineering be more chaotic? For someone newer to this:

  • ML projects involve a lot of experimentation. Code changes frequently to try new models or tune parameters. This can make it hard to maintain tests – a test might break because the expected output changed when the model improved, and updating the test constantly is a pain. So sometimes they just let the test fail and note “we’ll update or remove that test later.”
  • Some ML code is hard to test in small pieces. For example, how do you unit-test a function that trains a neural network for 5 hours? You can’t easily; instead you do a full integration test (train on a small sample and see if it learns at all). Those tests can be finicky or slow, so teams might not write many of them.
  • Time pressure and different priorities: If your boss cares more about “did the model’s accuracy improve?” than “do we have 100% tests?”, the team will focus on the former. They might treat CI failures as mere warnings, not stop signs.
  • Tools like GitHubActions or Jenkins are used, but they may not be fully integrated into the team’s culture. Perhaps only one person checks the CI, or it’s not enforced. It might be a relatively new addition to a team that historically didn’t have it.

  The phrase MLOps (Machine Learning Operations) refers to efforts to make ML projects as streamlined and reliable as traditional software projects: version control, testing, continuous integration, proper deployments. The meme suggests that MLOps is still maturing – ML engineering hasn’t completely caught up to the rigor of classic software engineering yet, thus the half-broken pipelines.

• ML Research: Now, this is the world of researchers (like data scientists in R&D or academic labs). The third panel’s caption “You guys are getting tests?” is a spin on a meme to highlight that in research, writing tests or having CI is exceedingly rare. If you’re new to programming, imagine this scenario: In a university lab, a PhD student writes some Python code to try a new algorithm. They run it, tweak it, run it again, and eventually get a result (say a new record accuracy on a benchmark). They publish a paper about it. The code might be thrown together quickly, possibly in a single notebook or script, just to get the result. They rarely write additional functions or break it into well-structured modules, and they almost never write unit tests for it. Why? Because their goal is insight and results (the accuracy number, the graphs) – the code is a means to an end. As long as the final run produces the graph for the paper, they’re happy. There’s no customer using the code, no plan to maintain it long-term (beyond maybe another student trying to replicate it). So, testing and QA are not rewarded in that environment.

  The meme’s researcher is basically saying, “Wait, you actually have tests for your code?!” like it’s an unheard-of luxury. In some way, it’s poking fun at researchers for being a bit oblivious to good engineering practices. But it’s also understandable – if you’ve only ever worked in research codebases, you might never have seen a comprehensive test suite or a CI pipeline. The incentives are different: professors and researchers care about publications, not whether your codebase has 100% test coverage. In fact, spending time writing tests might even be seen as a distraction from doing more experiments. (Of course, this is a generalization – some research groups do value good coding practice, but the stereotype exists for a reason.)

So, to recap in simpler terms: the meme compares strict vs. lax vs. nonexistent testing practices:

• In normal software, testing and CI are first-class citizens. Everything is monitored with version control and automated pipelines. Green badges everywhere means the code has passed all automated quality gates.
• In ML engineering, there’s some testing, but it’s a bit of a dumpster fire 😅. They have pipelines, but it’s pretty common to see some red badges and failing checks that everyone kind of ignores for the time being. It’s a phase many teams go through as they try to bring more discipline to an ML project that maybe started more loosely.
• In ML research, the concept of automated tests is almost alien. The process is often manual: run code, see result, tweak, repeat. If the researcher in the meme is a stand-in for, say, a grad student, their idea of “testing” might just be “run the experiment on a known dataset and see if the accuracy is high.” They wouldn’t write a separate program to validate each function – they test by observing outcomes and plotting graphs, not by writing assert statements in code.

For someone newer to development, this meme is highlighting a kind of culture shock you might encounter. For example, perhaps you learned programming in a data science course where you never wrote a formal test. Then you join a software company and suddenly every time you push code, dozens of tests run and your code won’t be accepted if any fail – that’s a big adjustment! Conversely, a software engineer might join an AI research lab and be shocked that people commit directly to main with no review or tests at all. The meme uses humor to say “look, these worlds operate differently.”

Also, note the memes within the meme:

• The second panel’s “This is fine” dog is a classic meme indicating denial or ignoring problems. It’s saying the ML engineers are basically acting like everything’s fine despite clear signs it’s not (failing tests).
• The third panel references the “You guys are getting paid?” meme. In that original, an underprivileged character is surprised others got something he didn’t. Here, it perfectly conveys the surprise of a researcher realizing that in other areas of programming, tests are a given (implying he never got any).

All the tags like BuildPipeline, TestingHumor, AIHumor, GitHubActions connect to what we’ve described:

• GitHub Actions is likely what’s running those 58 checks (it’s a popular CI service integrated with GitHub). So, panel1 is basically a GitHub Actions success screen.
• CICD (Continuous Integration/Continuous Deployment) and Build Systems are what orchestrate these tests and builds.
• Testing and UnitTesting are about writing those asserts and checks.
• TestCoverage (73% seen in panel2) is a number indicating how much of the code is exercised by tests – a common quality metric.
• CodeQuality includes things like linting and maintaining good standards, which clearly differ across the panels.
• MLOps is the practice trying to unify these worlds: bringing software engineering rigor (panel1) to ML (panel2, and maybe panel3).

By explaining all this, we demystify the joke: It’s funny because each panel resonates with a truth about how code is handled. If you’ve never seen CI or tests, now you know what those are (and you might chuckle that anyone would let them fail). If you only know rigorous processes, now you see why some ML projects might seem chaotic. And if you come from a research background, well, you might realize why engineers chuckle about your lack of tests – it’s just a totally different mindset.

Level 3: A Tale of Three Pipelines

This meme is a tongue-in-cheek take on the CI/CD pipeline culture across three different programming domains: standard software engineering, machine learning engineering, and machine learning research. Each panel shows a wildly different attitude toward testing and code quality. Experienced developers and MLOps folks laugh (perhaps a bit bitterly) because they’ve seen or lived these differences:

• Panel 1 (Normal software engineering code): The first image isn’t a joke; it’s basically an ideal reality in many mature software projects. It shows a GitHub pull request with a wall of green checkmarks: “All checks have passed – 58 successful checks.” This is a familiar sight in well-run projects using Continuous Integration (often via GitHub Actions or Jenkins). There are dozens of automated jobs: unit tests on multiple OS versions and Python versions, a docker image build, running full pytest suites, checking code formatting with isort, building documentation with Sphinx, etc. Seeing 58 checks all green signals a high bar for quality – nothing gets merged unless everything passes. Senior devs know each of those checks is a safeguard: tests prevent bugs, linters enforce consistency, docs builds ensure the documentation isn’t broken, etc. Achieving all-green can be non-trivial for large codebases, so it’s a point of pride. The humor setup here is that this level of rigor is labeled “normal” – implying, this is how software engineering should be. It’s a subtle flex: CodeQuality is taken so seriously that anything less than 100% passing is failure.

• Panel 2 (ML Engineering code): Now we descend into what many industry folks will recognize as reality on ML-centric teams. The picture used is the famous “This is fine” meme – a dog calmly sipping coffee while flames rage around. Except here, the flames are replaced with red failure badges: “tests failing,” “lint failing,” “TPU tests failing,” “coverage 73.00%,” “build failing,” “docs failing.” In a normal software project, even one red X would sound alarms and halt the release. But in some ML engineering projects, a bunch of failing checks might barely get a shrug – the team carries on as the dog does, pretending “This is fine.” It’s funny because it’s true: ML engineers often grapple with a much less stable pipeline. Why? A few typical reasons a senior dev would know:

  • Flaky tests: ML code involves randomness (initial weights, random shuffling of data) and external data dependencies. Tests that rely on training a model might pass one day and fail the next (e.g., if a model’s accuracy is 0.730 one run and 0.740 the next, a test expecting >=0.75 fails intermittently). Over time, teams might ignore these failures as long as the important parts (like final model metrics) look okay. The result is a perpetually “red” CI that people stop paying attention to.
  • TPU tests failing: This suggests they run tests on specialized hardware (Google’s TPUs) to ensure the code works on that accelerator. It’s not uncommon for something to work on a CPU/GPU but break on TPU due to hardware-specific issues (like precision differences or unsupported operations). Fixing TPU-specific bugs can be hard without access to the same environment, so such failures might be backlog-ed while the team focuses on more immediate deliverables. Meanwhile, the CI keeps reporting them as failing.
  • Lint failing: Lint checks (style checks, like PEP8 compliance) might be considered “nice to have.” When rushing to train models and hit performance numbers, ML teams might deprioritize fixing minor styling issues. A seasoned engineer has likely seen PRs where the code works but the linter flags unused imports or long lines – instead of polishing it, the team might merge anyway "for now," planning to clean up later.
  • Coverage 73.00%: Many software teams enforce a coverage threshold (often 80% or 90%). Seeing 73% would usually block a merge. In ML projects, though, you often find lower coverage. Some code paths (especially error handling or rare data conditions) are left untested, and writing tests for complex data pipelines is time-consuming. Teams under pressure may accept lower coverage. 73% in the meme is probably an optimistic number for comedic effect – plenty of ML projects hover in the 50-60% range or don’t measure coverage at all.
  • Build/docs failing: Documentation might be auto-generated (like API docs from docstrings) and can break if docstrings are malformed. Similarly, the build (packaging the project, etc.) can fail if dependencies changed. In a strict software team, these failures stop the line. In an ML team, they might be seen as “non-critical” – after all, the model still trains, right? So those red badges accumulate and everyone kind of collectively ignores them while focusing on the next experiment or model tweak.

  The meme exaggerates for effect (hopefully no real team has all those failures at once… or do they? 😅). But the core joke lands because many of us have experienced this gap in discipline. The phrase “ML Engineering” here implies these folks do have a pipeline and some tests – they are trying to be like software engineers – but the culture hasn’t fully caught up. It’s a classic MLOps problem: how to get research-y code to meet production-grade reliability. Everyone knows it should be better (hence the irony of the dog saying it’s fine when it’s not). The humor has a dark truth: ignoring failing tests is dangerous. A cynical veteran might quip, “we’ll regret those ignored failures at 2 AM in production.” Yet it happens, often because short-term goals (deliver the model, hit the accuracy target) trump long-term quality. The result is accumulating technical debt – messy code, brittle pipelines – which echoes the famous truism: “Machine Learning code is the high-interest credit card of technical debt.” In other words, that burning room can explode later. Seasoned engineers cringe-laugh at this panel because it’s TestingHumor that hits home – it’s both funny and a bit painful.

• Panel 3 (ML Research code): Here we reach rock bottom (or total freedom, depending on perspective). The image is the meme from “We’re the Millers” where a character incredulously says, “You guys are getting paid?” It’s repurposed to “You guys are getting tests?” The ML researcher – often a PhD student or research scientist – is portrayed as having zero awareness of testing. This is a playful jab at the research community’s notorious lack of engineering rigor. Why would a researcher be surprised that people write tests? Because in many research projects, unit tests and CI are virtually non-existent. Code is just a means to an end (the end being a paper or a proof-of-concept result). It’s typically run in notebooks or scripts on some lab server or Jupyter notebook, and as long as the graphs and metrics look good, nobody writes additional verification code. The meme exaggerates the obliviousness: “You get tests?” as if tests were a luxury or salary bonus that others receive and the researcher doesn’t. This echoes common industry banter: software engineers teasing that data scientists “live in Jupyter notebooks with copy-pasted code” or researchers delivering code that only works on their machine, once, for the figures in their paper.

Experienced folks know real examples: maybe you’ve tried to use an open-source implementation of a research paper and found it broken, undocumented, or producing different results because it wasn’t thoroughly tested. Or a research colleague hands over a model that works in their environment but has no CI, no version pinning for dependencies, so getting it running elsewhere is a nightmare. It’s so common that this meme elicits a knowing chuckle. In fact, bridging this gap is a whole job these days – MLOps engineers or “Research Engineers” often take that rough research code and refactor it, add tests, add CI, etc., to make it usable in a production setting.

To summarize the senior perspective: The meme is funny because it’s painfully accurate about the cultural divide: - Traditional software engineering: obsessed with reliability, testing, and process (green means 👍). - ML Engineering: caught between scientific experimentation and productization – some process exists, but it’s often bent or broken (red marks everywhere, yet “oh well”). - ML Research: moves fast and breaks things (often deliberately ignoring process), focusing purely on experimental outcomes (no tests in sight and surprised anyone would bother).

It highlights an MLOps truth: as you move from research to production, you must travel the long road from “no checks” to “some checks (on fire)” to “all checks green.” Each panel is like a snapshot of that journey (or a comparison of three distinct cultures). For a seasoned dev or researcher, it’s both relatable and a satirical reminder: we should probably aim to be more like Panel 1, but reality often dumps us in Panel 2, and we all started in something like Panel 3 at one point.

To drive the point home, consider this comparison that a senior engineer or tech lead might make:

Aspect	Software Engineering 🚀	ML Engineering 🔥	ML Research 🧪
CI pipeline	Strict and always green. All merges blocked until every check passes (tests, build, etc.).	Pipeline exists but often red. Failing tests are sometimes ignored or quarantined (“it’s fine, we’ll fix later”).	Rarely any CI at all. Code run in notebooks or ad-hoc scripts; no automated pipeline.
Tests & Coverage	Extensive unit tests + integration tests. Coverage ~90% or higher is expected.	Some tests, focusing on critical parts. Coverage often lower (50–80%). Tests can be flaky due to randomness.	Little to no testing. Maybe a few manual tests or sanity checks. Success is measured by experiment results, not test coverage.
Code Style & Lint	Enforced by linters (e.g. PEP8, flake8, Black, isort) on every PR. Clean, consistent code is a must.	Linting is set up, but not always followed strictly. In crunch time, style checks might be skipped. Code may have more TODOs, commented-out sections, etc.	Style is informal. Might be a mixture of prototypes and hacks. If it runs and plots the graph, it’s “good enough.” No one is running a linter here.
Documentation	Well-documented. Uses tools like Sphinx to auto-generate docs. PRs require updating docs and docstrings, which CI verifies.	Basic docs exist (a README, some docstrings), but often out-of-date. Doc build might fail if neglected. Not the top priority with looming model deadlines.	Minimal documentation, if any. Perhaps a README guiding how to run the experiment. Knowledge often resides only in the researcher’s head or an associated paper’s appendix.
Main Focus & Incentives	Delivering a maintainable, scalable product. Bugs and downtime are costly, so process and quality are paramount. Incentive: user satisfaction, reliability.	Delivering a working model to production quickly. Balancing model accuracy with engineering best practices. Incentive: beating the competition to a better model, while keeping the system running “well enough.”	Proving a hypothesis or achieving state-of-the-art results. Code is a vehicle for research ideas. Incentive: publications, discovery. Clean code is secondary.

Every senior engineer reading that table is nodding (or smirking) because they’ve lived these scenarios. The meme humorously condenses it: from “all checks passed” nirvana down to “you have checks?” chaos. In practical terms, the meme is a lighthearted call for better MLOps: how do we get our ML pipelines out of that burning house and inch them closer to the green-check utopia? And it’s a reminder that your perspective on testing depends on where you sit: a platform engineer might be horrified by a 73% coverage, whereas a researcher might think 73% coverage is astonishing (since they usually have 0%!). That contrast – between what’s normal in one field and unheard of in another – is exactly why this meme is funny for those in the know.

Finally, to illustrate the difference in tests that might run in these scenarios, consider how a simple code function vs. an ML model might be tested:

# Traditional software unit test (deterministic, straightforward)
def add(a, b):
    return a + b

def test_add():
    assert add(2, 3) == 5  # expected output is exactly 5, always

# ML code test (probabilistic, can be flaky)
def test_model_accuracy():
    model = train_model(training_data)       # train a model (could involve randomness)
    acc = model.evaluate(test_data)          # evaluate accuracy on hold-out data
    assert acc >= 0.90                       # expect at least 90% accuracy

# The model training might yield 89% one run and 91% the next due to randomness.
# So this test could fail occasionally, even if nothing is "wrong".

In a strict software project, a test that passes only sometimes is unacceptable – you’d fix the cause or adjust the test. In ML, this kind of test (asserting a performance threshold) is common, and teams learn to live with a bit of instability or add code to make tests less flaky (like setting random seeds, or using larger sample sizes to reduce variance). The meme’s ML Engineering panel essentially shows a team that has such tests but is half-ignoring their red results, whereas the research panel doesn’t have them at all.

All in all, this level dives into why the meme’s scenario exists. It’s not just poking fun; it’s reflecting real challenges:

• Ensuring high code quality in ML is harder than in traditional software because of the unpredictable, data-dependent nature of the work.
• There’s a historical culture gap: software engineering matured with practices like testing and CI, while ML grew out of research and academia, where those practices were not initially common.
• The industry is now trying to enforce more “software-like” discipline on ML projects (hence the rise of jobs in MLOps and tools to automate model testing, data versioning, etc.), but as the meme jokes, it’s a work in progress.

The humor lands with senior devs because they appreciate the irony: you can have all the fancy GitHubActions pipelines in the world, but if the team doesn’t treat red flags seriously (or never set them up in the first place), it’s like having a fire alarm unplugged while the kitchen is burning. It’s a cautionary laugh – we laugh because it’s true, and maybe because we’ve all been that dog saying “This is fine” at one point in a project 😅.

Level 4: Testing the Un-Testable

At the most theoretical level, this meme highlights a fundamental quality assurance paradox between traditional software and machine learning code. In deterministic software, we can specify exact expected outcomes for given inputs – every feature or function can be unit-tested with a known oracle (the correct result). We strive for provable correctness or at least exhaustive test cases. In contrast, machine learning (ML) introduces probabilistic behavior. An ML model’s “correctness” is usually measured statistically (e.g. 95% accuracy on a validation set) rather than absolutely. There is no simple oracle for each individual input – the "oracle" is often the real world or a distribution of data, which is fuzzy. This makes traditional testing tricky: how do you write a unit test for “recognize all cats correctly”? You can’t enumerate all possible cat images, so you rely on sample datasets and metrics. In formal software testing terms, ML faces the test oracle problem – we often cannot know the expected output for arbitrary input, only that aggregate performance meets some threshold.

To cope with this, quality assurance for ML turns to statistical methods. Instead of a single pass/fail assertion on one input, we evaluate on large test sets and assert properties about distributions of outputs. For example, we might assert “the model’s accuracy on 1000 test images is at least 90%”. But even that introduces uncertainty: run the same training process twice and you might get 89.7% one time and 90.3% the next due to randomness (initial weights, random data augmentations, nondeterministic hardware optimizations, etc.). Traditional CI systems, which expect consistent pass/fail results, struggle with this inherent variability. It leads to flaky tests in ML – tests that sometimes pass and sometimes fail even if nothing is “wrong” per se. This could be because a model just barely misses an accuracy cutoff on a given run. These sporadic failures can turn a CI pipeline into a false alarm generator, which teams may start to ignore. The meme’s second panel (the “This is fine” dog amid failing checks) satirizes this exact scenario: the ML engineering team sees red failing tests but knows they might be due to expected variance or minor issues, so they carry on as if “all is fine.”

From a research perspective, things get even more Wild West. Academic or exploratory ML research code often pushes the frontier of algorithms without well-defined expected outcomes – the goal is to discover new behavior, not conform to a spec. It’s not just untested, it’s untestable in a formal sense because researchers themselves might not fully know what output to expect until they run the code. The ultimate “test” is whether the experiment yields a publishable result or a new insight. This leads to a culture where rigorous software process is viewed as overhead. Why write unit tests for code that will be thrown away if the idea doesn’t pan out, or completely rewritten for the next experiment? The meme’s third panel (“You guys are getting tests?”) captures this mindset: the ML researcher is genuinely surprised that anyone bothers with tests at all, since in their world, success is measured by papers or benchmarks, not by a green build badge.

Ironically, the lack of tests and process can hurt scientific reproducibility – a hot topic in ML research. Many published results are hard to reproduce because the accompanying code (if released at all) is messy, environment-specific, and undocumented. Ideally, even research code would have certain checks: for example, verifying that results are statistically significant and not a fluke. There’s ongoing work on bringing rigor to ML: frameworks for reproducible experiments, seeds for randomness control, and even research into metamorphic testing for ML (where instead of expecting a specific output, you validate that the model’s output changes in expected ways when you modify the input – e.g., adding noise to an image shouldn’t wildly change a classification result). Advanced methodologies like property-based testing or formal verification haven’t fully caught up to complex ML systems yet, but they’re being explored — for instance, attempts to formally verify parts of neural networks (for safety or fairness guarantees) or to use symbolic techniques to prove certain network properties. However, these are cutting-edge approaches and far from standard practice.

In summary, the meme humorously exposes a deep truth: classic software engineering and ML live on different ends of a “testability” spectrum. Traditional apps can be made predictable and thoroughly verifiable; ML models always retain some uncertainty. This isn’t just a cultural gap, but almost a mathematical one. As a result, Continuous Integration (CI) in ML requires redefining “pass/fail” (allowing tolerance, statistical checks, retraining with fixed seeds, etc.), and many teams haven’t fully solved this. Thus, we get the absurd scenario depicted: the farther you go from straightforward software (Panel 1) into the realm of experimental AI (Panel 3), the more the classical idea of exhaustive testing melts away – going from green to red to nonexistent. The punchline lands because anyone experienced in both worlds recognizes this as a painful reality grounded in the very nature of ML algorithms versus deterministic code.