The Absurd Pace of AI Progress

Level 1: The Uninvited Winner

Imagine you set up a puzzle contest between people and robots. 🍏 A big company (let’s call them Apple, like the fruit) says, “Look, people are much better at these simple puzzles than our best robot. See? The people solved 92 out of 100 puzzles, but the robot only solved about 70 out of 100. Humans win!” They feel pretty good showing that humans are still champs at these little games.

But here’s the funny twist: the company forgot to invite one more robot to the contest – let’s call this robot o3. When someone finally lets o3 try the same puzzles, o3 solves 96 out of 100 – even more than the humans did! 🏆 In fact, o3 is so good it beats the human score easily, like it wasn’t even hard. It’s as if the company was saying “No robot can beat us at this!” and then o3 walks in late and says, “Hey, I can do it better than you guys.” Surprise!

This is funny because it’s like a teacher announcing, “None of the students can solve this riddle,” but then a new kid (who wasn’t in the class earlier) shows up and solves the riddle without breaking a sweat. The big company was trying to show how limited robots still are, but they ended up getting surprised by a robot they didn’t test. In simple terms: they spoke too soon! Everyone chuckles because the unexpected winner (the o3 robot) proved the boast wrong. It shows how fast things can change – one moment you think humans are ahead, the next moment a machine comes out of nowhere and steals the show. And that makes it a little bit funny and a little bit amazing at the same time.

Level 2: Puzzle Performance Primer

Let’s break down what happened in plain terms. Apple (yes, the iPhone and Mac Apple, which also does a lot of secretive AI research) published a study about AI limitations. In this study, they gave both humans and AI systems a set of little puzzle tests – things that are pretty easy for people, to see if AI can solve them too. These puzzles included:

• Perspective taking (PTT): For example, a question might say “Pretend you are standing at X and facing Y. What angle is object Z relative to you?”. You’d need to imagine yourself in a different position and figure out directions – something humans do with mental pictures.
• Maze completion (MCT): A tiny maze puzzle. Think of a simple blue maze where you have a start and finish; humans can usually find the exit quickly, but can an AI find the path by reasoning?
• Judgement of Line Orientation (JLO): This is a classic visual puzzle where you see lines at various angles and must identify which lines match a given angle. Humans use their visual/spatial skills to judge angles – like a mini game of matching orientations.
• Selective attention (SAtt): Kind of like a “find the thing” puzzle. For instance, a grid of objects (say, apples 🍎 and other items) is shown, and the task asks for the location of a specific item or count of items meeting some criteria. It tests if you can focus on the right details given some clues, ignoring distractions.

Apple’s results showed that humans did really well on these puzzles (no surprise, since they were designed to be easy for humans). On average, people got about 92.7% of the questions right. Now, they also tested a version of GPT-4 (one of the most advanced AI language models, think of it like the brain behind ChatGPT). GPT-4 is usually super smart with language, but on these puzzles it didn’t do so hot – only about 69.9% correct. That’s a big drop from human performance. In simple terms, Apple was demonstrating that “See? Even our best AI today struggles with some basic reasoning puzzles that we find simple.” This was an example of AI research highlighting a gap: the AI might be great at writing essays or code, but ask it to do a little spatial reasoning or visual puzzle in text form and it gets confused. This underscores the AI hype vs reality issue: AIs seem all-powerful, yet they can fail at tasks a child finds easy, reminding us they’re not truly intelligent in the way people are.

Now, here’s where the meme part comes in. A researcher named Dan Hendrycks pointed out on Twitter that Apple’s study missed something important. There’s a newer AI model – referred to as “o3” – that Apple didn’t test. And guess what? When you test o3 on those same puzzles, it scores around 96.5%. That’s even higher than the humans’ 92.7%! 🔥 In other words, the very gap Apple highlighted (humans beating AI) was already closed – and even reversed – by this new model. o3 beat the humans at their own puzzle game.

This is a classic case of the AI world moving very fast. In the time between Apple doing their study and sharing the results, someone built or released a model that’s better at these reasoning puzzles. Apple either didn’t know about it or didn’t include it, so their paper made it seem like “AI can’t do it,” but the reality (as pointed out on Twitter) is “actually, one AI can now do it better than people.” The tweet even compares the scores side by side:

Solver	Puzzle Accuracy
Humans	92.7% 👍
GPT-4 (old)	69.9% 🤖❌
o3 (new)	96.5% 🤖✅

You can see: GPT-4 got about 70%, nowhere near human level on this test. But o3 got an impressive 96.5%, which is above the human average. That means o3 solved almost all the questions correctly – even more than the people did on average. It casually outperformed humans. This was a bit of a “Whoa!” moment.

Let’s clarify a few terms and why this is notable:

• GPT-4: A very advanced AI model developed by OpenAI. It’s great at understanding and generating text, and it has some reasoning ability, but it wasn’t specifically trained for these puzzles. Think of it as a very smart generalist.
• o3: This is the name given to a newer model or version that specializes in reasoning tasks. It’s not a household name like GPT-4, but in research circles it’s apparently making waves. Maybe it’s fine-tuned or designed to handle puzzle-like reasoning better. In any case, it’s the new challenger here.
• Human baseline: In AI evaluations, we often compare how humans do on a task to how AI does. The humans getting ~92.7% is the baseline to beat – the bar for “as good as people.” If an AI goes above that, it’s a headline-worthy achievement (at least for that narrow task). That’s what happened with o3.
• Reasoning evaluation: Apple was evaluating “reasoning” – not just rote question-answering, but the kind of thinking where you might have to imagine something or logically work through a puzzle. These particular tasks test spatial reasoning and attention. They’re like an IQ test for AIs in some sense.

So, Apple’s study was showcasing a limitation: “Aha, here’s something AI can’t yet do as well as us!” It’s a bit of a reality-check against AI hype, showing that AI assistants and models still struggle with some basic cognition. But then the tweet turns it into a bit of humor by saying, “If only they had tried this new model, they’d see it actually does better than humans.” It’s funny because it’s ironic – the whole point of the Apple paper gets kind of undercut by a development they overlooked. In the AI industry, this kind of thing happens a lot: one month you declare a problem, next month someone fixes it. It’s hard to keep up!

This resonates as a form of AI humor among developers and researchers. Everyone shares that feeling of “oh wow, that was fast!” It’s also a gentle jab at Apple: a company known for being on top of things “forgot” to check the latest model. Kind of like a tech version of “you missed a spot.”

In summary, the meme is pointing out that:

• Apple said: Current AI can’t solve these easy human puzzles (humans > AI).
• Reality (very quickly) said: Actually, a new AI can solve them even better than humans (AI > humans now).

It’s a mix of amusement and amazement. Amusement because it’s a bit embarrassing for Apple’s study (imagine publishing something and almost immediately being outdated), and amazement because o3’s jump in performance is impressive — it highlights how quickly AI is advancing in areas we thought we had an edge. Today’s joke, in a way, is that you can’t blink in this field or you’ll miss the next breakthrough!

Level 3: Overlooked Overachiever

In this meme’s scenario, Apple basically got caught publishing yesterday’s news about AI limitations. Their paper proudly pointed out a set of “easy for humans, hard for machines” puzzles – and indeed, they showed humans scoring ~92.7% versus GPT-4’s 69.9% on a custom Apple puzzle benchmark. That’s a significant gap, implying current AI still fumbles on tasks a toddler might ace. The humor hits when Twitter discourse (via Dan Hendrycks’ tweet) says, “Surprise! You forgot to test the latest model, and it casually tops humans.” It’s the classic tech punchline: one more thing was missing, and it was a big thing. Apple, of all companies, left out the “one more thing” – in this case, the o3 model – and that model swooped in with a 96.5% score, dunking on both humans and GPT-4 (GPT-4o). The tweet’s tone is half informative, half gotcha!: it’s poking at Apple’s evaluation methodology for being a bit behind the times. In the fast-moving world of AI industry trends, not checking the newest model is like publishing a smartphone review and ignoring the latest iPhone – a recipe for immediate obsolescence (and a little embarrassment).

So why is this funny (especially to AI folks and developers)? For one, it highlights the AI hype-vs-reality whiplash we face daily. Apple’s paper painted a cautious reality: “look, even GPT-4 struggles with these reasoning puzzles; humans still reign supreme on them.” That feeds a certain narrative in AI research and safety circles: current AIs have glaring blind spots, lacking true reasoning. But reality didn’t sit still. An up-and-coming model turned that into hype again by beating the human baseline. If Apple’s message was “don’t over-hype AI, it can’t do this yet,” the o3 model promptly responded, “hold my 🍎 (apple) – I can do it now.” It’s a comedic reversal where Apple’s claim aged about as well as week-old milk. The FOMO (fear of missing out) in benchmarking is real – nobody wants to be the researcher who overlooked the new champion model. Here Apple inadvertently became that researcher, and the AI community is chuckling because we’ve all seen this before. Today’s limitation might get obliterated by tomorrow’s breakthrough, especially in machine learning.

There’s also an undercurrent of AI humor in how the tweet is phrased and presented. The collage image beneath the tweet is almost meme-like in itself: it shows bits of the actual puzzles (little diagrams and grids) and a results table with “JLO” (Judgment of Line Orientation) and a big green 99.5 score for one model. It’s as if to visually say, “the new model absolutely nailed this test.” Seeing a bright green 99.5 next to human 92 and a poor 50.5 (presumably GPT-4’s JLO score) is absurdly satisfying for those following AI benchmarks. We’re used to hearing “X model now beats humans at Y task” in headlines, so much that it’s become a bit of a trope in MachineLearningHumor. This tweet plays right into that trope knowingly. It’s funny because of the timing and the casual tone: “Oh BTW, o3 gets 96.5%, beating humans.” Casually beating humans – as if it were no big deal, when in reality that’s a pretty big deal! The implication: Apple’s researchers must’ve been momentarily proud of finding a domain where humans > AI, only to discover a new AI went “+1 up” immediately.

From a senior dev or researcher perspective, there’s a lot of subtext:

• Benchmark leapfrog: We recognize a pattern where as soon as someone identifies a performance gap or publishes a benchmark, a flurry of activity (or an existing project like o3) closes the gap. It’s a constant game of leapfrog. You can almost hear seasoned engineers sigh, “Of course a new model already beat it… figures.”
• Testing methodology: Apple’s omission hints at either the new model not being widely known or available during their study, or a deliberate focus on certain models. It raises an eyebrow: did Apple cherry-pick models that would support their point about AI limitations? Or were they just unlucky with timing? In any case, not testing “the latest and greatest” is a faux pas in competitive benchmarking. Every experienced ML engineer knows you check if someone posted a new arXiv paper or GitHub repo claiming better results before finalizing your own claims. Here it looks like Apple’s team either didn’t know about o3 or underestimated it. The tweet’s author (Hendrycks) is basically giving them a friendly ribbing for that oversight.
• Real-world scenario: Imagine this in a dev context: It’s like writing a report, “Our system can’t handle more than 1000 requests/sec,” but forgetting that a patch came out that same week doubling the throughput. Everyone who reads your report goes, “Well actually, if you applied the patch…” 😅. This feels similar. Apple announced “AI can’t solve these human-easy puzzles,” and the community answer is, “Well actually, the new model can.”
• Organizational dynamics: It’s interesting that this comes from Apple, a company known for secrecy and less involvement in public AI bragging contests. So when Apple does publish something about AI, the community pays attention – and apparently isn’t above fact-checking it in real time on Twitter. The meme captures a bit of that big-corp vs open-research dynamic: the open community (or other labs) might move faster in some areas, and a corporate research paper can accidentally look out-of-date by publication time. It’s a subtle poke at how fast the AI industry moves and maybe at Apple’s expense for not keeping up with AI industry trends outside their bubble.

To a seasoned AI developer, there’s also the recognition that beating humans on a benchmark is a moving target. “Humans: 92.7%, AI: 96.5%” – celebrated today, but what about tomorrow? The meme hints that rushing to claim AI limitations can backfire spectacularly. Today’s AI assistants and models might falter on puzzles, but with a bit more data or a clever tweak (hey, maybe just prompt engineering or a fine-tune), they leap ahead. It’s both exciting and comical. As an enthusiastic observer, you can’t help but grin at how AI hype vs reality sometimes flips overnight. Apple said “AI can’t do it”; reality (and Twitter) retorts “Actually, it just did.” 🤖🎉

And of course, let’s not forget the lighthearted Apple vs (orange) o3 pun in the title: comparing apples to… o3s? 😅 In any case, this meme is a nerdy joyride through the rapid progress of AI, and a reminder not to underestimate how quickly a machine learning model can go from zero to hero on a task. One day you’re writing about AI’s failure, the next day you’re writing an erratum because someone’s new model aced it. It’s all in good humor – a shared laugh at how even experts can’t keep up with the breakneck pace of AI progress.

# Pseudo-code of Apple's evaluation pipeline (humorously simplified)
models_to_test = ["GPT-4", "OtherModel_v1"]  # Apple's list of current AIs
scores = {m: evaluate_on_puzzles(m) for m in models_to_test}
print(scores)  
# Output might be: {"GPT-4": 0.699, "OtherModel_v1": 0.505}  (i.e., 69.9%, 50.5%)
# Oops, they didn't include the new 'o3' model in models_to_test...
# ...so they missed that 'o3' would score ~0.965 (96.5%)!

Level 4: Emergent Spatial Reasoning

At the cutting edge of AI research, even simple human puzzles reveal deep technical challenges. Apple’s paper zeroed in on tasks like perspective taking, maze navigation, line orientation judgement, and selective attention – classic spatial reasoning tests. These puzzles might look trivial, but they encapsulate what early AI theorists called Moravec’s paradox: skills that are easy for humans (like visual puzzle-solving) can be hard for AI. Apple observed a large gap – GPT-4 struggled (about 70% correct) while humans breezed through (~92.7%). This gap hints at AILimitations in how current models reason about space and geometry.

Why would a state-of-the-art language model stumble here? It’s partly because transformer AIs like GPT-4 are built for sequential pattern processing (words and tokens), not innate geometric intuition. Solving a judgement of line orientation (JLO) puzzle or a maze requires constructing an internal mental model – effectively performing transformations in a 2D/3D space or performing search algorithms. A human brain’s visual cortex and spatial reasoning can do this effortlessly; an AI must approximate it with computations over text or coordinates. Without special handling, a prompt about rotated lines or a tiny blue maze might as well be a foreign language to a vanilla GPT-4. It lacks an explicit physics engine or a built-in sense of space, so it has to learn any spatial reasoning from data alone.

However, large models have shown emergent abilities: at a certain scale, they begin to show flashes of reasoning or pattern recognition that weren’t explicitly programmed. The new o3 model seems to have tapped into such an emergent capability for spatial puzzles. Scoring 96.5% (above the human baseline!) suggests it either learned a remarkably general strategy or was fine-tuned extensively on similar tasks. Perhaps o3 employs advanced techniques – think of methods like chain-of-thought prompting, where the model is coaxed to reason step-by-step (“First, imagine rotating object A... then align line B...”). It might also leverage hybrid approaches, combining neural networks with symbolic logic or search. This touches on a theoretical question in AI research: did o3 truly generalize spatial reasoning, or did it just memorize how to solve those particular puzzles? In other words, is it displaying genuine reasoning or just extraordinarily good pattern matching on this benchmark?

From an evaluation theory standpoint, Apple’s omission of the latest model underscores a classic pitfall: benchmark saturation and the ever-moving target of SOTA (state-of-the-art). An academic paper is a snapshot in time. Here that snapshot aged in mere months – a new model’s capability overshot the human level, turning Apple’s finding on its head. It’s a reminder that in the fast-paced AI industry, evaluating “reasoning” isn’t one-and-done; it’s an ongoing race. The deeper irony is almost philosophical: we design these puzzles to test human cognitive quirks (e.g. mental rotation tasks were once used to test IQ and spatial awareness), and now an AI not only learned to pass them, but to outperform the average person. Does that mean the AI truly “understands” spatial relations like we do? Or has it discovered a cheat code – an algorithmic way to solve the puzzle without “seeing” it the way humans do? These questions lie at the heart of assessing AI capabilities and AI limitations. The humor in the meme belies a profound achievement: a once-insurmountable cognitive puzzle gap might have lasted only a fleeting moment before computational brute force, clever algorithms, and massive data closed it. We’re watching AI hype vs. reality collapse into a single point – today’s AI reality was yesterday’s hype.