DeepSeek versus Claude 3.5: Debunking the $6M AI training myth

Level 1: Keeping It Real with an Analogy

Imagine you hear a story at school that a kid in another class built a robot that’s just as smart as the latest iPhone’s voice assistant, but they only spent $6 of their allowance on it. 😮 Sounds amazing, right? It’s like saying they did something in their garage for pocket change that big tech companies spend millions on! But then a teacher comes along and explains what really happened: the kid did build a pretty cool talking gadget, but it’s only about as good as last year’s iPhone assistant, not the very newest. And it actually cost a lot more than $6 in total – maybe not as much as a big company would spend, but definitely more than the rumor claimed. In the end, the kid’s project is still pretty impressive and cheaper than the original, but it isn’t a magical shortcut to instantly beat the best out there. The big lesson? Don’t believe every super-hypey story without checking the details – usually the truth is cool, but not that crazy.

Level 2: Breaking Down the Hype

Let’s unpack what’s going on in this meme for those not neck-deep in AI jargon. We have two AI models being talked about: DeepSeek and Claude 3.5 Sonnet. Claude 3.5 is an AI assistant from a company called Anthropic (think of it like a cousin of ChatGPT). It’s described as a “mid-sized model,” which means it’s a pretty large machine learning model but not the absolute biggest out there. Large models are measured by the number of parameters they have – basically billions of little dials that the training process adjusts so the AI can generate human-like text. Claude 3.5 having tens of billions of parameters (typical for a mid-sized LLM) is still a huge deal: training it took a lot of time, data, and computing power. When the Anthropic rep says it cost “a few tens of millions” of dollars to train Claude 3.5, he’s talking about the cloud computing bill – imagine thousands of powerful GPUs running around the clock for weeks, plus all the engineering work. That’s the model_training_costs here. It’s expensive, but not “billions” expensive; more like the budget of a big movie, not a whole NASA mission.

Now enter DeepSeek. This is an AI model from China that caught attention in late 2024. People on social media started claiming that DeepSeek’s team managed to train an AI just as good as top American models, but only spent $6 million. For context, $6M is dramatically less than what big labs like OpenAI or Anthropic typically spend to train their best models (which is why some were saying “billions” – likely exaggerating, but those big labs might spend upwards of $100M on a truly cutting-edge model). So the claim was basically: “Wow, a Chinese lab reproduced ChatGPT-level tech for pennies on the dollar!” That’s huge if true, and it feeds into a narrative of East vs West AI competition (and ties into why the original post mentions export controls, since governments care who’s ahead in AI). This claim set off a lot of buzz in AIHype circles. Developers and engineers started asking, “How on earth did they do it so cheaply? Did they find some new trick?”

The text in the meme is an official-sounding answer to those questions. The speaker (from Anthropic) is carefully mythbusting that rumor. Point by point:

• DeepSeek did not do for $6M what cost others billions. In plain terms, the new model from China isn’t actually at the same level as the absolute latest from the US, and the cost savings aren’t as jaw-dropping as people claimed. Maybe it cost less, but not that dramatically less.
• Claude 3.5 Sonnet, the model we’re comparing against, is 7-10 months older than DeepSeek’s model. In AI, that’s like comparing last year’s flagship phone to this year’s mid-range phone. Sure, the mid-range might stack up decently because time has passed and tech gets cheaper/improves – but it’s not a direct apples-to-apples with the newest flagship. The highlight around “9-12 months ago” in the text emphasizes that Anthropic trained Claude 3.5 quite a while back (probably early 2024), whereas DeepSeek did theirs at the end of 2024. So DeepSeek had the benefit of starting later, using newer techniques or cheaper hardware.
• The Anthropic guy also mentions that Claude 3.5 was not trained using a larger, more expensive model. This addresses a specific rumor: sometimes AI teams use a tactic called knowledge distillation – where a big model serves as a teacher, and a smaller model learns from it, which can sometimes save training cost or time. There were whispers that maybe Claude 3.5 (or even DeepSeek’s model) cheated by having some bigger model guide it. He flat-out says that’s not the case for Claude 3.5. It was trained on its own, from scratch (well, from lots of text data). So any notion that Anthropic had an unfair advantage or secret method there is wrong.
• Internal and external evals: these are evaluations or tests to see how good the model is. Internal evals are tests Anthropic runs internally (like having the model solve problems, answer questions, follow instructions and measuring performance). External evals might be public benchmarks or feedback from real users. When he says Sonnet remains “notably ahead” in many evals, it means that in a lot of tests, Anthropic’s model is still better than DeepSeek’s model. So despite the hype, the new challenger hasn’t actually surpassed the older model in quality. It’s close in performance, but not a leap ahead.

Finally, the quote ends with a “fair statement” summary: DeepSeek made a model that’s roughly as good as models that were 7-10 months old in the US, and they did it for a lower cost than those older models – but nowhere near as low-cost as the rumors suggested. In simple terms, DeepSeek might have spent, say, tens of millions instead of hundreds of millions, and got a model that’s almost as good as last year’s best. That’s still an achievement! It’s just not the sensational David-beats-Goliath story that “$6M vs billions” sounded like.

For a junior engineer or anyone new to the AI space, the takeaway is: be careful with hype. Big claims often have fine print. Training cutting-edge AI models is inherently expensive because it involves massive computation (and data and expertise). When you hear a claim that someone did it for super cheap, ask how. Did they use someone else’s pre-trained model? Did they cut corners on quality? Did they actually build something a bit less advanced than it sounds? In this case, the answer was that the story got exaggerated in the retelling. The reality: progress in AI is happening globally, and costs are improving, but there wasn’t a magic shortcut here – just normal advancements and maybe some efficiency gains. The meme is notable because an expert directly stepped in to clarify, which is like getting an official fact-check on an viral tech story. Always useful in the era of AI hype vs reality!

Level 3: The Six-Million Dollar Myth

In the world of AI hype vs reality, this meme is a satisfying dose of truth. It features a bullet point from an Anthropic exec (likely Dario Amodei) calmly debunking a sensational rumor making the rounds. The rumor? DeepSeek built a ChatGPT-level model on a shoestring budget. Specifically:

DeepSeek does not “do for $6M what cost US AI companies billions.”

Experienced engineers immediately raise an eyebrow at that line. We’ve seen this movie before: bold claims in IndustryTrends_Hype that a scrappy team achieved a miracle for a fraction of the cost. It’s the classic AIHype narrative – David beats Goliath, presumably by being clever where the giant wasted money. But the hype vs reality gap here is exposed by someone with insider knowledge. The Anthropic rep basically says, “Nice try, but not so fast.”

He reveals that Claude 3.5 Sonnet (Anthropic’s own AI assistant model) is a “mid-sized” LLM that cost on the order of tens of millions to train. In other words, even their model – which DeepSeek is being compared to – wasn’t some billion-dollar behemoth, but it sure wasn’t cheap either. The quote emphasizes that 3.5 Sonnet wasn’t trained using any larger, more expensive model as a crutch (debunking a rumor of knowledge distillation or using a teacher model). This detail matters because one conspiracy floating around was that maybe Anthropic secretly used a giant model to guide Sonnet, which would make the cost comparison muddy. The answer: nope, Sonnet was trained straight-up, no giant teacher in the shadows.

The timeline highlight (literally highlighted “9-12 months ago” in green) is another key: Sonnet’s training finished nearly a year ago, while DeepSeek’s model trained in Nov/Dec just recently. That means DeepSeek’s team had an extra year of industry advances and price drops to leverage. In fast-moving AI, a year lag is significant – it’s like comparing a current smartphone to one from last year. So when the insider says DeepSeek’s model is close to the performance of US models 7-10 months older, it’s a polite way of saying, “They’re catching up, but they haven’t leapfrogged the state of the art.” It also implies that DeepSeek’s achievement, while impressive, is more about playing catch-up with less money, not about breaking new ground beyond the likes of Claude or GPT.

For senior developers, the humor here is in the mythbusting tone. You can almost hear the sigh and see the eyeroll. The quote essentially translates to: “No, they didn’t magically do $Billion_Work for $6M. What they did is build something roughly on par with our last-gen model, and sure, they spent less than we did – but not that much less. Let’s keep it real.” This is a familiar refrain in tech: wild claims meet the cold shower of facts. It’s reminiscent of those moments when someone on Twitter brags about rewriting an app in a weekend that took a team years – the seasoned folks know there’s more to the story (missing features, different scope, etc.). Here, the AI model economics are the unglamorous reality.

This post also hints at the internal_eval_comparison: Anthropic has internal benchmarks where Claude 3.5 still “remains notably ahead” of the new DeepSeek model. In plainer terms, they ran both AIs through a battery of tests (both internal tests and external evals like standardized benchmarks) and Claude is scoring better. That’s a subtle flex: despite all the talk, our older model still wins in many categories. It undercuts the hype that DeepSeek had matched the best – in truth, it’s close, but not quite there. It’s like saying, “Their rookie is good, but our veteran’s still got the edge.”

The comedic flavor for insiders comes from how measured and detailed this rebuttal is. It’s written in that classically understated engineer-speak, carefully qualifying each point. The author even says “I won’t give an exact number” about training cost – a nod to NDA/secret sauce, but gives a range of “a few tens of millions” to ground the conversation. And the final quote about “not anywhere near the ratios people have suggested” gently pokes fun at the absurdity of the rumor without outright trash-talking. It’s the professional version of “those Twitter numbers were totally bogus.”

In summary, Level 3 perspective sees this meme as a case of AIHypeVsReality where an industry expert sets the record straight. It highlights how AI model training costs aren’t trivial, the competitive gap isn’t closed overnight, and sensational social-media narratives often get the nuance wrong. For those of us who’ve weathered many hype cycles, it’s a satisfying told-ya-so moment packaged in a single bullet point. 😏

Level 4: No Free Lunch in Machine Learning

At the cutting edge of AI/ML, there’s a hard truth: you can’t cheat the compute. The wild claim that an upstart model matched frontier performance for only $6M triggers every expert’s skeptic circuits. Why? Because underlying large language models (LLMs) are well-charted scaling laws – empirical power-laws describing how model performance grows with more data and parameters (and thus more compute). Training a competitive GPT-like model demands an astronomical number of FLOPs (floating-point operations). You need massive GPU clusters crunching for weeks or months. Unless someone found a miraculous new algorithm, a budget that’s orders of magnitude smaller simply can’t push enough bits. This isn’t just penny-pinching; it brushes up against physics and chip constraints.

Consider the Chinchilla scaling law (a famous result in AI research): it tells us there’s an optimal balance between model size and training data for a given compute budget. If a lab truly achieved near-state-of-the-art with only ~$6M, they’d have to either exploit a radically new architecture or piggyback off an existing giant model’s knowledge. One known trick is knowledge distillation – using a larger “teacher” model to train a smaller “student” model more efficiently. But as the Anthropic insider clarifies, Claude 3.5 Sonnet was not trained via any larger model. No secret teacher, no hidden GPT-4 under the hood. That rumor of a magical shortcut? Pure sci-fi.

In raw terms, a mid-tier LLM like Claude 3.5 likely has tens of billions of parameters and was trained on trillions of tokens. That entails many petaflop/s-days of computation. Even at bargain cloud rates, running thousands of high-end GPUs (like A100s or H100s) burns through millions of dollars fast. The claim that DeepSeek managed to replicate this feat for pocket change would violate these well-established compute requirements. It’d be as if someone claimed to break a hashing algorithm without the requisite time – akin to violating a mini “conservation of energy” in model training. In reality, AI model economics adhere to tough laws of scale: if you slash compute by an order of magnitude, you typically lose some capability or leverage others’ work. There’s TANSTAAFL (“there ain’t no such thing as a free lunch”) in machine learning, and hype doesn’t alter the math. The bottom line: that $6M vs billions meme was a red flag to seasoned engineers, and the detailed rebuttal confirms our suspicions – no one secretly reinvented AI physics overnight.