Big ideas in AI, illustrated · No. 1

The Big Blob of Compute

In 2017, Dario Amodei wrote a short internal memo making an unfashionable bet: the path to smarter AI isn't a stream of clever inventions. It's a simple, general recipe, scaled up until it's enormous.

By Eduardo Yi and Claude · October 9, 2026 · An explainer for non-engineers · about 10 minutes · 5 things to play with
Watercolor: a trail of small colorful dabs flowing into a large blue blob
Data goes in, one dab at a time. The blob is the AI model; it grows more capable as it gets bigger and is fed more.
The idea

Imagine two chefs trying to make the world's best soup. The first spends years inventing secret techniques: a special knife cut, a precise stirring rhythm, a sauce only she understands. The second does something that sounds almost lazy. She gets a bigger pot, more and better ingredients, and lets it simmer longer.

For most of AI's history, researchers acted like the first chef. Dario's memo argued that the second chef wins, and that she keeps winning as the pot gets bigger. He called the idea "the Big Blob of Compute": a blob, because the model is a big shapeless mass of adjustable numbers, and compute, because what feeds it is raw calculating power.

Most progress in AI comes from a bigger pot, more ingredients, and a longer simmer, not from cleverer recipes.

He wrote it before ChatGPT, before GPT-3, before almost anyone outside a few labs had heard of a large language model. In a February 2026 interview with Dwarkesh Patel, he said he still holds the same hypothesis and hasn't seen much that contradicts it.

Two ways to build a mind

Writing rules vs. learning from examples

Say you want a computer to recognize dogs. The old way is to write down what a dog is: four legs, fur, barks. The blob way is to show a big, general-purpose learner millions of photos labeled "dog" or "not dog" and let it work out the pattern on its own.

Watercolor: a rigid ink grid with some ochre squares filled in, next to a soft blue blob
Left, the rulebook: tidy, human-designed, brittle. Right, the blob: shapeless, but it fills in whatever the examples teach it.

Try it: is this a dog?

Pick an animal. See how a hand-written rule copes, versus a learner trained on millions of photos.

IF legs = 4 AND has_fur AND barks → "dog"
The rulebook
The learner

Every fix to the rulebook ("dogs can have three legs… unless it's a cat") makes it longer and more fragile. The learner doesn't need fixes. It needs more examples and a bigger brain to absorb them. That's the trade the Big Blob makes: give up on hand-crafting, and spend on scale instead.

The bitter lesson

Why clever loses in the long run

Two years after Dario's memo, the computer scientist Rich Sutton published a short essay called "The Bitter Lesson" that reached the same conclusion from 70 years of AI history. In chess, Go, speech recognition and computer vision, researchers built systems around human knowledge. Those systems won early. Then simpler methods that could soak up more computing power overtook them, and the clever versions were abandoned.

It's "bitter" because it stings: the cleverness researchers were proudest of turned out to be a short-term advantage.

Drag the slider: how much computing power do you have?

The rulebook approach starts strong and stalls. The general learner starts weak and keeps climbing.

Illustrative shape, not measured data.
The recipe

What actually goes into the blob

The memo's core claim is that only a handful of things really matter. Dario has said he listed seven. Here they are, in kitchen terms.

Raw compute

The size of the stove

How many calculations you can afford. Today that means warehouses of specialized chips running for months.

Quantity of data

How many groceries

The amount of text, images and examples the model gets to learn from. A big pot with three carrots makes thin soup.

Quality and breadth of data

Fresh, varied ingredients

A cookbook that only has soups won't teach you desserts. The data has to be good, and it has to cover the range of things you want the model to do.

Training time

How long it simmers

How long the model keeps practicing on that data. Pull it off the heat too early and the flavors never come together.

An objective that scales

A goal that never runs out

A task you can keep getting better at forever. "Guess the next word" is the famous one: there's always another sentence to practice on. Newer models also learn from rewards for good answers, the way you'd train with practice problems.

Stability (normalization and conditioning)

Keeping the heat even

Technical tricks that stop the giant calculation from boiling over or going cold, so, in Dario's word, it flows "laminar": smooth instead of turbulent.

Notice what's missing from the list: there's no ingredient called "a brilliant new idea about how thinking works." Ideas still matter, but mostly as ways to let the blob get bigger without falling apart. The next companion piece, on the 2017 paper "Attention Is All You Need", is a perfect example: its big contribution was a design that let computers learn from far more data, far faster.

Balance

The weakest ingredient sets the size

The ingredients have to grow together. Picture a barrel made of wooden planks: it can only hold water up to its shortest plank. Pour in a mountain of computing power with too little data, and most of it is wasted.

This isn't hypothetical. In 2022, researchers at DeepMind showed that many of the largest models of the time had been fed too little data for their size. A smaller model trained on more text, which they called Chinchilla, beat much bigger rivals for the same budget.

Mix your own blob

Move the sliders. The blob only grows when the ingredients grow together.

A cartoon of the idea, not a real formula.
Scaling laws

Every 10× buys another step

Here's the part that turned the hypothesis into a business. In 2020, a group of researchers at OpenAI, Dario among them, measured how a language model's mistakes shrink as you scale it up. The result was startlingly regular: each time you multiply the compute by ten, the error drops by about the same proportion. They called these patterns scaling laws.

Think of a staircase where every step costs ten times more than the last, but the steps are reliably the same height. Expensive, yes. But predictable, and predictability is what lets a company justify spending a billion dollars before it knows exactly what it will get.

Watercolor: three blue blobs, each much larger than the last

Climb the staircase

Each click multiplies the compute by 10. Watch the error fall by the same fraction each time, and the bill grow.

Training bill: $10K
Illustrative numbers. Real scaling curves have the same shape: straight lines when both axes use powers of ten.
Stability

Keep it flowing smoothly

The last two items on Dario's list sound the most technical, but the idea is familiar. Inside a model, numbers get passed through hundreds of layers, multiplied again and again. If each layer makes them just a little bigger, they explode. A little smaller, and they fade to nothing. It's the screech of a microphone held too close to a speaker: a tiny signal, fed back on itself, becomes a roar.

Normalization is the fix. After each layer, the numbers get gently re-centered, the way a thermostat keeps a room at the same temperature. Boring, essential, and the reason the blob can grow huge without collapsing.

Watercolor: long parallel bands of blue and teal flowing smoothly
Laminar flow: every stream stays in its lane. This is what a healthy training run looks like.

Flip the switch

Each layer nudges the numbers 40% bigger. Watch what happens with and without re-centering.

The debate

Is that really all it takes?

It's a hypothesis, and smart people push back on it. Here is the strongest version of each side.

The case for the blob

  • The bet has paid off for almost a decade. Models kept getting more capable as they got bigger, roughly on schedule.
  • Many celebrated "new ideas" turned out to matter mainly because they let models scale further.
  • Abilities nobody trained for directly, like writing code or explaining jokes, appeared as models grew.

The case for caution

  • High-quality human text is finite. At some point the groceries run low, and synthetic data may not fully replace them.
  • A child learns language from a tiny fraction of the data a model sees, which suggests something about learning is still missing.
  • Each step up the staircase costs ten times more. Money, chips and electricity may run out before the steps do.
How the bet played out
  1. 1997
    IBM's Deep Blue beats chess champion Garry Kasparov, mostly by searching through enormous numbers of moves rather than "understanding" chess.
  2. 2012
    A neural network called AlexNet, trained on gaming chips, crushes hand-engineered systems at recognizing images.
  3. 2017
    Dario writes the Big Blob of Compute memo. The same year, Google researchers publish "Attention Is All You Need", the design that makes scaling language models practical.
  4. 2019
    Rich Sutton publishes "The Bitter Lesson".
  5. 2020
    Scaling laws are published. GPT-3 shows what a much bigger language model can do.
  6. 2022
    DeepMind's Chinchilla shows data has to grow along with model size. ChatGPT launches in November.
  7. 2026
    Dario tells Dwarkesh Patel he holds the same hypothesis he wrote down in 2017.
One-minute recap
  1. Don't hand-craft intelligence. Build a simple, general learner and let it learn from examples.
  2. A few ingredients decide how good it gets: compute, lots of good and varied data, training time, a goal that never runs out, and tricks that keep the math stable.
  3. Grow them together, and the results improve predictably with every 10× step. That predictability is why the industry spends so much on chips.
Sources: Dario Amodei on the Dwarkesh Podcast (Feb 2026) · Rich Sutton, "The Bitter Lesson" (2019) · Kaplan et al., "Scaling Laws for Neural Language Models" (2020) · Hoffmann et al., "Training Compute-Optimal Large Language Models" (2022). The 2017 memo itself is internal; its contents here come from Dario's public descriptions. Watercolor plates generated with OpenCV.