The Law of Large Numbers Is Not What You Think It Is

Most people have heard of the Law of Large Numbers. Most people are wrong about what it says.

Photo by ClickerHappy on Pexels.

Not subtly wrong. Structurally wrong — in a way that leads to bad bets, misread data, and a false sense of statistical security. The misunderstanding even has its own name: the Gambler's Fallacy. But knowing the fallacy's name doesn't mean you've escaped it.

What the Law Actually Says

Here's the real statement: as the number of independent, identically distributed trials increases, the sample mean converges to the true population mean. That's it. Flip a fair coin enough times, and your observed proportion of heads will get arbitrarily close to 0.5.

Simple enough. But notice what it doesn't say.

It does not say the universe will compensate for a streak. It does not say that after 10 tails in a row, heads is "due." It says nothing about individual outcomes at all. Each flip is still 50/50. The coin has no memory. The law operates on proportions over time — not on the next event in the sequence.

Yet ask someone who just flipped eight tails in a row what they expect next, and watch how quickly their brain starts bargaining with probability.

The Gambler's Fallacy Is Just the Beginning

Gambler's Fallacy gets all the press, but there's a subtler version of this error that shows up constantly in professional settings: assuming that a small sample is already "large enough" to invoke the law.

Say your A/B test has run for three days. You have 400 observations per variant. The winning variant is up 6%. You feel good about this — after all, you have hundreds of data points. But 400 is not large in the sense the Law of Large Numbers requires; it's large relative to zero. Whether it's large enough depends entirely on the variance of the thing you're measuring and the effect size you're trying to detect.

People hear "law of large numbers" and mentally bookmark "large samples = trustworthy results." Then they decide what counts as large by gut feeling rather than by power calculations. The law gets invoked as rhetorical comfort rather than mathematical justification.

Convergence Is Slow. Painfully Slow.

Here's something that surprises almost everyone when they first see it visualized: convergence to the true mean can take an extremely long time, especially when the underlying distribution has high variance or heavy tails.

Consider a random variable drawn from a Cauchy distribution — no finite mean, no finite variance. Technically the Law of Large Numbers doesn't even apply. Run "experiments" with it and your sample mean will wander indefinitely, never settling. No amount of data fixes this.

Even for well-behaved distributions, the rate of convergence matters. The standard error of the mean shrinks at a rate of 1/√n. To cut your uncertainty in half, you need four times as much data. To cut it by a factor of ten, you need a hundred times as much data. That's the reality hiding behind the reassuring phrase "the law of large numbers will take care of it."

graph TD
    A[Collect Sample Data] --> B{Is n actually large enough?}
    B -->|No: gut feeling said yes| C[Premature Conclusion]
    B -->|Yes: power calc confirmed| D[Sample Mean Converges]
    C --> E[Gambler's Fallacy or Underpowered Result]
    D --> F[Valid Inference]

The Version That Hurts Businesses Most

Product teams do this constantly. They see a metric stabilize over a week and declare the experiment done. What they're observing isn't convergence — it's noise that happens to look stable. Real convergence means the sample mean is reliably close to the population mean, not just that the daily numbers stopped moving dramatically.

There's also a selection effect: people tend to stop collecting data when results look the way they want. At that point, invoking the law of large numbers as post-hoc justification is backwards. You didn't let the law work. You interrupted it.

What to Do Instead

Before running any study or test, calculate the sample size you actually need — not the sample size you're going to get in a week. Use a proper power analysis. Pick your significance threshold and minimum detectable effect in advance, not after you've peeked at the results.

And when you catch yourself thinking "we've collected a lot of data, we're probably fine" — that's exactly when to stop and check the math. The law of large numbers is real and powerful. It just requires large numbers, correctly applied, to distributions that cooperate.

Your intuition about what counts as "large" is almost certainly too small. The mean is converging. It's just not there yet.