Small Samples: Why Short Runs Tell You Little

A fair coin that looks unfair

Flip a fair coin ten times. Most people expect five heads and five tails. How often does that exactly happen? Only about one time in four. The number of ways to get exactly five heads in ten flips is 252, out of 1,024 equally likely sequences, so the chance is 252/1,024, about 24.6%.

And seven or more heads? That has 120 + 45 + 10 + 1 = 176 sequences, so the chance is 176/1,024, about 17.2%. In other words, roughly one in six ten-flip runs of a perfectly fair coin will show 70% heads or more. Nothing is wrong with the coin. Ten flips is simply too few to show its true nature.

That is the central idea of this article: short runs are noisy, and noise can look like a pattern.

A note on gambling

Slots are games of chance with a built-in house edge. No session length, strategy or timing alters the odds, most players lose money over time, and no one should gamble money they cannot afford to lose. Understanding small samples helps explain why a few good spins can feel meaningful while telling you almost nothing. Gambling is age-restricted or illegal in many places, and regulations differ by country.

How big is the noise?

For a fair coin, the typical amount by which the share of heads misses 50% shrinks as the number of flips grows. As a rough guide, the typical miss is about half divided by the square root of the number of flips:

So in ten flips, a result like 30% or 70% heads is ordinary. In 1,000 flips, the share usually lands between about 47% and 53%. These are typical sizes, not limits, and unusual results can still occur. The pattern to notice is that to cut the noise to a third, you need about ten times the trials.

Why this matters for games of chance

A game’s true odds describe what happens across an enormous number of plays. Your own experience covers a tiny slice of that. A few hundred spins is a small sample for most slot-style games, especially those where the big payouts are rare.

Consider a hypothetical game where a big payout happens, on average, once in 5,000 plays. Some players will see it in their first 50 plays; the vast majority will not. Either experience says little about the game. The lucky player has not found a “loose” machine, and the unlucky player has not found a “tight” one. Both experienced the normal spread of a rare event.

This is also why a game’s hit frequency or its return to player cannot be judged from a short session. Those figures are long-run averages, and a short session can land far from them in either direction.

Why short-term experience feels convincing

Several habits of thinking make small samples seem more informative than they are.

• Vivid memories. A big win is remembered; many small losses are not. This is the pull of selective memory.
• Pattern seeking. People look for explanations of streaks, which relates to the clustering illusion.
• Small numbers feel like evidence. “I won three times in a row on this one” feels like information, even though three trials can happen by chance alone.
• Recent results feel representative. People tend to expect short sequences to look like the long-run average, but they rarely do.

These tendencies are examples of the cognitive biases that shape risky choices. Knowing about them does not make them disappear, but it helps to pause and ask: how many trials is this based on?

What small samples do not mean

Small samples do not mean you should expect the opposite result next. A fair coin that gave seven heads in ten is not “behind” on tails. The next flip is still 50-50. What happens over a bigger run is that the early excess becomes a smaller share of the total, a dilution rather than a correction, which is explained further under the law of large numbers.

Nor do small samples mean nothing can ever be learned. They mean that conclusions should be proportionate: a few results can rule out only extreme claims, and distinguishing small differences takes many trials.

Practical rules of thumb

1. Ask how many trials a claim is based on. Be skeptical of conclusions drawn from a handful.
2. Expect swings. A winning streak and a losing streak can both occur for a game with a negative expected value.
3. Do not judge a game by one session. A good or bad night says little about the odds.
4. Remember the direction of the average. A game with a negative expected value loses on average over the long run, however a short session turned out.
5. Be careful with other people’s stories. A friend’s big win is a single data point, and the many people who lost rarely tell their story.

The last point also matters for advertising: highlighted winners are a small, selected sample.

A word on risk

Variability is one reason people lose more than they planned: a few early wins can encourage longer play, while losses can encourage trying to recover. Chasing losses is a well-known way for spending to grow beyond what anyone intended. A simple protection is setting a budget before you start, treating it as the price of entertainment, and stopping when it is gone. If you notice that you cannot stop, or gambling is causing stress or money problems, a national problem-gambling helpline or local health service can offer confidential help.

Seeing it yourself

You can watch small samples at work with a coin or a spreadsheet. Run several sets of ten flips and see how much the share of heads varies, then run sets of 1,000. The contrast is often a more convincing demonstration than any formula.