That 'suspiciously frequent' number pair? Meet the multiple-comparisons trap

Somewhere on the internet right now, a lottery site is announcing that 12 and 35 have appeared together twice as often as expected, and that this "affinity" is worth money. The frequency claim will be true. The conclusion is a statistical crime with a name — the multiple-comparisons problem — and once you've seen the arithmetic you'll spot it everywhere, from lottery pairs to health-news headlines.

The setup: 1,176 suspects under surveillance

How many distinct pairs of numbers exist in a 6/49 game?

C(49,2) = (49 × 48) / 2 = 1,176

Each draw of six numbers contains C(6,2) = 15 pairs, so pair-appearances are common. The probability that one specific pair — say {12, 35} — appears in a given draw:

P(both drawn) = C(47,4) / C(49,6) = 178,365 / 13,983,816 ≈ 0.01276 (about 1 in 78)

(Equivalently (6/49) × (5/48) × 2… no wait — cleanest as: 6×5 / (49×48) = 30/2352 = 0.01276. Both routes agree.)

Over 1,000 draws, each pair's expected appearance count is 1,000 × 0.01276 ≈ 12.8, with standard deviation √(1,000 × 0.01276 × 0.98724) ≈ 3.5. So a typical pair shows up about 13 times, and most pairs land between roughly 6 and 20.

Why some pair is always "amazing"

Now the trap springs. You're not watching one pair; you're watching 1,176 at once. Ask: how far above expectation should the most extreme of 1,176 counts sit, under pure chance?

Rough extreme-value logic: the largest of n roughly-normal quantities typically sits near √(2 ln n) standard deviations above the mean. With n = 1,176: 2 × ln(1,176) = 2 × 7.07 = 14.14, so √14.14 ≈ 3.8σ. Expected maximum ≈ 12.8 + 3.8 × 3.5 ≈ 26 appearances.

So in a perfectly fair lottery, after 1,000 draws, the luckiest-looking pair will have appeared around 26 times against an expectation of 12.8 — double the "expected" rate — and this is guaranteed by chance alone. The prediction site didn't discover anything; it ran a beauty contest among 1,176 contestants and reported that the winner was above average. Some pair had to sit at the top. The specific pair holding the crown changes as draws accumulate — which the sites never mention, because their "hot pairs" list quietly reshuffles between visits. (Pair counts share draws so they're mildly correlated, which nudges these figures slightly — the point survives untouched, and Genest, Lockhart and Stephens handle the exact joint distribution in their peer-reviewed pair-level analysis below.)

Same trap, single-number version: run a naive 5% significance test on each pair and you "expect" 1,176 × 0.05 ≈ 59 pairs to flag as significant in a perfectly fair lottery. Fifty-nine false alarms, every time, by construction. A site that mines draw history for "significant" pairs will never come back empty-handed.

The fix: Bonferroni, explained with one division

Statisticians have a beautifully blunt correction for this, named after Carlo Bonferroni. If you run n tests and want at most a 5% chance of any false alarm across the whole family, demand that each individual test clear a threshold of 0.05/n instead of 0.05.

For our 1,176 pairs: 0.05 / 1,176 ≈ 0.0000425. A pair now has to be so extreme that fair chance would produce it less than about 1 time in 23,500 — roughly 3.9 standard deviations, i.e. about 27+ appearances in our 1,000-draw example rather than the 19 that naive testing would flag. Notice something elegant: the Bonferroni threshold (27) lands right where the expected maximum (~26) sits. That's not coincidence — the correction is calibrated so that the ordinary champion of a fair contest just fails to qualify as evidence. Only genuine bias clears the bar.

Why division works: the chance of at least one false alarm among n tests is at most the sum of the individual chances (the union bound). n tests × (0.05/n) each = 0.05 total. Crude, slightly conservative for correlated tests — and exactly the right kind of crude for consumers of statistics: if a claim can't survive Bonferroni, it doesn't deserve your attention, let alone your money.

The professionals did this properly

This isn't just our arithmetic. The peer-reviewed literature has run exactly this analysis on real lotteries: Genest, Lockhart and Stephens (2002) tested equiprobability of lottery numbers "individually, in pairs or in larger subsets" across roughly twenty years of Canada's Lotto 6/49 — deriving the correct joint distributions rather than pretending pairs are independent — and found no serious evidence of non-uniformity at any level. The pairs pass. They always pass. Our is-it-random pages run the single-number version of these tests live on every game we track; and the randomness tester will run them on any sequence you paste in.

The general-purpose lesson

The multiple-comparisons trap is the engine under most modern data superstition:

• Lottery pairs: 1,176 comparisons → a "hot pair" always exists.
• Triples are worse: C(49,3) = 18,424 triples → the champion triple looks astonishing.
• "Number 7 loves Fridays", "sums between 130–160 are due", "ball set B favours low numbers" — every slice you add multiplies the comparisons and manufactures fresh miracles.

The rule that inoculates you: decide what you're testing before you look. One pre-registered hypothesis, one test, 5% means 5%. Anything discovered by scanning is an audition, not a finding — it earns, at most, the right to be tested on future draws it hasn't seen. Lottery "pair affinity" has auditioned for decades. It has never once passed the callback.