Reading — error bars and the next edition
An essay that reads the metric pages in one pass. Where interpretation enters, the limits of the evidence are stated alongside.
- Uncertainties fell steadily, yet values moved between editions by more than those uncertainties allow, and more often than a normal distribution predicts.
- A published uncertainty is better read together with the heavy tails the field's record shows than as a promise that the true value lies outside the stated range only 5% of the time.
What was counted
About every four years CODATA gathers the measurements and theoretical calculations of the period, performs a least-squares adjustment, and publishes recommended values and standard uncertainties for the fundamental physical constants. This site takes 17 core constants, none an exact function of the others, from the 1969 adjustment (pre-CODATA) and the nine CODATA editions, and compares each move between editions with the stated uncertainties.
What it shows
Uncertainties fell steadily: among the 16 core constants whose uncertainty declined, the median half-life of the relative uncertainty is 5.4 years. Yet moves often exceeded what the uncertainties allowed. Between consecutive editions, 12.9% of moves exceeded twice the combined uncertainty and 9.1% exceeded three times, against 4.6% and 0.27% if editions were independent and normal. Against the 2022 values, 29% of earlier ±2σ error bars miss.
Small moves, on the other hand, were rarer than expected: |z| > 1 in 26.5% of pairs, below the 31.7% expected. Editions share much of their experimental input. Without new data a value stays put (10 pairs did not change in any printed digit); when a new measurement or a corrected calculation arrives, it moves a long way at once. The CODATA 2010 report attributes the 2006→2010 move of the fine-structure constant (6.4 old σ) mainly to the correction of an error in eighth-order terms of the theory of the electron magnetic moment. The 2014→2018 move of the proton charge radius (5.5 old σ) came with the first inclusion of the muonic hydrogen and deuterium Lamb-shift measurements, together with improved hydrogen and deuterium spectroscopy.
How far this goes
- These numbers are the share of moves that the reported uncertainties did not cover. Whether the cause is the researchers' overconfidence, unknown systematic errors, or the adjustment procedure (which data to keep, how far to expand uncertainties) cannot be told from these data.
- The 2022 values are not the true values either; a different reference would give different miss rates.
- Editions are not independent and constants are correlated (joint adjustment, the theoretical link between α and aₑ, spectroscopic input shared by R∞ and nuclear radii). Intervals for rates are cluster bootstraps over constants, which handle dependence within a constant but not correlation between constants; and there are only 17 constants.
- Henrion & Fischhoff (1986) reported the same tendency for the reviews from 1929 to 1969, measured against the 1973 values, and Bailey (2017) for some 41,000 individual measurements. The data here are narrower (recommended values, core constants) and longer (to 2022). The direction of the result is the same.
One practical lesson follows. A published uncertainty is better read together with the heavy tails that the field's record shows than as a promise that the truth lies outside the stated range only 5% of the time. The size of those tails is what this site measures.