Cover of How Not to Be Wrong

How Not to Be Wrong

Jordan Ellenberg

6 ideas

  1. Wald's missing bullet holes on bombers

    During WWII, analysts found returning planes had more bullet holes in the fuselage than the engines and proposed armoring where the holes were. Abraham Wald argued the opposite: the planes hit in the engines were the ones that never came back, so the armor belonged where the holes weren't. The data only came from survivors, and that selection hid the real pattern.

  2. Nonlinearity defeats more-is-better policy reasoning

    Many policy arguments quietly assume a straight line, so that if some of a thing is good then more is better and if some is bad then less is better. Relationships like tax rate versus revenue are usually curved, with an optimum somewhere in the middle. Knowing which side of the peak you are on matters more than knowing which direction the curve points.

  3. MIT students beating Cash WinFall lottery

    Massachusetts' Cash WinFall lottery had a roll-down rule: when the jackpot went unclaimed past a cap, the money flowed to lower-tier prizes. In roll-down weeks each ticket's expected value exceeded its price. Organized groups, including MIT students, bought hundreds of thousands of tickets so that variance averaged out and they reliably profited from a game designed to take money.

  4. Regression to the mean masquerading as causation

    Any outcome that mixes stable skill with random luck will drift back toward average after an extreme result. Outstanding performers seem to decline and terrible ones seem to improve with no cause at all. Researchers who read this drift as an effect of treatment, punishment, or decline invent false explanations, as Horace Secrist did when he claimed businesses converge toward mediocrity.

  5. Statistical significance is not truth or importance

    A p-value below 0.05 only means the data would be unlikely if there were no effect. When many hypotheses are tested, some will clear that bar by chance, as in the fMRI study that found 'brain activity' in a dead salmon. Without accounting for prior plausibility and the number of tests run, significant findings turn into a steady supply of false positives.

  6. Aggregating preferences can produce incoherent public opinion

    Each voter can hold perfectly consistent preferences while the majority's preferences are contradictory, as in Condorcet cycles where A beats B, B beats C, and C beats A. Polls that report 'the public wants' lower taxes, less cutting, and a balanced budget can reflect majorities that no single coherent person holds. 'The will of the people' is often an artifact of how votes are counted, not a fact about the people.

Save and mark ideas in the app