Cover of Expert Political Judgment

Expert Political Judgment

Philip Tetlock

6 ideas

  1. Experts barely beat simple statistical baselines

    Across roughly 28,000 probabilistic forecasts on political and economic outcomes, specialists performed only marginally better than chance and worse than simple extrapolation algorithms such as 'assume the status quo continues' or 'predict the base rate.' Expertise helped people generate sophisticated explanations but did not translate into proportionally better calibration or discrimination of what would actually happen.

  2. Hedgehogs versus foxes cognitive styles

    Hedgehogs extend one big theory aggressively to every problem, while foxes draw on many small ideas, tolerate contradiction, and aggregate conflicting signals. Foxes consistently outpredicted hedgehogs because their self-critical, eclectic reasoning made them less overconfident and quicker to update when evidence cut against them.

  3. Fame and confidence predict worse forecasting

    The traits that make an expert attractive to media audiences, such as a decisive single narrative, confident sweeping claims, and quotable certainty, are the same traits that correlate with poorer forecast accuracy. As a result, the public marketplace for punditry systematically rewards and amplifies the least accurate predictors.

  4. Belief system defenses after failed predictions

    When forecasts fail, experts rarely concede error and instead deploy defenses: 'I was almost right,' 'the unexpected exogenous shock doesn't count,' 'wrong on timing but it will still happen,' or 'it was the right mistake to make.' These rationalizations protect prior beliefs and block the updating that would improve future judgment, and hedgehogs use them most.

  5. Score judgment on calibration and discrimination

    Judgment can be measured rather than argued about by requiring explicit probability forecasts on well-defined, time-bound questions and scoring them with metrics like Brier scores. Calibration asks whether events you call 70% likely happen about 70% of the time, and discrimination asks whether you assign higher probabilities to things that happen than to things that don't.

  6. Hindsight bias erases experts' past uncertainty

    After outcomes were known, experts systematically misremembered their earlier forecasts as closer to what actually occurred than they had been, and they judged outcomes as having been more predictable in retrospect. This retroactive rewriting inflates perceived foresight and makes learning from error nearly impossible without written records of prior predictions.

Save and mark ideas in the app