Forecast scorecard
A probability is only worth something if you can check it. These are short-range, checkable forecasts, each with a deadline and a resolution rule, scored in public when they resolve. The A–D probabilities themselves can't resolve; these short-range forecasts are how you can check our judgment.
Open forecasts
The amber bar is our probability. A hollow ring marks prediction-market odds where a comparable market exists. Each shows when it was made, its deadline, its resolution rule and any dated corrections.
Calibration
When we say 70%, it should happen about 70% of the time. The chart appears once 15 forecasts have resolved; before that, a group of one or two forecasts would read 0% or 100% by chance. It then groups them into three bins (five once 40 have resolved), each with its count and a 90% interval, and points near the diagonal are well calibrated.
Resolved
Withdrawn forecasts
Withdrawn forecasts aren't scored. Each shows why it was withdrawn.
How scoring works
Each resolved forecast scores (probability − outcome)², where the outcome is 1 or 0. The Brier score is the average: 0 is perfect, and always guessing 50% scores 0.25. A raw Brier score depends on how predictable the questions were, so we also show a skill score against always forecasting the base rate of the resolved questions (above 0 beats it), and, where a prediction market priced the same question, the market's Brier score on those same questions. Every score shows how many forecasts it rests on; a handful proves little.
New forecasts are added in each weekly wrap-up, and nothing is edited after it's made; if a forecast's stated context turns out to be wrong we add a dated correction and still score the original probability. Ill-posed forecasts can be withdrawn unscored; the reason is always shown.