Track record

Are our forecasts any good? These are short-range, checkable forecasts, each with a deadline and a resolution rule, scored in public when they resolve. The A–D probabilities can't resolve, so this is how you can check our judgment.

27 forecasts: 17 open, 2 withdrawn, 8 resolved; next deadline Nov 20.

Coming upNext: Nov 18, Xi–Trump meeting in Shenzhen at APEC; US establishing incident channel with China Weekly steelmanThe strongest case against our numbers, every Friday. Changes and correctionsMethod v2.0 (Oct 6) · alarm criteria v1.2 · 62 corrections, newest Nov 16

Open forecasts

The amber bar is our probability; a hollow ring marks prediction-market odds where a comparable market exists.

Calibration

The chart appears once 15 forecasts have resolved.

Resolved

How scoring works

Each resolved forecast scores (probability − outcome)², where the outcome is 1 or 0. The Brier score is the average: 0 is perfect, and always guessing 50% scores 0.25. A raw Brier score depends on how predictable the questions were, so we also show a skill score against always forecasting the base rate of the resolved questions (above 0 beats it), and, where a prediction market priced the same question, the market's Brier score on those same questions. Every score shows how many forecasts it rests on; a handful proves little.

When we say 70%, it should happen about 70% of the time. The calibration chart waits for 15 resolved forecasts, because a group of one or two would read 0% or 100% by chance; it then groups them into three bins (five once 40 have resolved), each with its count and a 90% interval, and points near the diagonal are well calibrated.

New forecasts are added in each weekly wrap-up, and nothing is edited after it's made; if a forecast's stated context turns out to be wrong we add a dated correction and still score the original probability. Ill-posed forecasts can be withdrawn unscored; the reason is always shown. How we round and rate evidence: Method.

ShareXLinkedInFacebookBlueskyThreadsRedditHacker NewsEmail
Get Hidden AGI watch by email. The latest reading, what changed, and the news that matters. Free.
Subscribe