Brier Score
The Brier Score is the mean squared error between a forecaster's predicted probabilities and the binary outcomes that occurred.
The math
BS = (1/N) Σᵢ (fᵢ − oᵢ)²fᵢ = predicted probability for outcome i, oᵢ = actual outcome (1 or 0), N = number of forecasts.
Why it matters
Brier Score is the standard metric for evaluating probabilistic forecasts. A perfect forecaster scores 0.0; predicting 0.5 on every market scores 0.25 (the "no-skill" baseline). Lower is better. On prediction markets, comparing your Brier score to the crowd's (market-implied) Brier score isolates alpha from luck. Brier (1950) introduced it for meteorological forecasts; it is now used across prediction markets, clinical risk scores, and AI calibration.
Brier Score is quadratic — it penalizes confident wrong predictions much more than uncertain ones. This is usually desirable (you want to penalize overconfidence), but it can make scores hard to compare across markets with very different base rates. The decomposition into reliability + resolution + uncertainty (Murphy 1973) gives more diagnostic information but is not displayed on Pancake receipts.
Published source
Brier, G. W. (1950). "Verification of Forecasts Expressed in Terms of Probability." 78(1), 1–3.