3 May 2018 - predict probabilities 2

Forecasting ability can be measured by two numbers, called “calibration” and “discrimination”. Calibration tells whether the percentages are right, and it is easy: Adjust your predictions to match your historical accuracy. Discrimination tells whether the percentages mean anything. If you predict each year that about 5 wars will start, because in an average year about 5 wars start, then you’re right but the future is uniformly gray to you. Meteorologists understand that discrimination is frowned on, and call it “skill,” a better name. Unfortunately, out in the real world, skill is also frowned on.