Superforecasting: The Art and Science of Prediction (2015) by Phillip Tetlock and Dan Gardner is a follow on to Tetlock’s Expert Political Judgement. In Expert Political Judgement Tetlock wrote up his long term study of how good people were at making political judgements. Expert Political Judgement showed that most people are fairly poor predictors, in particular those with strong beliefs about the path of events. However, there were some people who were fairly good predictors. In Superforecasting Tetlock looks at the people who were good predictors and studies their behaviour.
For Superforecasting Tetlock created the ‘Good Judgement Project’ and asked people to sign up and then sought out the people who were really good at prediction. Tetlock selected those who were really good by getting those people with a good Brier score and concentrating on them. A Brier score is a statistical measure of probabilistic forecasting accuracy.
The book discusses how government analysis on Weapons of Mass Destruction in Iraq was wrong, how it was wrong on Cuba and how it could be improved. Funding for the project was obtained from the government to work on better prediction for intelligence agencies.
The characteristics of those who were good forecasters were obtained. They were intelligent, but not outrageously so, very curious, were happy to adjust their judgements and they also were fairly numerate. The way they broke down questions, revised their scores and operated in teams was also examined. Tetlock also outlines how people who are interested in current affairs can change how they operate to improve their own predictions and understanding to be more like the super forecasters.
It’s a really interesting book for anyone interested in current affairs. By identifying those characteristics that enable good forecasting it also identifies those characteristics that show good understanding of current affairs and the limits of our own knowledge of events.