Lessons from Nate Silver’s success in 2012 election
Not surprisingly, Nate Silver’s simulation models got it right 100% And deservedly, he is getting a lot of praise for this success (he can do with some after all the flak he took pre-election).
We at Mu Sigma have always believed in some of the core philosophies that made Nate and his fivethirtyeight blog successful. Some of these beliefs are:
Methodology before data: We believe in front loading all the thinking. The analytical methodology should be decided based on the business situation and data available to perform the analysis and not based on output. The moment an analyst starts “tweaking” the methodology because a hypothesis is proven wrong, they are on a very slippery slope. That was one of the standout differences between the traditional pollsters and Nate Silver. His methodology was pre-determined, while traditional forecasters kept tweaking their sampling methods etc.
White-box approach of analytics: Nate’s entire methodology is well documented/publicized. It allowed him to enhance/augment his methodology using inputs from various critics/supporters. It eventually resulted in a more robust methodology and, more importantly, wider acceptance of the predictions.
Focus on consumption is equally important: One of the standout features of Nate’s blog is how he simplifies his model’s outputs for a reader to consume. This sort of easy-to-consume representation enabled a wider audience to appreciate and participate in building of his simulation model. It deserves a mention here that, subconsciously, every reader on his blog while consuming the analysis also contributed to making it better through their criticism, comments and inquisitive questions.
Learning is more important than knowing: Polls believe in knowing who is a “representative” voter and asking them how they are likely to vote. That system doesn’t learn – unless the analyst/user makes changes to the system; it believes what it believes. But the methodology Nate adopted focused on learning from multiple polls, correcting for their historical performances and applying it to project the current/future scenario. He set up a learning system. This is a significant departure from how polls are conventionally conducted.
Again, Learning is more important than knowing: Experience has lost its edge. What was true four years ago does not hold good anymore and what is true now won’t be two years from now. The value a political (or for that matter any) pundit brings to the table has diminished significantly; that is unless the pundit is willing to learn. It is a classic battle between Knowing and Learning. We believe Learning won in Presidential Elections 2012 and will continue to win in the future.