Nate Silver Went 100% on Election Day: Super Accurate Statistical Analysis or Luck?

By Harris Effron
Daily Finance

Now that the election results are in and President Obama has been declared the victor, we can look back and see how accurate New York Times polling guru Nate Silver's characterization of the race was. As it turns out, Silver's model, which was run for the last time on the morning of the election, ended up right 100% of the time. In all 50 states and the District of Columbia, the final electoral map shared the same coloring as his predictive map.

For some background, Silver's website, FiveThirtyEight.com uses a complex model that takes into account various state and national polls, economic trends, and other factors to determine the odds that a candidate would win a certain state. Since the model gave Obama a 90% chance of winning -- despite most media outlets and the candidates characterizing the race as being very close -- Silver suffered some public backlash.

It's impossible to say for sure if Obama really had a 90% chance of winning (given that it's impossible to go into a very large lab, repeat the election a few hundred times, and see what percentage of times Obama wins). But what we can do is figure out the probability of Silver's model getting all of its predictions correct, using its own probabilities.*

For example, on the morning of the election, his model gave Barack Obama a 79.7% chance of winning Colorado, and an 84.3% chance of wining Iowa. So the chances of Barack Obama winning both Iowa and Colorado were, according to Silver's own model, (.797 x .843 x 100%) 67.2%.

Using this methodology, we can calculate the odds of him predicting accurately where the odds favored each candidate in all 51 cases.To simplify matters, in the list below, I've left out all the states where the model predicted a 100% certainty of a win for either candidate -- because it wouldn't alter the results of the calculation. (Multiply by 1 all you want, it doesn't change the final number.) However, it's worth noting that if any of those states had gone the other way, it would have immediately proven Silver's model wrong. In this calculation, I assume that Florida will go to Obama, which is the way the vote count is currently leaning.

Here we go:

State: Percentage chance Silver's model gave the named candidate of winning it

Alaska: 99.9% Romney
Arizona: 98.0% Romney
Colorado: 79.7% Obama
Delaware: 99.9% Obama
Florida: 50.3% Obama
Georgia: 99.8% Romney
Indiana: 99.6% Romney
Iowa: 84.3% Obama
Kansas: 99.9% Romney
Maine: 99.9% Obama
Michigan: 99.3% Obama
Minnesota: 99.7% Obama
Missouri: 99.6% Romney
Montana: 97.9% Romney
Nevada: 93.4% Obama
New Hampshire: 84.6% Obama
New Mexico: 99.4% Obama
North Carolina: 74.4% Romney
Ohio: 90.6% Obama
Oregon: 99.5% Obama
Pennsylvania: 98.6% Obama
South Carolina: 99.5% Romney
South Dakota: 99.9% Romney
Virginia: 79.4% Obama
West Virginia: 99.9% Romney
Wisconsin: 96.7% Obama

Multiply all of these percentages together, and the answer is that Lady Luck helped Nate Silver look slightly better than he deserves. The odds that all 50 states were going to go the way that his model characterized them as leaning are: 12.8%. That's about a 1 in 8 shot.

So, while the results of this election rightfully strengthen the reputation of the predictive power of Nate Silver's FiveThirtyEight model, at least part of his 50 State accuracy sweep has to be attributed to luck. I'm sure that in the coming days, Silver will readily admit this (as he did for getting 49 out of 50 right the last time). But for now, it looks like he can predict the future.

Note: Since Florida was officially characterized as a "Tossup" in his model, I did another quick calculation excluding it. The odds of his model having been right on all of the other states come in 25.4%, around 1 in 4. (Which makes sense -- take out that last 50/50 coin toss, and you double your odds of batting 1000.)

*ADDENDUM: This calculation assumed that each state's percentages is independent of the outcomes of other states.