Something very important happened during the 2012 US presidential election-campaigns -- and I'm not referring to the campaigns themselves, or to the behaviour of the candidates.
A New York Times blogger called Nate Silver predicted all of the key results with unerring accuracy. He got each of the 50 states right. This meant that he got the key 'battleground' states right, including those with very close margins, such as Florida. He got the popular vote right (he predicted that Obama would get 50.8 per cent against Romney's 48.3 per cent -- the actual margin was 51.1 per cent to 47.2 per cent); and he got the electoral-college vote right (he predicted that Obama would get 313 electoral votes against Romney's 225 -- the actual margin was 332 to 206).
I could go on, but you get the picture. What makes this all the more impressive is that the US's voluntary voting system makes it very difficult to make accurate forecasts in tight races, because it's hard to know what the turnout will be like, or whether it will differ significantly either demographically or ethnically from previous turnouts -- and opinion polls have to not only capture voting intentions accurately, but to build in assumptions about both these key factors. He was also making forecasts that differed significantly from those provided by large, well-endowed national polling companies.
At a personal level, Silver kept updating and holding to his predictions (which he calls 'projections') while he was under ferocious criticism from conservative pundits as being a liberal who was running a self-serving Democratic Party line. The same commentators criticised all opinion polls they didn't agree with on similar grounds. (As it turns out, they'd swallowed their own Kool Aid.)
Silver's blog, FiveThirtyEight (which is the total number of electoral-college votes up for grabs) has been hosted by the Times for a couple of years. So far as I know, Silver hasn't revealed the full details of the model he uses, but it's apparent that it's based on a judicious interpretation of opinion polls. Because of the US's electoral-college system (which we in Australia would think of as an indirect first-past-the-post system), Silver is less influenced by national opinion polls than the media and the usual suspects are. Instead, he has compiled a history of opinion-polling results in each state, a set of local characteristics that he calls 'State fundamentals', and a discount factor that he applies to the polling results to produce what he calls an 'adjusted polling average'. This ends up being expressed as a projected vote-share, and a probabalistic figure for the outcome, which he summarises as the candidates' 'chance of winning'.
For example, in one of the key battleground states, Virginia, where the opinion polls had Obama leading by only just over 1 per cent, Silver came up with a projected vote for Obama of 50.7 per cent to 48.7 per cent, and gave him a 79 per cent chance of winning the state. Obama won by 50.8 per cent to 47.8 per cent.
Silver seems to apply discounts to national polls as well, and is clearly much more interested in updated polling averages than in the latest poll/s at any given time. Because of America's size and relatively dense population, he's helped by the vast number of local polls that are carried out during election campaigns. In other words, he has a lot of information -- both current and historic -- at his disposal. But, clearly, he applies a great deal of statistical expertise to these numbers.
Readers of Silver's bestselling book, The Signal and the Noise, will know that he comes from a background of statistical analysis of baseball, and of professional poker-playing. He understands the basis for establishing probabilities in a wide range of fields (including weather forecasting), and isn't frightened of following the evidence wherever it takes him.
Apart from anything else, he could see that Obama was ahead where it mattered -- in the battleground states -- and that he wasn't relinquishing that lead as the election neared.
This is where Nate Silver's achievements have local significance. Before and during the election campaign, he kept showing Obama as having a very high chance of winning; in fact, by election eve, he nominated that chance as being 90.9 per cent; conversely, he said that Romney had a 9.1 per cent chance.
This, at a time when national opinion polls showed only a small gap in Obama's favour. In Australia, US-based print and TV correspondents all opined that the election was -- you guessed it -- 'too close to call'. This was particularly noticeable on the ABC's radio and TV news programs, as well as on its flagship TV current-affairs program, 7.30.
Not to put too fine a point on it, these journalists didn't know what they were talking about. The election was only too close to call if you weren't looking in the right places. They were doing what was easy, and what they thought was safe.
Their failure is the take-home message from Nate Silver's achievements. General correspondents have no right to editorialise about the implications of opinion polls if they don't have a grasp of the nuts and bolts behind the polls -- or at least don't talk to people who do. In Australia, The Australian's Dennis Shanahan has been shown up by electoral-specialist bloggers several times for his inadequate rendering of opinion polls conducted by Newspoll, the company owned by his employers. And our major newspapers, which commission the main polls, have a self-serving habit of splashing the latest poll results as though they were definitive.
Journalism is under enough external challenges without scoring own goals. Australia's electoral system is very different from America's (voting is compulsory for a start, so there's much less need to enrol voters or to get them to vote), but in 2013, an election year, there will be immense pressure on political journalists to behave professionally. Some of them could do worse than start the year by researching the track record of Australia's opinion-polling organisations, and try to come up with evidence-based probabilities of their own.
Henry Rosenbloom