Bogus numbers

By Gordon Prather

Today we will discuss statistical analysis as it is applied – usually improperly – to public opinion polls and to how-did-you-vote exit-polls. The principal practical difference between opinion-poll and how-did-you-vote exit-poll “predictions” is that there is no reality-check on opinion-poll predictions. But as we discovered two years ago in Florida, there eventually is a reality-check on how-did-you-vote exit-poll predictions.

In days of yore, the media elite used to conduct their own polls at the zillions of voting precincts in this country. That cost a ton and frequently resulted in very different predictions being made on who was going to win.

For the 2000 Bush-Gore election many media pooh-bahs got together to establish the Voter News Service. VNS hired thousands of worker-bees, to staff thousands of selected “key” voting precincts on Election Day, to ask stupid questions of exiting voters.

Periodically, the VNS worker-bees phone in their latest “key” precinct data to statistical geniuses who feed it into VNS supercomputers. Shazzam! Out pops a single statewide projection, instantly supplied to all TV talking heads.

So early in the evening, before the polls had closed anywhere in the country – Shazzam! – the Voter News Service “awarded” the election to Al Gore. As the evening wore on, and additional reports came in from worker-bees in Florida, Voter News Service discovered to its horror that its supercomputer kept changing its mind.

The media elite would like you believe that “scientific projections” of the final outcome can be made by applying statistical sampling techniques to selected exit-poll data, long before the polls are closed.

Howzat? Well suppose you manufacture ball-bearings. It’s critical that all of them be almost exactly spherical and have almost exactly the same diameter. Once you have set up a process that consistently produces ball-bearings – all of which meet your specifications – you can use statistical sampling techniques to obviate the need for checking, thereafter, the specifications of every single ball-bearing. Perhaps you start out by checking one in a hundred. If all those meet specifications, maybe you try checking one in a thousand.

But you should not fool yourself about what it is you are actually doing. You are not predicting that all ball-bearings you produce later in the day will meet specifications. What you are actually doing is periodically checking your manufacturing process to see if it is working properly. If ever a sample doesn’t meet specifications, you shut down your machinery, and try to figure out what’s gone wrong.

So what has this got to do with statistical analysis of exit-poll “sampling” data?

Basically, nothing. Voters can not be “sampled” like ball-bearings. No amount of statistical analysis of who-did-you-vote-for exit-poll data – taken at a few selected precincts, long before the polls are closed – can possibly tell you what the final statewide vote tallies will be. As the media elite found out in Florida, two years ago, sometimes you can’t even be certain who won after all the votes have been counted and recounted.

So, for this national election, the media pooh-bahs decided to forgo making any prediction that might later be shown to be dead wrong. Instead, the TV talking heads decided to concentrate on the why-did-you-vote-the-way-you-did opinion-polls taken by Voter News Service at selected precincts. Since there can never be reality-checks for opinion-polls, there is no way for you to know when you are being fed nonsense.

Voter News Service worker-bees could ask voters as they leave the polls such questions as – “On a scale of 1-10, how scared are you of Saddam Hussein?” The answers to that question constitute a set of numbers to which statistical analysis techniques can be applied. The VNS worker-bees can tally the answers according to the voter’s sexual orientation, ethnicity or political party affiliation and report it all back several times a day to the statistical geniuses at VNS Central.

The Voter News Service told its members and subscribers that it planned to report the results of its statistically analyzed statewide why-did-you-vote-the-way-you-did opinion polls by 2 p.m. on Election Day, update those reports later in the afternoon and provide a final report when the polls closed in each state. But – according to the New York Times – VNS failed to even make the initial 2 p.m. EST reports, and at about 4 p.m. EST VNS told its members and subscribers across the country that it would not be able to produce any reliable statistical analyses at all.

Apparently, VNS machinery was making some very strange-looking ball-bearings.