A couple of weeks ago WorldNetDaily reader Paul V. Bass asked Joseph
Farah a two-part question about Internet polling, which was passed on to
me. The first half asked why the results of Internet polls were not more
widely reported. The second half questioned the importance of sample
sizes.
Last week I addressed the first part so now we can move on to sample sizes.
Mr. Bass asked, “How come most of the polls reported on the TV news or in the newspapers show maybe a thousand or less people being polled supporting Al Gore, but the polls on WND or even AOL show many thousands of people supporting Bush?”
He asked this prior to George W. Bush’s steady gain in the polls, but the question is still valid. Many people simply assume that a survey of 60,000 must be more valid than a survey of 1,000 participants. As I discussed in last week’s column, the biggest problem with the online results is that they are not drawn from a random sample. Internet polls self-select participants, which skews the results.
Still, one thing pollsters hear all the time is that interviews with 1,000 randomly selected people cannot possibly represent the opinions of 250 million Americans. How can one respondent represent a quarter of a million citizens? This perspective, while understandable, is actually looking at the situation backwards. Instead of focusing on the size of the U.S. population, attention should be focused on the sample.
Rather than saying how could one person represent the views of so many, we should be asking about the odds of asking 1,000 randomly selected people a set of questions and getting anything other than a representative sample.
It’s like tossing a coin. Do it 1,000 times and you’ll get pretty close to 500 heads and 500 tails. In fact, statistics show that if you flip a coin 1,000 times, you’re very likely to get between 469 and 531 heads or tails. In fact, the results will fall in this range 19 times out of 20.
So, imagine that the country is almost equally divided over an issue. When you ask 1,000 randomly selected people whether they favor or oppose the issue, it’s the polling equivalent of flipping a coin 1,000 times. Just like tossing a coin, you’ll end up within 3 percentage points of 50/50 just about every time.
Reality backs up the theoretical proof. People have been conducting scientific polls for over 50 years now and have accurately portrayed the public’s opinion on a wide range of issues. Election outcomes are rarely a surprise anymore because polling has proved it can work to predict the way a population will vote.
While it is possible to accurately predict elections and other events with samples of 500 to 1,000 voters, there are reasons to survey more than 500 or 1,000 people. For one thing, it lowers the margin of sampling error while decreasing the volatility of the survey over time. Most importantly, however, large samples raise the reliability and value of demographic information. If you wanted to highlight the gender gap in a survey of 500 respondents, you would have to depend on only about 250 female respondents. If you wanted the views of, say, white women, black women or married women, the sample’s size would decrease even more.
That is why Rasmussen Research has included at least 2,250 respondents in its Presidential Tracking Poll since March 20. We now include 3,000 likely voters to get an even more accurate reading as Election Day draws near. If we want to break out the gender gap, we have roughly 1,500 women in our sample. This gives us a big enough group to work with while comparing the views of, say, married women to unmarried women.
As a general rule, any demographic subset that includes less than 100 respondents should be disregarded or viewed with extreme caution (in other words, don’t try this at home).
So, while survey size is important, it is possible to achieve accurate results with samples as small as 500 randomly selected people. In fact, it is rarely the sample size that gets pollsters in trouble. Question wording and the analysis of polling data are far more likely sources of trouble. That will give us something to talk about next week.
