Sunday, May 06, 2007

Worthless reports

Why the media, especially reputable media like the BBC, publish reports about worthless, unscientific polls continues to puzzle and annoy me.

Is it that they don't recognize a bad poll when they see it, or that they don't care? I suppose there may be some other reason, some agenda behind publishing such a piece, but I can't think of any legitimate excuse.

Giving uncritical space to unscientific polls is not harmless. Sponsoring them, as CNN and others seem to do all the time, is contemptible even if it does get people to click or call.

... with 50% of respondents saying they had watched the Star Wars films more than 20 times.
... was unavailable to say how many people took part in the survey.

Knowing how many people took part in the survey would not legitimize the poll in any way.

It is clear that this is an unscientific poll, which necessarily means the results are illegitimate. That the BBC puts this story on the front of their web page, even if it is in the entertainment section, is not up to the standards of a legitimate news source.

1 comment:

jj mollo said...

It's so easy to do a bad sample and so hard to do a good one that it's no wonder that people cheat. It would be nice, however, if everyone asked what kind a sample was used. Ignore the bad ones and try to forget what the results were.

The Lancet/Hopkins Iraq Mortality study, by the way, had a bad sampling procedure. That was its biggest flaw, IMO. They went to one house after another, skipping houses where no one answered, which were many. There was no further effort to collect any data from those not-homes. Of the people who did answer, over 90 percent agreed to answer the questions. This is a very high response rate. If I were to go down the street interviewing only people who initiated conversation, it would be about the same thing.