The Unbearable Wrongness of Survey Reporting

The Unbearable Wrongness of Survey Reporting

Frankly speaking, there’s something blatantly wrong about the way media outfits have been reporting political surveys. In fact, I would like to suggest that more than wrong, it’s criminal.

Journalism is supposed to be an account of facts that actually existed and occurred in the spacetime continuum of our trusty universe. Before getting distributed to the information-starved mass audience, however, the facts to be reported must have been verified either by eyewitness accounts or official documents. But even these “evidences” should not be taken at face value.

Instead, our dashing journalists must open their Pulitzeresque senses to the reality of fraud and deceit, which are among their mortal enemies. If journalists develop a gullible belief behavior to whatever press release is fed them by just about any Poncio Pilato, our beloved evening news and dailies would be yellow-sick with sensationalized false truths. We don’t want that to happen, do we? (Teka, hindi pa ba ganyan ang sitwasyon?)

Well, I have a not-so-surprising announcement to surprise you with: Our news media today couch survey reports in terms that blindly give credence to survey bodies. They treat these surveys’ claims as though they’re gospel truth, immaculately conceived and prophetically accurate.

Some examples:

“The survey, held from March 28-30, gave Villar a 29-percentage point or just an eight-point difference from Aquino’s 37 percent.” (From the Inquirer, 04/11/2010)

“In the latest survey, 2,100 respondents were asked: ‘Kung ang eleksyon ay gaganapin 0ngayon, sino ang pinakamalamang ninyong iboboto bilang mga Senador ng Pilipinas? Narito po ang listahan ng mga pangalan ng mga kandidato. Maaari po kayong pumili ng hanggang 12 pangalan’… The respondents were asked to choose from 62 names. The survey used face-to-face interviews of 2,100 adults with a margin of error of plus/minus 2.2%.” (From abs-cbnnews.com, 02/02/2010)

A report aired over GMA News’ ‘24 Oras’ said of the 2,400 respondents, 28 percent chose Villar while 26 percent picked Aquino. The survey was conducted from March 3 to 10.” (From gmanews.tv, 03/22/2010)

“This was the latest findings in the pre-poll city-wide survey conducted at random by The Center for Issues and Advocacy from March 26-31, and from April 4-5, using a face-to-face survey multi-stage probability sampling method and a skip-interval method of 4 to 5 houses to obviate any possible bias. The Center reported that… the results clearly reflected the issues pervading among the city’s registered voters.” (From Mindanao Examiner, 04/12/2010)

Press releases from SWS, Pulse Asia, The Center and Ibon are information they claim to be data which they claim were gathered on a date they claim they were, in a manner they claim they were, with margins of error they claim they do have. They are all claims which truthfulness journalists don’t have any way of verifying unless they personally go to the places where survey bodies claim these surveys were conducted at the time they are claimed to have been conducted.

(Pero siyempre, hindi nila gagawin ‘to dahil mahal ang gasolina at kulang sila sa tao para gawin ‘yon. For most media outfits, their ability to pursue the truth could only be energized by money advertisers pay them.)

Journalists shouldn’t present survey press releases without constructing their attribution with the use of the word claimunless they themselves have taken the trouble to become eyewitnesses of these claims. Thus, the excerpt from the Inquirer above should have read: “The survey, which claimed that it was held from March 28-30, gave Villar a 29-percentage point or just an eight-point difference from Aquino’s 37 percent.” Otherwise, they manufacture their overblown news about what a diminutive 2,100 Filipinos think on the shaky basis of the perceived credibility of charts and graphs typewritten on bond papers purporting to be true and microcosmic of the 50 million registered voters. This is not a good fact finding habit.

And sadly, habits are hard to break.

Jade

Jade is naturally horrified by Mathematics in the same degree that activities which require stamina, agility and athleticism ruins his day. But far from being a couch potato, he bought a brand new Ab Rocket (for an amazingly discounted price) which he plans to use while watching TV and DVDs—activities with which, for him, either society’s moral ascent or descent can be pretty much discerned. He co-hosts a web-radio show on The Edge, and recently went back to school studying Film.

Website - More Posts

You might want to read also:


Share this post
  • http://jeromemartinez.wordpress.com Jerome

    “They treat these surveys’ claims as though they’re gospel truth, immaculately conceived and prophetically accurate.”

    hahaha. I like this line. haha.

    (repost sa fb)

  • Jade

    @Jerome- haha, thanks for reposting :)

    @Pugo- I remember Richard Gordon commending this as an “information-driven” elections. He’s missing something, I think. More correctly put, this is a media-driven elections. And I don’t think that the available mass media is equipped for such a responsibility.

  • Kurai

    I was wondering, is there a real survey happening in your neighborhood? Seems they don’t survey a lot… I haven’t received any survey questionnaire from them so I don’t know if it’s true. The only survey I’ve got is from GB Nielsen but the topic is about surveys for non-smoker and smoker and I was surprised because before I thought surveys are not real… Anyway, surveys are just a small hint but not a big factor… I hope that people would see that there’s more important than numbers and it should be which is right and not because a lot people is telling you to do so.

  • Niel

    I disagree with your observation. Survey firms do not only serve as plain sources but as expert sources in many democratic countries. With plain sources, journalists are supposed to take their statements with a grain of salt because plain sources usually have interests to protect (e.g. a rape victim is a source who would claim something that may be opposed to the claims a rape suspect would make); expert sources on the other hand do not have such interest to protect (usually, they have business interests which is best protected by being credible).

    Survey firms are expert sources because society accepts their proficiency in conducting their work; no journalist or network or publication can ever qualify a survey firm’s results even if they are willing to spend for it (for one, survey results are released after the fact and verifying it even a day later may change the results even if you track back the particular respondents; if I’m not mistaken, exposing the respondents identities and details is even considered unethical by polling firms), hence the whole concept of tracking survey results and comparing them from one polling firm to the other.

    I agree in principle that some journalists who have been reporting survey results do not understand the concept; but it’s also worth noting that survey results reporting have improved from 2004 and 2007. Today, even the public’s appreciation of surveys has improved as majority does not use survey results as a basis for voting a candidate.

    While your suggestion (adding “claim” to correct what you refer to as an error) may well be easily accommodated per se, it does not improve on the quality of reporting. In fact, saying “The survey, which claimed that it was held from March 28-30…” creates more confusion – why would you doubt the date when it was conducted? How does that affect the results? If you assume that if it was held three days earlier or after it would have produced different results, then that’s precisely why polling firms track results.

    There have been much ado about surveys and polling firms and I’m not plainly defending them; I’m sure they can do that better. What I’m pointing is our skewed and oftentimes double standard appreciation of surveys and polling firms, that we accept it only when we find it useful and reject it if it runs contrary to our views (I’m speaking of us as a people in general here).

    For further reading, you may want to check out Donsbach’s paper presented durng the World Association of Public Opinion Research Regional Conference in November 1996 at Tokyo, Japan, titled “Survey Research at the End of the 20th Century: Theses and Antitheses” and Spangenberg’s report on polling published by the Foundation for Information, titled “The Freedom to Publish Opinion Poll Results: Report on a Worldwide Update”.

  • http://zenmoran.multiply.com zen

    > nag-aantay din po akong madatnan ng survey…pero wala naman hanggang ngayon…iilang percent lang yan sa voter’s population…not reliable…di nila ako mapapaniwala…

  • jen

    “our skewed and oftentimes double standard appreciation of surveys and polling firms, that we accept it only when we find it useful and reject it if it runs contrary to our views”

    I believe we reject the current surveys NOT because of its contradiction to our personal views per se, but its contradiction to reality or at least to what we see, hear and know first-hand.

  • http://blogs.kkbmovement.org/jade Jade

    @niel- Hi, thanks for sounding off. I’m afraid your view espouses an attitude of naiveté and unthinking absorption of the messages that bombard society and individuals everyday, courtesy of mass media.

    “Survey firms are expert sources because society accepts their proficiency in conducting their work…” What society? This is a sweeping generalization.

    And yes, adding “claim” to reports would “creates more confusion” to the already babylonian survey reports. You hit the bull’s eye. My point is that there’s really no point in overblowing these types of reports on front pages as their value depend merely on claims which reliability depend merely on naïve generalized perceptions that they are “expert” sources.

    There are more newsworthy facts and moments out there. So would our mass media please stop feeding us with overblown reports on political surveys.
    Thanks for the suggested readings. :)

    @Zen AND Jen- Thanks for sounding off
    :)

  • Randy Jay

    The news last night got me raising my eyebrows. Sen Villar was now doubting the credibility of the most recent 2010 surveys? This was after the said survey showed him trailing behind Noynoy and Erap is now on the same spot with him. The funny thing is, a couple months ago when the same survey firms showed results with him on the lead, he acknowledged them as “credible”.

    Now what??

  • http://zenmoran.multiply.com zen

    > may napansin naman akong media na nag-conduct ng survey…syempre may question na kung ngayon daw ang election day, sino ang iboboto mong presedente, choices follows…
    a.Nonoy
    b.Villar
    c.Erap
    d.OTHERS
    …TAMA BA YUN??? others na lang yong iba?

    > may isa pa akong inaabangan eh…yong result ng Absentee voting…bakit po ba di yun ipinapalabas dito sa pinas eh lantad na yun sa news ng ibang bansa…bakit wala sa atin? kung meron man sa “MB” lang, napakaliit pa, halos di pa makita…

  • http://zenmoran.multiply.com zen

    > pa-off topic…
    pwede mag-request? i have known someone who blogs about media and i really love it…natatauhan ako sa mga articles nya…i think kilala nyo rin po siya…at sana may section din po siya dito to share her blogs…she is VJ dela Calzada…