Sloppy reporting with surveys

It seems everyone and their brother has a survey. I am reminded of the Governor role in the movie “The Best Little Whorehouse in Texas” played so beautifully by Charles Durning, who stole the show. Durning’s Governor role would not do anything without seeing a survey poll. Today, seems to be no different, as our leaders want to rely on a less informed public to dictate policy decisions or at least pretend that we have an opinion by asking. Yet, all surveys are not equal nor are they reported well by a sloppy and, at times, conflicted news industry.

First, some surveys are biased from the outset. I have participated in these surveys where the questions are leading you to an answer by not giving reasonable choices. Or, a few surveys play mini-commercials at first before you answer the questions. I have seen surveys with altruistic purposes fall flat by leading questions, which is unfortunate, as unbiased  results would be impactful. I have seen surveys with more malevolent purposes guide you to the conclusion the survey sponsor wants to tout. Then, there are the informal surveys that are biased by who they are asking. The sad part is when these surveys get repeated by others, the surveys gain more validity than deserved as they were just an informal poll of online readers.

Second, playing off the reporting of bad survey information noted above, many news and online sources are lazy or sloppy with survey results. The survey source, timing, number of respondents, margin of error and types of questions are important. Also, a breakdown of the results by demographic group is critical. Here are two easy examples that affect a read on our President or his namesake piece of legislation.

Various surveys indicate that Obamacare still has a larger negative poll rating than positive. But, that does not tell the story. There are about 15% of respondents who tend to be more liberal who do not like Obamacare because it did not go far enough as they wanted national healthcare insurance. Contrary to a belief of far right conservatives, Obamacare is not national healthcare insurance. If you dig further, when the name the Affordable Care Act is used, the survey results get a little better as the nickname Obamacare has a more negative connotation. Yet, when you ask people whether they like provision by provision, with the exception of the mandate, the pieces of Obamacare poll positively.

The same could be said on Obama’s handling of ISIL. I saw this morning that 51% of Americans disapprove of his handling of ISIL with 40% in favor. This data needs to be unpacked more, as a number do not like his handling as they would rather us not go in at all, while others do not like as it as they want “boots on the ground” or at least not taking the question of “boots on the ground” off the table. So, what is the preferred approach per the survey?

I fully recognize we do not have a perfect President, but he has done more good things than given credit for. I do recognize what some do not, that he is moderate left and has been from the outset. This is one reason he does not poll very well, as he upsets apple carts on both ends. One of my more liberal friends refers to Obama as the best Republican president we have ever had, which is probably not how my friends on the right would describe him. Here in the North Carolina senate race, some people are having a hard time reconciling that Senator Kay Hagan was voted the most moderate senator, yet she voted with the president 96% of the time. Part of the reason is Obama is moderate left just like Hagan. Part of the reason is the GOP has moved further right than it was ten years ago, so even moderate right Republicans are called RINOs.

So, to understand what people say they want, we need to have the news agencies dig deeper with their reporting. Who said what, how, when and to whom? Otherwise, the assumption is the opinions come from one direction. And, if you believe that, then you might misunderstand (like many news and pseudo-news reporters do) what the real story is. For now, I will set aside the issue of surveying an under-informed public.

Advertisements

4 thoughts on “Sloppy reporting with surveys

  1. I don’t know how the previous comment got posted so fast. I started to say I hate surveys, because their reliability is exceedingly suspect. And people take them as gospel.

    • VG, your elaboration has merit. I was reminded how certain surveys could be biased without intention. In the 2012 presidential race, one of the reasons some of the conservative commenters were so surprised by the result is their surveys which used “land phone lines” showed Romney fairing better. Yet, older voters tend to have more land lines percentage wise than younger voters. The person whose opinion matters most is Nate Silver, as he had a huge success rate on predictions as he blends survey results together using his weightings. He had Obama winning with over 90% probability just before the election and correctly predicted 49 of 50 states. Yet, keep your healthy skepticism of surveys. Thanks, BTG

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s