Welcome To SurveyWednesday
Written by Evan Schuman‘Tis the time of year for journalists to be buried in meaningless surveys, all of which come down to "Holiday shopping will be up (or down) this shopping season, depending on which is better for the surveying company."
My favorite survey this year thus far came Tuesday from a company called BuySafe, which sells purchase insurance. One of their highly-credible conclusions? "Two out of three shoppers would spend more online if all online purchases were guaranteed" and " an overwhelming 65% of shoppers feel more comfortable buying from a Bonded Merchant."
Gosh, think those conclusions were perhaps a tad bit preordained? Yes, I’d certainly believe such conclusions from a vendor that sells this stuff. I’m tempted to quote from a classic old Robert Klein routine about a hair replacement commercial, where the spokesperson being filmed said, ‘You can trust me. I’m the president of the company.’ How’s that for logic? ‘What possible reason would I have to mislead you about the quality of this product?’"
Most of the surveys are either about Black Friday or Cyber Monday. The best part is that those are terms that no longer have—and frankly never did have—any real meaning because shopping always peaked long after Black Friday and online purchases fluctuate and Cyber Monday is not likely to be anything special. No, the terms were coined by PR people to make this stuff easier to cover for consumer media.
In that spirit, let me welcome you to SurveyWednesday, which I will now proclaim to be the day when journalists are flooded with more surveys than on any other day in the year.
We’re going to be watching the major retail sites Friday to see if any major site crashes happen. But of course we’ll find one because somebody somewhere is going to either crash or have a serious slowdown. We’ll try not to elevate the inevitable—and not terribly meaningful—crash of some Fortune 500 retailer into an omen for the industry.
-Marc
November 21st, 2007 at 7:58 am
Glad you changed your mind and published this week. What a great read these stories are!
Survey Wednesday – I love it.
November 22nd, 2007 at 11:08 am
Evan – The questions you pose are interesting, but for the sake of your own credibility, it would be nice if you provided us with a few facts of your own to support your cynical/negative assessments. If you doubt the credibility of these ecommerce surveys, you should probably independently run your own survey to validate your intuition/cynicism before disparaging both the industry and specific companies/individuals. It would take you less than 20 minutes of effort to set up your own survey with one of the many online survey companies. Use the same questions these firms use, and see what you come up with “independently”. Your strong assertions would then be viewed as credible.
November 24th, 2007 at 10:44 am
Editor’s Note: If I may, I think you’re confusing credibility with accuracy. We never questioned the accuracy of the surveys, merely their credibility. A source’s credibility doesn’t change if some media outlet runs its own testing. That’s quite irrelevant.
If Exxon performed the most meticulous survey ever and accurately concluded that Americans want to spend more for gasoline and that Americans are proud of oil companies such as Exxon, it might be true but it wouldn’t be credible, given the source. But if The Wall Street Journal or the Associated Press did the identical survey and came up with identical results, it would be credible.
How so? Credibility speaks to whether the source–given their position and biases and what they have to gain–is in an unbiased position to make such a statement. In the Exxon example, they have a strong incentive to say that Americans love them. But AP and the Journal are not, so people tend to find those media reports more credible, more believable.
That’s all we were saying. We do this often in our reporting of the industry. We’ll quote industry experts, but not typically analyzing their own space. For example, a firewall vendor’s EVP can have credible and useful insights on a major data breach. But that same exec would NOT be credible discussing how the top firewalls compare.
It’s not about knowledge. It’s about whether the source has an obvious reason to slant the answers. In the story we ran, we questioned the survey from a company involved in insuring purchases because it concluded that: “Two out of three shoppers would spend more online if all online purchases were guaranteed†and †an overwhelming 65% of shoppers feel more comfortable buying from a Bonded Merchant.â€
We didn’t say those conclusions were not necessarily true, but merely that it’s not credible coming from a vendor selling those products in that space.