Evan Schuman's StorefrontBacktalk

We’d love to know which are the 15 or so L1 merchants that are not compliant (4 percent of 360, for those of you who are not former math teachers), and the bad guys would like to know this information, too—that is if they don’t already.

For analysis purposes, the lack of specifics makes meaningful conclusions impossible. It’s accepted fact—especially with Visa—that there’s a huge difference between a retailer being truly compliant and being certified as compliant.

What’s the difference? The certified chain hasn’t been breached yet. Yes, the compliance certification is only good until it’s actually needed.

Quick quiz: How many PCI-certified retailers in the last few years did not quickly lose their compliance shortly after a breach? They had to. How else could Visa keep saying that no PCI-compliant merchant has ever been breached if it didn’t quickly cancel the certification after a breach?

Still, a rate of 96 percent PCI compliance is pretty good, so for L1 retailers we give the industry a Pass.

The story with other large merchants is similarly positive. These are the Level 2 merchants with between 1 million and 6 million Visa transactions annually. Here, PCI compliance increased even faster—rising from a dismal 62 percent at the end of 2007 to a whopping 94 percent by the end of 2009. Once again, 94 percent is an “A” in anybody’s book: hence, our Pass assessment. We have to credit Visa’s CAP in 2008 with stimulating compliance. This program offered a series of carrots and sticks aimed particularly at L1 and L2 merchants to encourage them to validate their PCI compliance. The results speak for themselves: L1 merchant compliance increased to 91 percent and L2 was at 87 percent by the end of 2008, each higher than the 2007 figures of 77 percent and 62 percent, respectively.

In the prohibited data realm, the numbers are somewhat better, but the conclusions are ambiguous. At the end of 2007, Visa reported that 99 percent of Level 1 and Level 2 merchants told Visa they did not store prohibited data. At the end of 2008, Visa reported the identical 99 percent for the same groups. At the end of 2009, the percentages for both Level 1 and Level 2 were bumped up to 100 percent.

First, everyone is going to raise their eyebrows at any report claiming 100 percent of anything. Second, Visa has introduced a subtle wording change. In the 2007 and 2008 reports (with 99 percent), it reported the retailers “confirmed that they do not store prohibited data.” In 2009, Visa said “validated not storing prohibited data.” Validated by whom? And how? In some cases, the retailer reported it directly; other times, confirmation came from an assessor.

If asked the question, “Are you still retaining stuff that you’re not allowed to retain?,” who’s going to reply, “Yep, we sure are.” (It’s like walking into an IRS audit and being asked, “Is there any significant source of revenue you’re not reporting?” and replying, “OK. You got me. Yep. May I go now?”)

But that scenario still assumes the person answering the compliance question even knows the answer.