Paper Compliance Vs. Operational Compliance

Written by Evan Schuman
March 14th, 2008

Guest Columnist David Taylor is president of the PCI Alliance and a former E-Commerce and Security analyst with Gartner.

One of the key conclusions to be found in the PCI Knowledge Base is that the rush to get compliant with PCI security standards over the past year has caused many retailers to focus on "paper compliance" – making sure their Report on Compliance (ROC) was filed by the deadline to avoid fines.

But based on more than 70 interviews conducted with retailers, PCI assessors, acquiring banks, processors and technologists, there is a big difference between paper compliance and "operational compliance" – which involves actually incorporating the protection of credit card and other confidential data into the way the retailer operates.

A major complaint from some of the assessors we have interviewed is that they have found themselves under severe pressure from retail executives, who have argued, coerced, manipulated and otherwise tried to use their power to get the PCI assessors to resolve various interpretations of PCI standards in their favor. Retailers, for their part, complain that many assessors only know the "letter of the law" of PCI, and make no effort to judge compliance based on the realistic risk to the confidential data. What both the retailers and assessors agree upon is that PCI is generally being treated as an IT project, managed out of IT, and not the enterprise-wide program that it needs to be.

Some of the key differences between paper compliance and operational compliance have do to with the management of all the data generated by security tools such as firewalls, intrusion detection, access controls, and file integrity monitors. More than 90 percent of the merchants and assessors we’ve spoken to report that most of this detailed, arcane data must be reviewed manually, with security professions having to comb through dozens, even hundreds of log files looking for (often hard to describe) evidence that a security breach is in progress or is imminent. Unfortunately, most of the PCI funds was spent on the tools, with not much left for building or training security staff to properly review this information.

This tremendous and very annoying manual effort is making some IT security people quite upset about the process. On the bright side, there are some real Best Practices to be found on this topic in the PCI Knowledge Base. For example, one Level 1 retailer has developed a partnership with Internal Audit and has supplemented their expertise with a staff person from network operations and a database administrator.

Collectively, this group owns the task of "operationalizing" PCI compliance. Another example is a Level 2 travel industry company that has identified five specific types of "compliance automation" which they will roll out during 2008 and 2008. Their top priorities are SIEM (security information and event management), identity management, and password management. They arrived at their priorities by estimating the amount of manual effort that was being expended, and developed an ROI analysis for each of their compliance automation programs.

If you want to discuss "compliance automation" or the differences between paper compliance and operational compliance, send me an E-mail at


3 Comments | Read Paper Compliance Vs. Operational Compliance

  1. Patrick Hazel Says:

    David Taylor is out in front on this issue of “paper compliance” v. “operational compliance”.

    I’ve heard this issue put a little differently (“checking the box on PCI v. definitive compliance”), but the point is the same.

    PCI DSS compliance and the retroactive auditing cycle it requires (for example) do not deliver definitive security except (perhaps) at the moment in time when the audit is complete. After that, or before that, the audit side of PCI DSS is not a measure or definitive security and neither is a healthy PCI DSS report.

    Keep in mind that the two highest profile data breaches in the last two years occurred within operations that were PCI DSS (or CISP) compliant or had received “waivers” related to an identified gap.

    CFO’s of public-traded retailers or processors whom I talk to are all over this irony. Their issue is definitive compliance (or, to use David’s term “operational compliance”), not PCI DSS compliance,and while they certainly want to keep their acquirer and VISA happy by checking the PCI DSS “box”, they are focused on meeting a higher standard, in their case rule 504 of SarBox.

    The best thing the PCI council could do at this point is to understand David Taylor’s point and raise the bar to reflect the dynamic nature of security threats in the card data sector. Static measures of security status are not going to get the job done.

  2. Mike Says:

    Evan, I like your work here on StoreFrontBackTalk, but I don’t know how you can reference the KnowPCI group as a panel of experts. Many of their members are smart folks but the group is driven by David Taylor, founder of PCI Vendor Alliance. I think groups like this should have full disclosure that they are run by product companies and/or QSAs – who may have a conflict of interest.

  3. Evan Schuman Says:

    Editor’s Note: Mike, you make a fair point. A couple of quick replies to the group, given that you posted this to the group.
    Not sure of your initial reference that I’ve referenced the “KnowPCI group” as a panel of experts. I don’t recall ever having made such a statement. Where did I say that?
    I checked the column you posted this to you and didn’t see that reference. But even if it had been there, that’s not my column. It’s a GuestView column by Dave Taylor. In his defense, Dave is one of the more knowledgeable (and dare I say independent) consultants I’ve ever run into, which is the key reason I asked him to pen these series of columns.
    That all said, we had somehow stopped posting Dave’s full idnentification (disclaimer?) on his columns. That’s now been corrected. They’ll routinely appear in the future and they’ve been added to the last couple of columns. That was our error. All guestviews are supposed to open with a prominent indication of the writer’s background.
    As for Dave’s group, yes, it is run by QSAs and product companies. But I’ve never seen any reference that was influenced by that fact. You’re right, though, that we should have trumpeted that fact more clearly.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.