This is page 2 of:
PCI’s Grading System Is Failing
The nature of the grading system, rather than the PCI standard itself, is responsible for most of the dissatisfaction we find in the merchant community. This dissatisfaction has resulted in many merchants seeking out assessors who are “easy graders” or take a fairly lax attitude when filling out the self-assessment. When searching our research database, we see numerous comments about how much merchants like certain assessors because they made the process “painless” or were “easy to work with” or other comments that indicate the merchant was more interested in scoring 100 percent on the test than on getting a thorough evaluation. It’s not that these merchants don’t want to be secure. It’s just that they object to being held to a standard where they have to score 100 percent to pass, and get fined if they don’t achieve it. Again: great standard, bad grading system.
The PCI SSC recently took a small step toward a risk-based approach to PCI with the spreadsheet-based prioritization tool. Unfortunately, this didn’t do anything to change the grading system. But, given that risk has been introduced into the process, I believe that it is possible to modify the grading system to address realistic risk levels for different types of businesses. After all, merchants pay more to process card not present (CNP) transactions because there is more risk, and most of the transaction charges and fees that drive the financial services industry are all risk-based.
The merchants and financial institutions that make up the PCI SSC and the card networks certainly have enough data to develop risk ratings for each of the mandated PCI controls and conditions of their implementation, so why not grade PCI compliance on a curve?
The card networks would be happy because they could show increases in their “percent compliant” statistics. The merchants would be happy because they could conduct more realistic self-assessments and the QSAs would be happy because they would have more flexibility to adjust the application of the controls to the particular situation of each of the organizations they are reviewing. This is the easiest way to “bring peace” to the increasingly contentious relationship between the card networks, the banks, the merchants, the assessors and, increasingly, the federal government.
I know that none of these arguments are “new” but I do believe that they are especially timely now and more likely to be considered, thanks to increasing government focus on the card networks and their impact on consumers and businesses. We welcome your comments, on either side of the argument. If you want to read some of the comments about the assessment process and the PCI grading system from our research, please visit the PCI Knowledge Base and if you want to discuss this topic, send us an Email at David.Taylor@KnowPCI.com.
-Marc
April 30th, 2009 at 12:43 am
David,
From time to time we have disagreements but here I am 100% in agreement with you. In the early days of PCI (actually pre-PCS SSC), I was told something that always sticks in my mind. An ex-mucky-muck from one of the associations said: “The card associations view every breach as a compliance failure.” If you put this single statement under a microscope, many of the problems you described in this article become clearer. The problem is, the card associations still defend this belief.
April 30th, 2009 at 7:45 am
Great assessment – we need more outside viewpoints into the PCI requirements and clarity as to how they should be implemented based on the entity.
April 30th, 2009 at 9:47 am
I think you are completely mistaken on your position that PCI requires 100% grade. What about compensating controls? The PCI standard allows merchants and service providers when they are not 100% compliant to the original control to use a compensating control to achieve compliance. The compensating control worksheet is basically a risk assessment of the requirement in question and justification of how the compensating control mitigates the risk to an acceptable level.
Also, comments below are incorrect
Thanks to the grading system, and the fact that many of the PCI controls are “volatile†and can be made ineffective by simple configuration or rule changes, this technically means that an organization may never actually be PCI compliant. That’s because, for a typical Level 1 merchant, an assessment will take more than a month, sometimes several months. Thus, it is very possible that between the time the first controls were tested and the time the last controls were tested, changes were made to the first controls such that they are no longer 100 percent compliant.
A QSA is required to revalidate all time based requirements before completing the ROC. A final compliant ROC does mean the client is compliant at the time of the signature of the Attestation of Compliance form.
April 30th, 2009 at 9:55 am
David – I agree 100% with the comments presented as well as your assesment of “When Bad Assesments are Good” I perform alternate security compliance audits to the PCI and all too often I see and hear from customers that the PCI assesor only took x days, why is my audit taking 2x days. A GOOD auditor is not there to make friends and make the process “painless”. They are there to collect and measure the facts, no matter how long or painful that process may be. IF a customer is following compliance requirements it will be “painless”, but if they are not then it takes longer and they feel the pain. It is all relative
Keep up the good reporting
April 30th, 2009 at 10:18 am
Jeff, I agree that compensating controls introduce risk awareness into the analysis, but my experience and over 370 hours of interviews with merchants, banks, QSAs and service providers indicates they are not used in a consistent manner. They seem like a “bolt on” to the assessment process – which is pretty much what they were.
My argument is that risk could be made much more an inherent part of the assessment process, with each of the controls risk-weighted. That is, IMHO, where the prioritization scheme published by the SSC recently is headed. I believe if the prioritization system were extended, based on the feedback of the 500+ members of the SSC, to create risk-weights that could be customized for different types of businesses, that the grading system could be improved, so that it would be less of a source of contention than it is today.
That’s my main point in all of this.
Dave Taylor
May 5th, 2009 at 11:04 am
PCIDSS 1.2 basically requires useless antivirus scans on Linux (not even mentioning AS/400 or Solaris); replacing it with *useful* malware scanning (not all malware are viruses) only counts as a compensating control. This is simply absurd.
(Tinfoil hat mode: I’m sure Microsoft must have lobbied hard to get the Unix virus scanning exemption removed after 1.1, it must have cost them a lot of biz)
May 6th, 2009 at 12:18 pm
I thought the original wording specifically excluding UNIX was irresponsible for a security doctrine. All operating systems can be susceptible to viruses and other malware. Ironically, the earliest recorded computer viruses were UNIX predecessors and then migrated to various UNIX flavors. Excluding any operating system conveys a false sense of security.
Now I’m not saying the current wording is perfect. Other forms of malware like key loggers, packet sniffers and memory sniffers are far more dangerous to compromising cardholder data than the average virus. Maybe the wording should be targeted more toward malware prevention, detection and removal as opposed to virus. To me, all viruses are a form of malware but not malware are viruses.
May 6th, 2009 at 3:23 pm
Great comments and a very provocative article !
But one thing is missing from the discussion: the value and importance of entity level controls,the “tone at the top.” As QSA’s, we should rely on the tone set by executive management and the Board to provide us reasonable assurance that controls are maintained. While the sign off is a point in time, good QSAs who care about their clients’ well being and who want to serve the public interest will consider the “tone at the top. ” If PCI compliance is not “baked in” or otherwise integrated with robust policy statements and employee education, PCI may be viewed as a check box exercise at best. “Tone at the top” has nothing to do with the size of an organization but rather with the unexpressed beliefs senior management has about PCI. The best way to discern management’s true opinions about the long term value of PCI is to look at the quality of policy statements and consider whether issues recur.
May 7th, 2009 at 5:17 am
@Steve Sommers: what definition of “virus” are you using? “Worms” are not viruses. A virus is a self-replicating program that propagates by inserting itself into executable files. POCs of Linux viruses exist, they have never been a problem in the wild, for a simple reason called “packet management” AKA RPM or DEB.
There is malware on Linux. There is no virus problem on Linux. This is a fact. (Go ahead and google “linux virus”; go read the pages: they mostly actually talk about non-virus malware)
I’m not playing on words. You can implement the best protection against Linux-affecting malware, but it won’t be an “antivirus” per PCI.
Here’s what we’re gonna do where I work: we’re gonna deploy an antivirus, knowing that it will significantly expand the attack surface, because they have to run as root to be able to scan all the files, for no significant gain. This antivirus will mostly scan for Windows virus, even though we don’t have a single Windows machine in our PCI perimeter (we use a completely isolated network, dual workstations and all). Just to check a box.
Security gain? None.
Security loss? The AV vendor could be compromised, allowing an attacker in. More likely, the AV daemon could be used to elevate privileges.
May 7th, 2009 at 11:20 am
Great article!
For some reason, everyone is fixated on the actual PCI process rather than the corruption that is behind the scenes.
The associations recently changed from non-profit organizations to for profit. With that said, and to the point in the article that the merchants are already paying higher fees on card-not-present transactions, ALL the risk is still on the merchant/Acquirer. So why would retailers/eCommerce businesses pay more for transactions if none of this risk is passed on to the association/issuing bank? They are collecting more for these transactions, so shouldn’t some of the risk be passed on to them? Merchants should pay the same for interchange (except maybe for rewards/commercial cards) on all transactions since there really is no risk to the association/issuing bank. Their wasted dollars on the so called “Higher Risk†should be spent on preventing risk instead of playing the Association game. Also, the Acquirer is ultimately liable for any / all compromises, yet none of the padded interchange is passed on to them.
Isn’t paying more based on risk considered insurance? Not here, it is a gimmick for the Associations to get the issuing banks to push their brand instead of their competitor. This is evident with the lawsuit that Amex filed against V/MC a few years back because they are demanding the banks that issue their brand only issue their brand.
May 8th, 2009 at 3:16 pm
In most philosophical discussions about payment card security (except when they emanate from visionaries like Steve Mott), what usually gets overlooked is the obvious. Credit cards were invented before the Internet, and were never conceived to exist in a card-not-present environment teeming with crafty and invisible electronic bandits.
For a two-sided marketplace, our current efforts for minimizing theft and compromise are very one-sided (all responsibility on the merchant, none on the cardholder). It’s like sending your innocent daughter to the store with no clothes on. Then demanding that the storekeeper ensure that she isn’t gawked-at or abused.
As long as we are stuck with a lop-sided program, meaning until some time when the issuing side takes up some responsibility, PCI DSS is our best bet for at least tossing her a blanket.
May 14th, 2009 at 8:31 pm
Chuck is almost right. It’s more like sending your attractive, voluptuous, naked, exhibitionist daughter to the store and insisting that it is the merchant’s responsibility to provide blindfolds for all parties and to enforce a no peeking rule. btw – Failure to do so makes you liable to great financial penalties and a public spanking.