Re-Thinking PCI Assessor Selection: Does Quality Matter?
Written by David TaylorColumnist David Taylor is the Founder of the PCI Knowledge Base and former E-Commerce and Security analyst with Gartner.
Back in high school, I had a Latin teacher who used to make her students recite in front of the room. We all dreaded this because nobody ever got it right. No matter how much we studied, she always found something wrong with our recitations. So, after a while, most of us stopped trying.
That’s sort of how merchants feel after they spend hundreds of thousands (even millions) of dollars to achieve PCI compliance. Then, if they have a breach, suddenly they are told that they weren’t really compliant after all. It’s not surprising some merchants view this as a “no win” situation.
In the case of the recent Network Solutions breach, after the company stated that they had been found to be PCI compliant, Bob Russo, head of the PCI Security Standards Council, said, via E-mail: “Until a forensics investigation is completed, an organization cannot comment accurately on its compliance status.”
As one PCI manager told me recently: “Why should we bother doing PCI ‘right?’ We’ve always stayed away from the low-ball QSAs that would just rubber stamp our compliance. We paid premium prices to get the best assessment. But, if we get breached, it probably won’t matter. The card brands will just bring in an ‘A List’ forensic team and all they have to find is one little thing (out of 200+ controls) and suddenly we’re not compliant, regardless of how much money, time and effort we put in. We should just hire an ‘easy grader’ QSA and get compliant at minimal cost as fast as possible.”
Now, I’m a pretty cynical guy, but this sentiment made me rethink some of the advice I’ve been giving to companies seeking to hire QSAs.
Ever since MasterCard launched a new “gold rush” for QSAs by mandating their use for both Level 1 and Level 2 merchants, we’ve been receiving more and more questions about how to choose a QSA. This comment was enough to make me re-think a sample QSA Request for Proposal (RFP) that we developed and placed in our Document Archive.
We’ve been telling merchants and service providers to reject “easy graders” who guarantee they will get merchants PCI compliant for a specific fee and in a specific timeframe. Our RFP template includes advice to merchants that tells them to select a QSA that has specific experience in their industry, who understand their business processes and have worked with their specific application platforms. The goal is to find the vulnerabilities before they are exploited to cause a breach.
Our QSA RFP template goes into great detail about how to evaluate the candidate QSA’s expertise in the testing of controls and his/her experience in designing or evaluating the effectiveness of compensating controls. Maybe we’re wrong. Maybe all that really matters is price.
Maybe what merchants should look for is “flexibility” when it comes to the evaluation of such areas as store-level security and application security. These controls can be expensive to test properly, involving many personal visits to randomly-selected stores and detailed application code reviews. However, if you choose “wisely,” you can find an assessor who will complete these assessments in a few hours by reviewing a couple of IT architecture diagrams and some screen shots sent via E-mail.
-Marc
August 26th, 2009 at 4:13 pm
“But the more immediate problem is that if you use a bargain basement QSA, your Report on Compliance (ROC) is more likely to be bounced by your acquirer or the PCI SSC’s Quality Assurance team.”
If that its the case, it only server to further proves what a crock PCI DSS and the QSA qualification process are. If an ROC is going to be rejected by the acquirer or the PCI SSC — the vary organization that said the QSA in question is qualified! — their credibility is out the window.
How can the PCI SSC have it both ways — telling us a specific QSA is qualified to do the assessments, and then reject the ROC from that QSA? Even a first year law student could win that court case me!
August 26th, 2009 at 11:22 pm
Hi, Cranston,
I agree that just taking a 2 day training class (and paying a bunch of money) to become a QSA is not exactly a high hurdle. There’s a long list of QSA companies and they’re certainly not all equal. I have talked with acquirers and other assessors who are skeptical of certain QSA companies, who have “reputations” as being easy. But there is actually far more variation by individual QSA than there is by company, according to a great number of our interviews with merchants, banks, service providers, and the QSAs themselves. I suggest you search our research DB for comments from our anonymous interviews about specific companies, or you can post questions to others in our discussion forums – you can do so anonymously if you wish. If you would like to discuss this, just email me. (david.taylor@knowpci.com). Thanks, Dave
August 27th, 2009 at 10:00 am
Pick a QSA that has something to lose – a 5 man shop has a lot less to lose than the partner signing the ROC at a Big Four firm.
There is a reason why you pay a premium at these types of firms.
August 28th, 2009 at 1:28 pm
@bob
“Pick a QSA that has something to lose – a 5 man shop has a lot less to lose than the partner signing the ROC at a Big Four firm. There is a reason why you pay a premium at these types of firms.”
Ah, yes — the same Big Four who’s activities brought us Enron and SOX. I suspect it’s *not* that they really have more to lose (seems to me these firms made a killing from SOX which was brought in to “fix” of the very problems they caused), but just that they have more resources to cause the PCI SSC pain and greif if they ever were challeneged.
August 28th, 2009 at 2:10 pm
I wanted to see if I could get a soft copy of the pci qsa article by David Taylor. It looks like a nice capsule summary of conversations my customers have had.
Thanks,
mark
August 31st, 2009 at 6:29 pm
Wouldn’t it be best to hire the QSA company that is also listed as one of the very select and few QIRAs also? After all, if you attain compliancy and then get breached, assuming you changed nothing, how could the PCI SSC ever come back and say your company was not compliant when the assessment was done not only by a QSA but one they have hand-anointed as a QIRA? That seems a sure-fire slippery slope for the PCI SSC if I ever saw one. Would the assessor investigate themselves? I think PCI SSC needs to re-think the QIRA list.
September 2nd, 2009 at 2:06 pm
The question is not just picking up a good QSA. Do we need to drive or be driven?
Its what the internal security team lays a strategy and plans and implements for data security rather than go get a PCI certification which is not an end in itself.
Its like a chain is as strong as the weakest link. The corporates have to pass budgets in time, plan for mitigation of risks which most of the time is legacy driven and ask ourselves “how do i improve this further?” Nevertheless, meeting the PCI standards is necessary but it does’nt cost running an extra mile to safegaurd the trust reposed by customers of their card data with us.
September 18th, 2009 at 11:10 am
@Chris Miller
“Wouldn’t it be best to hire the QSA Company that is also listed as one of the very select and few QIRAs also?”
If a Merchant has been breach, and QSA from Company X has submitted a passing ROC, then that Merchant needs to call QIRA Y or Z. Company X can not do the investigation.
While I do agree that the 7 on the list is short sited from the amount of other forensic firms. Lets hope that the SSC adjusts the list once they take over the QIRA program later this year.
Cheers,
Chris
September 18th, 2009 at 8:18 pm
If brands would invest in security and stop relying on other people to protect their magnetic stripe data for them, we wouldn’t need QSAs at all.