Re-Thinking PCI Assessor Selection: Does Quality Matter?

Written by David Taylor
August 26th, 2009

Columnist David Taylor is the Founder of the PCI Knowledge Base and former E-Commerce and Security analyst with Gartner.

Back in high school, I had a Latin teacher who used to make her students recite in front of the room. We all dreaded this because nobody ever got it right. No matter how much we studied, she always found something wrong with our recitations. So, after a while, most of us stopped trying.

That’s sort of how merchants feel after they spend hundreds of thousands (even millions) of dollars to achieve PCI compliance. Then, if they have a breach, suddenly they are told that they weren’t really compliant after all. It’s not surprising some merchants view this as a “no win” situation.

  • Are We Ever Really PCI Compliant?
    In the case of the recent Network Solutions breach, after the company stated that they had been found to be PCI compliant, Bob Russo, head of the PCI Security Standards Council, said, via E-mail: “Until a forensics investigation is completed, an organization cannot comment accurately on its compliance status.”

    As one PCI manager told me recently: “Why should we bother doing PCI ‘right?’ We’ve always stayed away from the low-ball QSAs that would just rubber stamp our compliance. We paid premium prices to get the best assessment. But, if we get breached, it probably won’t matter. The card brands will just bring in an ‘A List’ forensic team and all they have to find is one little thing (out of 200+ controls) and suddenly we’re not compliant, regardless of how much money, time and effort we put in. We should just hire an ‘easy grader’ QSA and get compliant at minimal cost as fast as possible.”

    Now, I’m a pretty cynical guy, but this sentiment made me rethink some of the advice I’ve been giving to companies seeking to hire QSAs.

  • Selecting a QSA – Is Price All That Matters?
    Ever since MasterCard launched a new “gold rush” for QSAs by mandating their use for both Level 1 and Level 2 merchants, we’ve been receiving more and more questions about how to choose a QSA. This comment was enough to make me re-think a sample QSA Request for Proposal (RFP) that we developed and placed in our Document Archive.

    We’ve been telling merchants and service providers to reject “easy graders” who guarantee they will get merchants PCI compliant for a specific fee and in a specific timeframe. Our RFP template includes advice to merchants that tells them to select a QSA that has specific experience in their industry, who understand their business processes and have worked with their specific application platforms. The goal is to find the vulnerabilities before they are exploited to cause a breach.

    Our QSA RFP template goes into great detail about how to evaluate the candidate QSA’s expertise in the testing of controls and his/her experience in designing or evaluating the effectiveness of compensating controls. Maybe we’re wrong. Maybe all that really matters is price.

    Maybe what merchants should look for is “flexibility” when it comes to the evaluation of such areas as store-level security and application security. These controls can be expensive to test properly, involving many personal visits to randomly-selected stores and detailed application code reviews. However, if you choose “wisely,” you can find an assessor who will complete these assessments in a few hours by reviewing a couple of IT architecture diagrams and some screen shots sent via E-mail.

  • advertisement

    9 Comments | Read Re-Thinking PCI Assessor Selection: Does Quality Matter?

    1. Cranston Snoard Says:

      “But the more immediate problem is that if you use a bargain basement QSA, your Report on Compliance (ROC) is more likely to be bounced by your acquirer or the PCI SSC’s Quality Assurance team.”

      If that its the case, it only server to further proves what a crock PCI DSS and the QSA qualification process are. If an ROC is going to be rejected by the acquirer or the PCI SSC — the vary organization that said the QSA in question is qualified! — their credibility is out the window.

      How can the PCI SSC have it both ways — telling us a specific QSA is qualified to do the assessments, and then reject the ROC from that QSA? Even a first year law student could win that court case me!

    2. Dave Taylor Says:

      Hi, Cranston,
      I agree that just taking a 2 day training class (and paying a bunch of money) to become a QSA is not exactly a high hurdle. There’s a long list of QSA companies and they’re certainly not all equal. I have talked with acquirers and other assessors who are skeptical of certain QSA companies, who have “reputations” as being easy. But there is actually far more variation by individual QSA than there is by company, according to a great number of our interviews with merchants, banks, service providers, and the QSAs themselves. I suggest you search our research DB for comments from our anonymous interviews about specific companies, or you can post questions to others in our discussion forums – you can do so anonymously if you wish. If you would like to discuss this, just email me. ( Thanks, Dave

    3. bob Says:

      Pick a QSA that has something to lose – a 5 man shop has a lot less to lose than the partner signing the ROC at a Big Four firm.

      There is a reason why you pay a premium at these types of firms.

    4. Cranston Snoard Says:

      “Pick a QSA that has something to lose – a 5 man shop has a lot less to lose than the partner signing the ROC at a Big Four firm. There is a reason why you pay a premium at these types of firms.”

      Ah, yes — the same Big Four who’s activities brought us Enron and SOX. I suspect it’s *not* that they really have more to lose (seems to me these firms made a killing from SOX which was brought in to “fix” of the very problems they caused), but just that they have more resources to cause the PCI SSC pain and greif if they ever were challeneged.

    5. mark sullivan Says:

      I wanted to see if I could get a soft copy of the pci qsa article by David Taylor. It looks like a nice capsule summary of conversations my customers have had.

    6. Chris Miller Says:

      Wouldn’t it be best to hire the QSA company that is also listed as one of the very select and few QIRAs also? After all, if you attain compliancy and then get breached, assuming you changed nothing, how could the PCI SSC ever come back and say your company was not compliant when the assessment was done not only by a QSA but one they have hand-anointed as a QIRA? That seems a sure-fire slippery slope for the PCI SSC if I ever saw one. Would the assessor investigate themselves? I think PCI SSC needs to re-think the QIRA list.

    7. Ramachandra Putti Says:

      The question is not just picking up a good QSA. Do we need to drive or be driven?

      Its what the internal security team lays a strategy and plans and implements for data security rather than go get a PCI certification which is not an end in itself.

      Its like a chain is as strong as the weakest link. The corporates have to pass budgets in time, plan for mitigation of risks which most of the time is legacy driven and ask ourselves “how do i improve this further?” Nevertheless, meeting the PCI standards is necessary but it does’nt cost running an extra mile to safegaurd the trust reposed by customers of their card data with us.

    8. Chris Says:

      @Chris Miller
      “Wouldn’t it be best to hire the QSA Company that is also listed as one of the very select and few QIRAs also?”

      If a Merchant has been breach, and QSA from Company X has submitted a passing ROC, then that Merchant needs to call QIRA Y or Z. Company X can not do the investigation.

      While I do agree that the 7 on the list is short sited from the amount of other forensic firms. Lets hope that the SSC adjusts the list once they take over the QIRA program later this year.


    9. Matt Harrigan Says:

      If brands would invest in security and stop relying on other people to protect their magnetic stripe data for them, we wouldn’t need QSAs at all.


    StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

    Most Recent Comments

    Why Did Gonzales Hackers Like European Cards So Much Better?

    I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
    Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
    A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
    The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
    @David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

    Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.