GuestView: Many QSAs Do Not Have The Background, Expertise To Assess PCI

Written by Evan Schuman
April 15th, 2008

Joel Weise is a Principal Engineer and Chief Technologist for the Sun Microsystems GSS Security Program Office. Prior to that, Joel was a security practitioner with Visa International where he established the original IT security department and worked in the R&D and Risk management departments.

Although there are many qualified security assessors (QSAs), a few who simply do not have the background and expertise in systems security manage to distort the original intent of PCI.

I often hear of QSAs that simply use PCI as a checklist, without thinking about an organization’s overall security posture or architecture. For instance:
• Do you or do you not have an antivirus package?
• Do you or do you not have a firewall?
• Do you or do you not have unique user IDs?

Great. But a good QSA would ask not only if an antivirus package existed or if a firewall appliance was installed or if a unique user ID policy was followed, but also how these were designed, architected, implemented, configured and monitored. In addition, a good QSA would ask to what security policy must applicable operational procedures adhere and whether anyone looks at the alerts and logs generated by the antivirus or firewall products.

When these questions are not asked, a poor understanding of the intent of PCI is typically at issue. But is this the result of a lack of qualification on the part of the QSA? Or is this poor understanding of PCI just one component of a larger problem where the inherent ambiguities of PCI are reflected in assessments? Or, worse still, is it because a QSA can both assess an organization and function as a consultant—someone who is available to remedy flaws uncovered in an assessment (which can, perhaps, suggest ulterior motives)? I would say that many of the current problems with PCI are a result of all of these possibilities.

Given the questions that I hear regarding how a QSA should apply PCI, it is clear that some QSAs are simply not qualified to function as security assessors. This is problematic in and of itself, but when we add in the issue that a QSA can not only find fault with an organization’s compliance with PCI but turn around and sell a solution to address out-of-compliance findings, it’s not unreasonable to question that person’s motives.

The obvious ambiguities we find in PCI complicate matters. But at the base of the problem are the associated struggles to address an organization’s insistence that its solution satisfies PCI when a QSA will not accept that solution. Although the PCI Security Standards Council is chartered to evaluate these ‘disputes,’ what we often see is the council’s deferral to the QSA.

For example, the FAQ on the PCI Security Standards Council Website ( lists the question, "What is meant by ‘adequate network segmentation’ in the PCI DSS?" The response, "….the PCI Security Standards Council is not able to offer an opinion about how your organization can achieve adequate network segmentation since it requires an understanding of security features and controls implemented in your environment. We encourage you to contact a Qualified Security Assessor (QSA) to assist in scoping your cardholder data environment and recommend methods specific to your organization to help reduce the scope of your PCI DSS assessment…."

What if your QSA thinks that segmentation must be done via physical separation and you are using a virtualization technique? Who is the ultimate arbiter here?

The Key PCI Questions

Many people ask me, "How does PCI really work? Why does it appear as a simple checklist? How do I ‘pass’ a PCI assessment (or better, why didn’t I pass my assessment)? Why is it such a hassle? What are compensating controls?"

All of these questions are quite understandable, given what PCI has morphed into and how the cottage industry that has grown up around it has interpreted PCI. To help you understand what PCI is and is not—right or wrong—here are some of my thoughts.

First off, some personal background, to give credence to my stance. I spent my formative years at Visa. It was great fun being on the bleeding edge and inventing new, innovative ways of greasing the skids of worldwide commerce. I still look back quite fondly on those days. I participated in the creation of many of Visa’s internal and external security efforts, including erecting the first security office internally. I also participated in the development of technology such as SET, EMV and the Open Platform chipcards. One of my favorite activities was working on technology risk.

Among other reasons, PCI was originally developed as a response to some of the early attacks on e-commerce sites. Merchants often would rush to build their e-commerce sites without giving much thought to the security of those sites. As with many things in life, it sometimes takes a catastrophe to get people thinking about the issues and risks at hand.

When e-commerce sites started to experience external attacks from the Internet, it was recognized that credit card holders, banks (both acquirers and issuers) and Visa (and other brands) were at risk for basic fraud. But more importantly for Visa, such attacks had an adverse impact on the brand. Trust in the brand and in merchant-bank-interchange relationships is paramount. PCI was the card associations’ response to this. It attempted to recommend best practices for securing sites.

The goal of PCI was to instill trust in e-commerce in general and in the brand and its merchant-bank-interchange relationships and capabilities that support e-commerce in particular, thereby enabling the reduction of risk and liability for the various participants (i.e., Visa, the banks and merchants). The issue of trust is critical to the success of e-commerce. It doesn’t take many poorly designed e-commerce systems that could enable identity theft to turn people away from the Internet as a safe
and sane place to do business.

PCI is a living standard. It is intended to grow over time and be flexible enough to allow the use of new technology (e.g., virtualization). This means that a QSA should not disallow the use of new technology such as virtualization simply because there is no provision within PCI explicitly allowing its use. It also means that a good QSA keeps abreast of current technologies and keep an eye out for future possibilities and how they affect the security of both the data and the systems that data resides on. A better QSA expands upon such knowledge and seeks to understand how new technologies can be leveraged to better secure an infrastructure.

Its intent was to ensure organizations implement a comprehensive security architecture. In other words, PCI is not a checklist but rather a baseline against which one can evaluate their security posture or architecture. It is not a hard and fast list of mandatory elements dictated by the powers that be. The intent is to ensure that a holistic security effort exists and includes various security elements and constructs to limit the threats and risks to sensitive information and processing resources.

Being a QSA means understanding security architecture for what it is—an art form. QSAs must be capable of understanding how PCI can be implemented in an unlimited number of ways. This is not to say that any organization that states they have a security architecture, published security policies, a complete security awareness training program, and manages and monitors their systems will necessarily comply with PCI. But those that have a security program that exhibits such characteristics have a better chance of satisfying PCI.

Besides getting hassled about PCI in general, most questions I get involved compensating controls. According to version 1.1 of PCI, "Compensating controls may be considered for most PCI DSS requirements when an entity cannot meet a technical specification of a requirement, but has sufficiently mitigated the associated risk."

This statement really gets to the core of PCI. For example, if you do not have an anti-virus system enabled, what compensating controls do you have that will prevent the spread of a virus and more importantly, prevent the disclosure, destruction, or loss of integrity of some sensitive data element? Possibly you are using a hardened Unix OS that is not "commonly affected by viruses." The question here would be, "If I’m using a hardened Unix OS, do I even need compensating controls?"

So what is a compensating control? For better or worse, this is left up to the QSA to determine, And where does that leave the organization being assessed?

The good news here is that the intent of compensating controls is to allow an organization undergoing a PCI assessment to argue that the particular controls they implemented have "mitigated the associated risk." Thus a risk analysis indicating that the controls put in place do in fact reduce risk should be sufficient justification to allow those controls to be used, assuming that a QSA is convinced the reduction in risk is adequate.

One of my favorite is "Requirement 2: Do not use vendor-supplied defaults for system passwords and other security parameters." Buried inside here is: "2.2.1. Implement only one primary function per server (for example, web servers, database servers, and DNS should be implemented on separate servers)."
Let’s forget for a moment the issue of why a requirement for passwords and security parameters includes a process separation requirement. I have heard this interpreted by QSAs as meaning any and all functions related to credit card processing must be on physically separate servers.

Notice that PCI does not make any mention of physical separation. The original intent here was to ensure compartmentalization of risk by isolating sensitive data. That of course is a reasonable goal. A good QSA should have a solid understanding of security architecture separation techniques. And one of the most popular today is virtualization. Thus virtualization is a reasonable security separation technique and a good QSA should consider a properly architected and configured system using it to be in compliance. Yet, many QSAs do not understand virtualization and then reject its use out of hand.

Key Management

One of the more critical yet esoteric requirements of PCI covers key management. "Requirement 3: Protect stored cardholder data" includes 3.5’s "Protect encryption keys used for encryption of cardholder data against both disclosure and misuse."

Unfortunately, this is yet another case of a laundry list of requirements. And of much greater concern for one being assessed, how many QSAs are actually qualified to evaluate cryptographic systems and operational key management processes?

Many people think that encryption of data is a general panacea for addressing all threats of confidential data disclosure. Encryption done properly will certainly help to address this, but often overlooked is the necessity for solid key management processes. A failure of key management will most likely nullify any benefits derived from using encryption.

Let’s look at just a few of the key management requirements listed in PCI. 3.5.1 notes that access to keys must be "restricted" and 3.5.2 states that one must "store keys securely."

Does this mean access to a public key should be restricted? Since PCI discusses encryption in a generic sense, we will put aside the discussion of public key [asymmetric] crypto-systems vs secret key [symmetric] crypto-systems for another day.

How would one go about restricting access to keys? And then how does one test that the restriction methods are adequate? Should keys be store securely via physical means such as locked in a safe or via logical means or both?

Cleartext secret key components, for example, are often stored on paper, tokens or chipcards. Must all of these be physically secured? It is left to the QSA to make the critical value judgments needed to evaluate the methods used for restricting this access and determine their adequacy.


3 Comments | Read GuestView: Many QSAs Do Not Have The Background, Expertise To Assess PCI

  1. Walt Conway Says:

    This excellent post should be required reading for every QSA in training and more importantly, every person responsible for their company’s PCI compliance.

    PCI is not a checklist; it is a data protection standard. It is evolving as we speak, and we can expect some changes (big or small, no one yet knows) in the fall. A checklist mentality will blind you to the bigger picture.

    Joel is also right in pointing out that security is an art. QSAs are people, so some will be better at their “art” than others. You can interview the QSA organization all you want, but the key is to meet and be comfortable with the individuals who will be working on your compliance project. Some questions you might ask are:
    — Tell me what you know about my industry and how my customers pay;
    — Can you help me move a lot of my systems out of scope (…my personal favorite question)?
    — How will we resolve our differences? or
    — What other (remediation) products and services are you selling?

    The PCI Council is releasing a quality assurance program for QSAs and ASVs, but it needs merchant participation to be successful. Did you complete a QSA satisfaction survey form after your last audit? (BTW, they are required to give this survey to you; if they forgot, it is on the Council’s website.)

    As Joel said, there are a lot of great, experienced, informed, and helpful QSAs out there. The idea is to make sure you get one of those.

  2. Branden Williams Says:

    Excellent piece Joel! You and I have had this very conversation many times, and we see it in the field ALL the time.

  3. David Taylor Says:

    I agree with the premise and would hasten to add that there are many examples of assessment process issues in the PCI Knowledge Base.

    Another issue that plays a major role in the QSA process is that assessors rely heavily on the vendors and products that they “know” or have experience with on other assessments. Some assessors go so far as to resell products, but most do not.

    A result is that if an assessor is not familiar with a particular security product, the merchant (and sometimes the vendor) is placed on the defensive and must go to greater lengths to “prove” that specific functionality matches specific PCI DSS requirements.

    This scenario tends to favor market leading security vendors and those vendors who have established “relationships” (formal or informal) with particular assessors. In some cases, assessors also sell their own security products, suggesting to merchants that there is a “Chinese Wall” that separates the different parts of the company.

    My personal suspicion is that a notable breach, or merchant outcry, or government intervention will, at some point, serve as a “friendly reminder” that there is a lesson from the Enron case that some companies still need to learn. IMHO.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.