Vulnerability Scan Often Says “Pass” Even If You’ve Flunked

Written by Peter Spier
July 25th, 2012

Peter Spier, CISSP, CRISC, CISM, PMP, QSA, PA-QSA is Manager of Professional Services at Fortrex Technologies.

Take a look at your most recent PCI vulnerability scan. Do you see any identified vulnerabilities that are assigned a score of 4.0 or higher? Are you still passing? Is the “Exceptions, False Positives or Compensating Controls” column blank? If so, your approved scanning vendor (ASV) is not doing its job, and this will prevent you from passing your next PCI assessment.

Since PCI 2.0’s implementation earlier this year ASVs have had QSAs closely reviewing their findings and comparing those results against the ASV Program Guide.

The problem here is beyond having the PCI assessment not pass. Retailers will now have to go back to their ASVs, explain the problems and get both sides to work together to fix it. (Yes, the ASVs are going to be thrilled.)

What is worse: That is exactly what the PCI Council’s selected Validation Labs are intended to test for, and solve, on an annual basis to prevent just such an occurrence.

The requirement forcing this change: the testing procedure of DSS v2.0 requirement 11.2.2.b, which instructs QSAs to “review the results of each quarterly scan to ensure that they satisfy the ASV Program Guide requirements (for example, no vulnerabilities rated higher than a 4.0 by the CVSS and no automatic failures).”

The ASV Program Guide has an exceptions column for a reason. When no one fills it out, then the retailer’s ability to be compliant won’t happen. Why would ASVs do this? The answer is simple—and it’s hardly a defense: The column is commonly overlooked. This fact raises a more frightening question: Why would ASVs, who focus on minute details for a living, not adhere to a written requirement? The answer: No one knows, including—perhaps—not even the ASV. In short, retailers will now have to go back to their ASVs.

Another aspect is the change in how security issues are ranked. The adoption of the CVSS and its open framework-based calculation methodology permits both ASVs and merchants/service providers to consider multiple vulnerability criteria and effectively determine the probable impact to affected systems. It is this very capability that establishes the foundation upon which DSS v2.0 risk ranking requirements are derived.

The shift from the PCI Security Scanning Procedures’ 1 to 5 severity rating method to the ASV Program Guide’s CVSS reflects a maturation of DSS-endorsed vulnerability management processes.

The ASV Program Guide gives four clear exceptions in its CVSS guidance:

  • The vulnerability is not included in the National Vulnerability Database (NVD). Such problems may be independently scored by the ASV using the CVSS scoring system, in which ratings of 4.0 and higher remain a failure.
  • The vulnerability is purely denial of service (DoS) related. In that case, it may be passed despite its CVSS score.
  • The identified vulnerabilities violate PCI DSS despite being labeled low severity.
  • The ASV disagrees with the CVSS score noted in the NVD. The retailer may determine a passing status.

In each case, however, ASVs are directed that they must “document these exceptions under Exceptions, False Positives or Compensating Control, as noted in Appendix B: ASV Scan Report Executive Summary.”

As a result, ASV-documented exception details help provide valuable insights into applied vulnerability management processes for scan customers. With such intelligence merchants/service providers are better empowered to consider the accuracy of independently scored ASV vulnerabilities as they relate to impacted system components and affected services. Further, the information provides useful reference to merchants/service providers that, as of June 30, are required to assess and appropriately rank vulnerability risks wherein overlapping occurrences are frequently identified in the course of internal scanning processes.

If you notice that the exceptions column is incomplete, consider requesting either corrected and certified reports or a letter of explanation and evidence of the process having been corrected in subsequent scans. Not doing so will impede the timely completion of your PCI assessment and threaten the accuracy of attestation.

If your assessment has already been completed and your QSA did not identify such errors, you should ask the QSA why, because it cannot safely be relied upon that he or she won’t again miss the violation in the next assessment. Indeed, this is the very sort of error that, if made, constitutes the potential for Security Standards Council (SSC) initiated QSA remediation. As such, QSAs should want to be aware of and correct the issue.

And if your ASV responds that it has completed annual SSC-required Validated Lab certification without this issue having been raised, it does not prove that the ASV’s procedure is correct. Rather, and far worse, it would only illustrate that the Validated Labs are not uniformly enforcing the ASV Program Guide in whole. Without ASV understanding and willing correction, it may prove very appropriate to offer the PCI Council your feedback via the ASV Feedback Form.

As a retailer, it is your responsibility to manage your compliance. In other words, if your QSA passes you and you know that you should not have passed, it’s your fault and it will be your neck. With ASVs, it’s no different; and it should be no different.

Disagree? Would love to hear from you, either with a comment below or you can zap me an E-mail.


2 Comments | Read Vulnerability Scan Often Says “Pass” Even If You’ve Flunked

  1. Walt Conway Says:


    Great column. You make some good points about external scanning and the importance of merchants checking their ASV results. While all ASVs are tested annually, not all ASVs or scanning engines are “equal.”

    The only thing I would add is to apply the same cautions to a merchant’s *internal* vulnerability scans, too (Requirement 11.2.1). I sometimes run into internal scans where the merchant did not configure their scanning tool properly. For example, it may not scan all ports, or it may “pass” vulnerabilities with a high (i.e., 4.0 or above) CVSS score. QSAs need to check for both these situations.

    Another issue with internal scans is finding and remediating vulnerabilities the merchant has classified as “high” in Requirement 6.2 (see

    If merchants rely on their service providers for these internal scans, they need to make sure the scans are run properly and the results documented. As we all know, a merchant can outsource their processing, but they cannot outsource their responsibility.

  2. Peter Spier Says:

    Thanks, Walt! I agree entirely.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.