Vulnerability Scan Often Says “Pass” Even If You’ve Flunked
Written by Peter SpierPeter Spier, CISSP, CRISC, CISM, PMP, QSA, PA-QSA is Manager of Professional Services at Fortrex Technologies.
Take a look at your most recent PCI vulnerability scan. Do you see any identified vulnerabilities that are assigned a score of 4.0 or higher? Are you still passing? Is the “Exceptions, False Positives or Compensating Controls” column blank? If so, your approved scanning vendor (ASV) is not doing its job, and this will prevent you from passing your next PCI assessment.
Since PCI 2.0’s implementation earlier this year ASVs have had QSAs closely reviewing their findings and comparing those results against the ASV Program Guide.
The problem here is beyond having the PCI assessment not pass. Retailers will now have to go back to their ASVs, explain the problems and get both sides to work together to fix it. (Yes, the ASVs are going to be thrilled.)
What is worse: That is exactly what the PCI Council’s selected Validation Labs are intended to test for, and solve, on an annual basis to prevent just such an occurrence.
The requirement forcing this change: the testing procedure of DSS v2.0 requirement 11.2.2.b, which instructs QSAs to “review the results of each quarterly scan to ensure that they satisfy the ASV Program Guide requirements (for example, no vulnerabilities rated higher than a 4.0 by the CVSS and no automatic failures).”
The ASV Program Guide has an exceptions column for a reason. When no one fills it out, then the retailer’s ability to be compliant won’t happen. Why would ASVs do this? The answer is simple—and it’s hardly a defense: The column is commonly overlooked. This fact raises a more frightening question: Why would ASVs, who focus on minute details for a living, not adhere to a written requirement? The answer: No one knows, including—perhaps—not even the ASV. In short, retailers will now have to go back to their ASVs.
Another aspect is the change in how security issues are ranked. The adoption of the CVSS and its open framework-based calculation methodology permits both ASVs and merchants/service providers to consider multiple vulnerability criteria and effectively determine the probable impact to affected systems. It is this very capability that establishes the foundation upon which DSS v2.0 risk ranking requirements are derived.
The shift from the PCI Security Scanning Procedures’ 1 to 5 severity rating method to the ASV Program Guide’s CVSS reflects a maturation of DSS-endorsed vulnerability management processes.
The ASV Program Guide gives four clear exceptions in its CVSS guidance:
- The vulnerability is not included in the National Vulnerability Database (NVD). Such problems may be independently scored by the ASV using the CVSS scoring system, in which ratings of 4.0 and higher remain a failure.
- The vulnerability is purely denial of service (DoS) related. In that case, it may be passed despite its CVSS score.
- The identified vulnerabilities violate PCI DSS despite being labeled low severity.
- The ASV disagrees with the CVSS score noted in the NVD. The retailer may determine a passing status.
In each case, however, ASVs are directed that they must “document these exceptions under Exceptions, False Positives or Compensating Control, as noted in Appendix B: ASV Scan Report Executive Summary.”
As a result, ASV-documented exception details help provide valuable insights into applied vulnerability management processes for scan customers. With such intelligence merchants/service providers are better empowered to consider the accuracy of independently scored ASV vulnerabilities as they relate to impacted system components and affected services. Further, the information provides useful reference to merchants/service providers that, as of June 30, are required to assess and appropriately rank vulnerability risks wherein overlapping occurrences are frequently identified in the course of internal scanning processes.
If you notice that the exceptions column is incomplete, consider requesting either corrected and certified reports or a letter of explanation and evidence of the process having been corrected in subsequent scans. Not doing so will impede the timely completion of your PCI assessment and threaten the accuracy of attestation.
If your assessment has already been completed and your QSA did not identify such errors, you should ask the QSA why, because it cannot safely be relied upon that he or she won’t again miss the violation in the next assessment. Indeed, this is the very sort of error that, if made, constitutes the potential for Security Standards Council (SSC) initiated QSA remediation. As such, QSAs should want to be aware of and correct the issue.
And if your ASV responds that it has completed annual SSC-required Validated Lab certification without this issue having been raised, it does not prove that the ASV’s procedure is correct. Rather, and far worse, it would only illustrate that the Validated Labs are not uniformly enforcing the ASV Program Guide in whole. Without ASV understanding and willing correction, it may prove very appropriate to offer the PCI Council your feedback via the ASV Feedback Form.
As a retailer, it is your responsibility to manage your compliance. In other words, if your QSA passes you and you know that you should not have passed, it’s your fault and it will be your neck. With ASVs, it’s no different; and it should be no different.
Disagree? Would love to hear from you, either with a comment below or you can zap me an E-mail.
July 26th, 2012 at 2:00 pm
Peter,
Great column. You make some good points about external scanning and the importance of merchants checking their ASV results. While all ASVs are tested annually, not all ASVs or scanning engines are “equal.”
The only thing I would add is to apply the same cautions to a merchant’s *internal* vulnerability scans, too (Requirement 11.2.1). I sometimes run into internal scans where the merchant did not configure their scanning tool properly. For example, it may not scan all ports, or it may “pass” vulnerabilities with a high (i.e., 4.0 or above) CVSS score. QSAs need to check for both these situations.
Another issue with internal scans is finding and remediating vulnerabilities the merchant has classified as “high” in Requirement 6.2 (see http://storefrontbacktalk.com/securityfraud/new-pci-rules-will-force-retailers-to-set-the-risk-level/).
If merchants rely on their service providers for these internal scans, they need to make sure the scans are run properly and the results documented. As we all know, a merchant can outsource their processing, but they cannot outsource their responsibility.
July 26th, 2012 at 3:14 pm
Thanks, Walt! I agree entirely.