May 11, 2018

Last Updated on January 19, 2024

Editor’s Note: This post was originally published in September 2015 and has been updated for accuracy and comprehensiveness.
For some time, CPA firms, qualified security assessors (QSAs) and similar entities that focus on security attestation have been including penetration testing as part of their attestation work. I believe you can make a strong argument for this as a means of substantiating the net effectiveness of the security controls. (A more cynical person might also add that penetration testing in this context is also a means for the attestation firm to increase its revenue.)
A trend we have seen for some time now is organizations conducting “preparatory” application and/or network penetration tests in advance of the penetration testing being performed by the attestation firm. They’re doing this to preemptively find and fix vulnerabilities, thus helping to ensure that their SOC1, SOC2 or PCI Report on Compliance ends up “clean.” The business seeking compliance doesn’t want the attestation firm to include “bad” pen test results in the reports it issues.

CREST Certification

I had an interesting call today from a client who wanted to conduct a penetration test ahead of the penetration test planned by their QSA.  He was not happy with the QSA, as he referred to the previous year’s test by them as “laughably bad” and bemoaned the fact there “isn’t a way to truly vet companies that offer pen testing.”
I had to smile because we just recently became a CREST certified Penetration Testing company.  In brief, CREST accredits companies by conducting detailed audits on their policies, procedures and working practices. They also certify the individuals conducting the work and ensure that both the company and individuals conform to a powerful code of conduct.
Our conversation had two positive impacts:

  1. Our conversation coupled with the CREST certification convinced him to move forward with the testing.
  2. He used CREST (and the previous year’s report) to eliminate the Penetration Testing from the PCI audit (thus saving enough to cover the cost of our test).

Photo of servers helping to illustrate SOC2 penetration testing Getting back to the question at hand:

Should you conduct a preparatory penetration test to help ensure you can acquire a “clean” report?

I “get” saying yes from a business perspective, and can see it’s an unavoidable byproduct of the “demonstrable security and compliance” competitive environment many organizations are living in. Heck, selfishly this is a positive trend for Pivot Point’s penetration testing practices. 
But the security practitioner in me shudders at this intense focus on a clean report as “the destination”—as opposed to ongoing monitoring as “the process” that will keep companies from having to endure an “Equifax moment.” This is especially important given that funds spent on what is effectively redundant penetration testing could potentially be used for some higher-value security initiative.