Information Security Blog

Web Application Penetration Testing: Production or QA?

Web Application Penetration Testing: Production or QA?

Appsec Webinar OWASP blocks_thumbRecently I spoke with one of our highly regulated clients in the financial industry, who was getting “beat up” in an audit because we had jointly chosen to run the web application penetration tests in a “near production” environment rather than in their production environment.  Our thought process was as follows:

  • The web application we were testing processes highly sensitive data, where integrity is critical.
  • Web application penetration testing tools attempt to identify persistent injection vulnerabilities that may result in data being written to the database, thus impacting data integrity.
  • Reversing successful injections in a production application/database may not be possible and/or would be more challenging if inserted data “triggers” additional actions (e.g., a stored procedure that processes our injected data).
  • Testing in a near production environment that is identically configured (e.g., code base, web server, app server, database tier) provides equivalent value at virtually zero risk.
  • The client would make any required changes (dictated by the test results) in production after those changes were validated in near production.
  • Pivot Point Security would perform a network vulnerability assessment and penetration test in the production environment to ensure the security of the underlying infrastructure.

I was surprised when the auditor disagreed with this approach, especially, as it is the approach I see leveraged by the vast majority of our client base. The auditor was insistent (despite the fact that our client could document the identical nature of the near-production and production setups) that the testing should have been done in production.  When it was explained that the risk associated with a successful injection was deemed too high, he cited that the web application vulnerability scanner should have been used in a “safe” mode.

Needless to say, running in a safe mode means that it is not performing those tests that are not deemed safe. So by running in a safe mode it may miss critical vulnerabilities – essentially a false negative – which is the most dangerous result you can have in a pen test.

So he asked the auditor “ Do you want me to run:”

  1. A “safe”  test in production that may miss things
  2. A “comprehensive” test in near-production whose results are applied to production
  3. A “comprehensive” test in production that may impact the integrity of client data

To my surprise he didn’t care whether it was ‘1’ or ‘3’ – as long as it was run in production.  I find the answer remarkable. Do you?

I found one other thing suprising — that I could not find any “standard” guidance on this from resources like OWASP or Microsoft.



Is ISO 27001 Right for (Y)our Organization?


Thinking about ISO 27001 Certification? View our free On-Demand ISO 27001 Webinar

  • How to deal with increasing threats
  • How to manage multiple regulatory requirements
  • How to handle client requests for attestation
  • To validate that significant changes did not have unanticipated results

Free Download: ISO 27001 Vendor Selection Toolkit

“ISOOur ISO 27001 Toolkit will help you to select an ISO 27001 consulting firm.
  • Review the Issues Critical to Your Environment
  • "Vet" Vendor Qualifications
  • Compare the Top 3 Vendors
  • Sample RFP Included

Download: Information Security Attestation Guide

Information Security GuideA Best-Practices Guide to Information Security Attestation

Download our proven Information Security Guide to simplify the process of protecting your data, proving you’re secure and growing your business.

Free Download: ISO 27001 Implementation Roadmap

ISO 27001 RoadmapHave no fear – our “roadmap” will guide you, step by step, through the entire ISO 27001 process.

Getting to ISO 27001 certification is a process made up of things you already know – and things you may already be doing!

About the Author:

John Verry (CISA, 27001 Certified Lead Auditor, CCSE, CRISC) is Pivot Point's resident "Security Sherpa". He is lucky enough to spend most of his day helping clients develop a road map to address security, compliance, and attestation requirements.


  1. Matthew  March 18, 2014

    Simply an ignorant auditor, or possibly mis-communication? If the auditor was saying that a scan against the production environment using the “safe” setting was needed, I see little reason not to do it IN ADDITION TO the proper, full-speed scan and pen test against the QA environment.

  2. Drama Monster  July 31, 2014

    I could write you a book on this subject, but a simple answer is that every situation is different. Those who are responsible for keeping the target environment in a working state, would be wise to create their own guidance for such testing events. Therein lies the problem, and it ranges from never having had a pen-test before all the way to full-cooperation, or even(shudder) compliance. It is easy to make an absolute statement about where pen-testing should be done, but that wouldn’t take into account the production reality all possible environments. There are lots of security training programs out there that warn against the woes of testing production environments. Other groups have reasons why they will never do so, and this completely relates to the type of infrastructure involved, they’re not all the same. If you want standard guidance for an internal team, then you best quickly go about making your own.

    Can you really say that a group who decides to create software, and decides to do all pen-testing on production (from day one), will never ever run into any snags? There might be a group wherein the risks of failure do not out weigh the costs leaving a systems unprotected, but I think we would all have to agree there is are certain polarity to such a statement which is quite extreme. The polemics involved in calculating these kinds of tradeoffs, most likely only apply to only a very few groups and virtually none of them are regular businesses. Its nice to think that you could pen-test in production, but face it pentesting/war is hell. Do you really want hell? Most likely what your are looking for is a bomb test site, not a full out war.


Add a Comment

Share This