NIST/FISMA information security guidance is generally outstanding and more prescriptive than most other forms of guidance. However, there are limits to any standard’s ability to be prescriptive. These limits generally relate to differing technological architectures, the evolution of technology/assessment tools, and differing risk levels (which mandate different levels of testing).
These limits are well illustrated by a recent call that I had with a potential client. Our conversation began with him telling me: “This should be a pretty quick conversation – I just need a price for a NIST Penetration Test.”
As you probably already guessed, the call was not nearly as quick as the client had hoped. The challenge is that there is no definitive definition of a NIST Penetration Test. The “centerpiece” of the NIST information security framework is NIST 800-53 – hence, it was the first document I referred to during our conversation. Since Revision 3 of the framework there has been some definition regarding penetration testing:
“Penetration testing exercises both physical and technical security controls. A standard method for penetration testing consists of: (i) pretest analysis based on full knowledge of the target system; (ii) pretest identification of potential vulnerabilities based on pretest analysis; and (iii) testing designed to determine exploitability of identified vulnerabilities. Detailed rules of engagement are agreed upon by all parties before the commencement of any penetration testing scenario. These rules of engagement are correlated with the tools, techniques, and procedures that are anticipated to be employed by threat-sources in carrying out attacks. An organizational assessment of risk guides the decision on the level of independence required for penetration agents or penetration teams conducting penetration testing. Red team exercises are conducted as a simulated adversarial attempt to compromise organizational missions and/or business processes to provide a comprehensive assessment of the security capability of the information system and organization. While penetration testing may be laboratory-based testing, red team exercises are intended to be more comprehensive in nature and reflect real-world conditions. Information system monitoring, malicious user testing, penetration testing, red-team exercises, and other forms of security testing (e.g., independent verification and validation) are conducted to improve the readiness of the organization by exercising organizational capabilities and indicating current performance levels as a means of focusing organizational actions to improve the security state of the system and organization. Testing is conducted in accordance with applicable federal laws, Executive Orders, directives, policies, regulations, and standards. Testing methods are approved by authorizing officials in coordination with the organization’s Risk Executive Function. Vulnerabilities uncovered during red team exercises are incorporated into the vulnerability remediation process.”
While I have read NIST-800-53 dozens of times, I did not recall the physical security element mentioned above. So one could argue that a NIST Penetration Test should include physical penetration testing, although fewer than 5% of all penetration tests that Pivot Point Security conducts include physical security testing. Further, most physical penetration testing naturally encompasses some form of social engineering (e.g., impersonation of an employee or authorized visitor, fake employee badges, tailgating).
As the guidance specifically cites “ …more comprehensive in nature and reflect real-world conditions” it is also arguable that a NIST penetration test would include some level of social engineering. The guidance is non-prescriptive in defining the extent/rigor of the testing Does reconnaissance include deep web? Does vulnerability assessment include Man-in-the-Middle Attacks? Does exploit include Denial of Service attacks? The use of social engineering in a NIST penetration test makes sense and is covered by the idea of “rules of engagement correlated … threat-sources carrying out attack.” In other words, the extent and rigor of the testing should be proportional to the risk of a breach. A system protecting information on troop movement in Afghanistan should be tested more thoroughly than one housing public land deed records. NIST does provide some penetration testing guidance in NIST SP 800-115; however, the standard relates to a more holistic/robust Security Assessment process. Further, the document is over five years old so unless a test is specifically referred to as an 800-115 test, we don’t generally use that guidance.
So during my initial call with the client, we agreed that a NIST penetration test is a test aligned with good practice where the coverage (e.g., application, network, exploit types, social engineering, and physical security) and extent/rigor is aligned with the risk profile of the solution being assessed. The challenge with this approach is that the test was being requested by a third party and we wanted to make sure that the testing was going to meet their requirements before actually conducting the test.
There are two approaches to this chicken-and-egg scenario: 1) Request clarification from the client on their expectation, or 2) Get the client’s blessing on the scope you intend to execute. While there is no right/wrong approach, over the years we have found that in most instances the former ends up with a smaller scope of work.
At the end of the day, this particular client was happy with a relatively straightforward network and application penetration test that did not include physical security testing or social engineering. This was the best fit with the risk profile of the system being tested — and also a good-practice application of NIST guidance to penetration testing.