The "RISKY BUSINESS" Blog

Had an interesting (and familiar) call with a potential client yesterday regarding a Penetration Testing proposal. 

"We really like your proposal, but honestly, we are trying to figure out why you are so much cheaper than the other firm that we are looking at ... "    

While it is not always true, I generally believe that there is some credence to the axiom, "you get what you pay for".  It was obvious the client was of the same mind - so our notably lower price put us at a disadvantage  We had a rather lengthy discussion about the issue, and as it is a conversation that I have had many times prior (as the higher priced vendor as well), and I have to assume others are experiencing the same issue, I figured it's a conversation worth sharing.

We agreed that there are 3 probable reasons for pricing to be "significantly" different across different vendors:

1. Project Approach - Unfortunately, there is really no standard definition for a Penetration Test or Vulnerability Assessment. Very frequently we find notable differences in approach that have a very significant impact on the time required to perform a test.
2. Personnel Cost - The hourly rate of the personnel conducting the testing are dependent upon a number of factors including organizational size, sales/marketing/project management costs, cost of the personnel.
3. Equipment/Materials Cost - Every pen testing organization incurs costs for the tools and equipment (vulnerability assessment tools, laptops, pen testing tools, etc.) that they need to "recapture" across projects.

Project Approach

As we don't want to compare Apples & Pomegranates, it is critical that you really understand how each vendor defines a Vulnerability Assessment/Penetration Test. To help clients understand our definition, we have published very specific details of what each class of testing includes.  In the conversation above we found that both our Vulnerability Assessment and Penetration Test were more comprehensive than our competitors.  However, as our methodology employs statistical sampling (validated by 9 years of testing experience) we were going to actively attack less systems than our competitor.  When we explained our rationale, the client agreed, and was satisfied that our approach would fully achieve their objectives.

Personnel Cost

We run a pretty lean organization.  We don't employ a sales team; we prefer to have our consultants who actually perform or manage the work, work with clients to find the best approach.  We are also lucky that the content on our website, recurring clients, and client referrals provide enough leads that we don't need to spend a lot of money on expensive marketing mechanisms (brochures, trade-shows, etc.).  Because we don't need to pay for salespeople and trade-show exhibits the hourly rate we charge for our consultants' time is often a bit lower than our competitors.

Materials  Cost

We don't use "junior" consultants; the least amount of experience that any of our current consultants has is 11 years. So, we don't need to employ expensive "automated" tools (e.g., Core Impact) to compensate for a lack of experience.  Accordingly, we don't need to pass the ~$40,000 per year per consultant in license costs along to our clients. 

So what do patios have in common with penetration testing?  We recently had a patio built in our backyard.  When I collected three proposals, two were very close in price, and the third was almost 40% lower.  I was tempted to dismiss the lowest proposal, but instead asked the contractor to explain the price difference:

1. Project Approach: He had structured the proposal assuming that we could make a slight change to the skirting on our current deck that would allow him to use a less expensive retaining wall in one part of the design as it would no longer be visible.
2. Personnel Cost: He explained that as a smaller company they didn't employ separate designers and project managers. This means less "lost" time.  Further, the designer who laid out the project would actually be the person onsite which also ensured that "nothing would be lost in translation".
3. Materials Cost: They were using pavers that were sourced locally (NJ) where both competitors were sourcing them from Canada. The pavers were indistinguishable from each other and we wouldn't have to pay for the cost to ship thousands of pounds of pavers all the way from Canada.

Once I was comfortable that I was comparing  "apples and apples" , it was time to "Trust, but Verify".  I met with three references where the work looked great and the owners were all extremely satisfied.

Right now I'm a happy camper - adult beverage in hand, notebook on lap, feet resting on a patio that we are absolutely thrilled with (built by the lowest bidder!).  I guess sometimes you can get more than you pay for!

Call me jaded but my level of distrust goes up when an organization:

• Pronounces itself as the "Information Security" guardian of an industry segment
• Develops a "proprietary" framework/standard that it rationalizes based on the uniqueness of the industry
• Develops its own certification programs and charges exorbitant fees to qualify entities for certification

Perhaps I am just cynical based on our mutual experience with the  "Payment Card Industry Standards Council" and the PCI-DSS program which has only succeeded in reducing the risk to the payment brands at the expense of the merchants and the card owners. 

My lack of trust in all things PCI-related is well illustrated by the recent "gift card" I gave to my nephew for graduating high school.  He called me embarrassed to report that the card balance was zero and that the credit card company had told him that it had already been used.  After a considerable time on the line the credit card company finally "admitted" that the card had been used five days prior to my purchasing it.   If the credit card companies can't prevent/detect this proactively - what faith can we have in them or their ability to govern?

On learning of HITRUST I was hopeful that it would be different.  But on first blush it looks like they have stolen a page or two out of the Payment Card Industry Standards Council playbook.  For example,  the costs involved for a consulting/auditing firm to be "qualified" exceed $20,000.  Initially the standard was only available at a significant cost. On a more positive note, the Alliance recently announced that the Common Security Framework (the standard) will now available for free (however, after 30 minutes on the site today - I still can't find it).

Hopefully, I am wrong and HITRUST will be a shining beacon for Information Assurance - but I'm not so sure.  I would rather have a comprehensive, non-industry specific, standard (e.g., ISO-27001) achieve sufficient status that these entities don't feel the need to develop proprietary standards.  The benefits would be significant.

While we can all agree on the importance of protecting information and information systems, it's much harder to agree on how we define the state of being secure.  Is it the state where all information security risks are completely eliminated (not possible)?  Is it the state where all security risks are mitigated to an acceptable level (which risks and whose definition of acceptable)?  Even if we reach consensus and achieve the agreed upon state - we all understand that it is fleeting and will only last until a change (often outside of our control) denigrates it.

It is because the assurance we receive is often only valid at an "instant in time" (e.g., a penetration test) that trust is so important.  If we don't trust that the person/team/organization responsible to ensure critical information is handled in the manner we aspire to moving forward, then we really have little or no assurance. 

We were recently engaged by a pharmaceutical firm to assess whether a third party developed and hosted implementation of a multi-million dollar clinical trials solution achieved the security objectives defined in the contract. The report would also form the basis of a new contract intended to "remedy" the existing challenges. 

The meeting took a very interesting turn when I asked "Should we really be discussing a new contract when it is obvious that the service provider has proven they are not trustworthy?" (the impact of the vendor failing to achieve critical security objectives could cost the pharmaceutical tens of millions)

For my money, no contract, no matter how carefully worded, will make a company trustworthy.  And without trust we have no confidence/assurance that critical information will remain secure. 

It wouldn't be a stretch to say that Windows 2000 and all the Windows Iterations before it were insecure by default and that Windows 2003 and forward are secure by default.  So why is it that Windows 2003 may make your environment more secure while at the same time it makes your environment less trustworthy?

I believe there are two predominant reasons:

1. As we become more confident in Windows security we become less vigilant in our due diligence to validate that Windows is actually operating as intended.
2. Even when we are vigilant, a network penetration test, the defacto "gold standard" for substantiative testing that an environment as a whole is operating is intended is no longer sufficient.

Windows 2003 is significantly more secure by default than Windows 2000, however, it is not immune to less than optimal configuration and vulnerability patch management issues.  Further, there are often other "less important" (and likely less secure) older platforms, network devices, or older applications that may denigrate the security (and trustworthiness) of the environment as a whole.  So Trust the environment, but remain diligent and verify by appropriate activities.

That segues directly to point two, the "appropriate" activities have changed. As an environment becomes more technically secure there is an implicit and incorrect assumption that the probability that it will be "hacked" definitely goes down.  In fact, I would argue that if the attacker is intentioned, that it does not go down at all.  An intentioned attacker will just alter his attack vector to seek the "new" weakest link (e.g., Social Engineering).   There is an old adage that professionals don't hack systems they hack people (think Kevin Mitnick).

So as your environment grows more secure the tests that you use to measure its security/trustworthiness need to change as well. Many of our more risk averse clients are adding social engineering and/or physical penetration testing into their "verify" activities.  Should you?

Our “slidepaper” on Network Vulnerability Assessment: Key Decision Points, will help broaden your knowledge on VAPT. Find this and more on our Penetration Testing resource page.