Re: Cigital Ponders: Is Penetration Testing Security Testing?

jOHN — Thu, 10 Apr 2008 10:42:42 -0000

Dave,

You have a crucial point. Penetration testing is a very common and valuable technique for security testing. The question I intended to ponder was more accurately, "Am I done security testing after I've done a penetration testing?" I believe the answer is no.

I'd like to respond to your notion of security requirements. I agree that documenting security controls/features/widgets is untenable. Almost every team is ill-equipped to do so. What teams do have--uniquely in fact--is an understanding of what failures would impact their system's business objectives. Security requirements (and misuse/abuse cases) should focus here, not on the technology-specifics of a particular control (use SHA-256 for...). Technology specifics can be pulled from security standards, reused across the organization to relieve individual teams of this burden.

This doesn't let teams off the hook for specificity. Once a stakeholder has figured out which impacts pose the greatest risk from intentional attack, they can engage in a rather detailed process of specifying how the application should resist attack. This proceeds much like performance and scalability requirements elicitation--entwined with design and architecture discussions. While a multitude of tactical vulnerabilities could be fleshed out by each attack, they shouldn't be at this level. Leave that to test design and test case creation. At the conceptual level, attacks just don't change that often. They shouldn't produce undue burden on the requirements management process.

Re: Cigital Ponders: Is Penetration Testing Security Testing?

ivan — Thu, 10 Apr 2008 00:45:23 -0000

It appears that everyone uses the term "penetration testing" in ways that convey multiple & different meanings and that creates a bit of confusion. In the context of Cigital's blogpost -and generally among the CS and software engineering crowds biased towards SDLC as the "right thing" to "really" improve security- penetration testing is implicitly assumed to mean "finding bugs in software artifacts" and "penetration testing tools" are software artifacts that find bugs in other software artifacts.

That interpretation of the term "penetration testing" is substantially different from the practice aimed at finding weaknesses in an *organization's security posture and exploiting them* as an attacker would (or as close as possible to that) and from the tools that assist that practice.

In the later context, penetration testing is much more than finding bugs in software and it may/should not even be considered a part of the software development lifecycle. There are tools that fit both definitions of "penetration testing" and I posit that they are both quite useful.

...or putting it in another way: l0phcrack and toneloc are definitely considered penetration testing tools within a large set of security practitioners (hm ok, I'm not sure about Toneloc) yet they provide 0% code coverage... are they useful?

Matasano Chargen - Latest Comments in Cigital Ponders: Is Penetration Testing Security Testing?

Re: Cigital Ponders: Is Penetration Testing Security Testing?

Re: Cigital Ponders: Is Penetration Testing Security Testing?