Know what you’re asking for and what to expect
People often ask for penetration testing without knowing what it really means or does. The word has become ubiquitous within the field of information security and means very different things to individuals and organisations.
Even security professionals are at fault here, interchanging words such as pen test, vulnerability assessments and other related security words to fit the current situation. In some cases the same term is used differently within the same conversation! Unfortunately this can this can lead to an organisation failing to gain the right level of assurance required.
To help organisations understand what it is they require and assist them in provisioning the right security test we have laid out the different types of tests that come under the security testing banner and what you can expect from that test.
An Overview of Security Testing
There are four key types of testing that come under the banner of Security Testing, the most commonly referred to being a pen test or penetration test. The following diagram lays out the different types of security testing and highlights the extent to which automated testing tools are used compared to manual testing. As we move up the pyramid the level of skill required of the tester increases. 
We will now take a look at each of the types of security testing, detailing what it is and what you get from each.
Vulnerability Scan
What is it?
The scan uses automated tools to identify known security issues through matching conditions with known vulnerabilities.
What do you get?
The tool automatically sets the risk level for the results of the scan and no manual verification or interpretation of the results prior to issue takes place. This is great for identifying technical vulnerabilities at a low financial cost. However, it also generates a high level of false positives while missing certain types of issues. This limits the overall level of assurance gained.
Vulnerability Assessment
What is it?
A vulnerability assessment takes a vulnerability scan a step further by using a security tester‘s knowledge to drive an appropriate use of automated tools and test scripts.
What do you get?
The report for the results should be manually created, which places the findings into the context of the environment under test. An example would be removing common false positives from the report and deciding risk levels that should be applied to each report finding to improve business understanding and overall context of a finding. It is great for increasing the level of assurance gained through automated testing, whilst still helping to keep costs low.
Security Assessment
What is it?
A security assessment builds upon a vulnerability assessment by adding manual verification of the results to confirm the level of exposure. It does not though include the use of exploitation code to gain further access to systems.
What do you get?
A security assessment is looking to gain a broad coverage of the systems under test but does not consider the depth of exposure to which a specific vulnerability could lead. False positives should be excluded through the analysis of the results. Security assessments are great for exposing business logic flaws and identifying security vulnerabilities that automated tools are unable to identify. This leads to a higher level of assurance. However, the time and effort required to complete a security assessment are higher than vulnerability scanning and assessments and require a higher level of technical skill to deliver. This will increase the cost of an engagement.
Penetration Test
What is it?
Penetration testing simulates an attack by a malicious party by using tools and manual investigation to identify weaknesses. Testing involves the exploitation of found vulnerabilities to gain further access. Using this approach will result in an understanding of the ability of an attacker to gain access to confidential information, affect data integrity or availability of a service and the respective impact.
What do you get?
This approach looks at the depth and impact of a potential attack, as compared to the security assessment approach that looks at the broader coverage. It is great for understanding the depth of exposure from a vulnerability but it can result in a narrow focus that potentially misses other vulnerabilities that would have been identified through a security assessment. The level of assurance gained is directly associated with the ability of the tester, the scope of engagement and the time and effort allocated.
Finding the right level – some considerations
All levels of security testing are valid assurance activities but it is important that you choose the level that is right for your needs. Organisations need to balance risk appetite, cost, the level of assurance required, the threat landscape and any regulatory requirements (if applicable). In our next blog post we will consider how to align security testing with the threat landscape.