Vulnerability assessment – Penetration testing, can things go wrong?
There seems to be a market trend to offer a vulnerability assessment and package it as a
penetration testing exercise.
Both are security controls in ISO/IEC 27001: 2013 Annex A and both have distinct purpose
and deliverables. In addition, they both feature quite heavily within the PCI DSS to varying
degrees, depending on the level of compliance required by the acquiring bank.
However, nearly all organisations that are required to be PCI DSS compliant will have to
conduct at least some vulnerability scanning. Having said that, every organisation should
have both in its arsenal to verify the effectiveness of its information security governance.
Vulnerability assessment is a process utilising people and applications to assess
organisational infrastructure for the presence of known vulnerabilities. By default,
vulnerability scanning applications are automated tools that conduct vulnerability
assessments with nominal human input. In these assessments, human input is
Configure the type of scan (authenticated or anonymous)
Identify target hosts (DMZ, specific VLAN, network equipment, etc.)
Review and prioritise findings.
Most, if not all, vulnerability assessment tools will, by default, prioritise findings based on
the criteria that are publicly available. However, it is of utmost importance that the security
team reviews the findings to ensure they are not false positives and to prioritise those findings
This is particularly pertinent for PCI DSS compliance, as the Standard has some strict
requirements around how quickly vulnerabilities should be remediated in order to prevent
Confusion surrounding vulnerability assessments, particularly from non-technical
businesses, is the perception that a vulnerability assessment is ‘finding’ security issues
and ‘hacking’ at the same time. There are automated tools that will do discovery and
exploiting, but these fall into a different category.
The essence of a vulnerability assessment is the discovery of potential security issues,
which should be reviewed against organisational risk appetite, and measuring the
effectiveness of other security controls (i.e. patch management).
With the increase in data breaches and regulatory penalties, there is growing interest
from organisations for independent verification of implemented security controls.
The common approach to evaluate the effectiveness of security controls is to involve an
independent penetration testing partner which acts on behalf of the organisation.
Again, the Standard has some stringent requirements regarding the methodology that is
used for penetration testing and details specific types of test that must be included within
a PCI DSS penetration test.
As its name suggests, penetration testing is an exercise to exploit vulnerabilities discovered
during the vulnerability assessment of a target. This is the crucial statement that differentiates
vulnerability assessment from penetration testing.
During a vulnerability assessment or discovery, the assessor will identify whether there are
any security vulnerabilities on target platforms and the penetration tester will attempt to
exploit those vulnerabilities.
Every serious penetration tester will conduct a vulnerability assessment on its target, commonly
known as part pen-test surveillance. Once the penetration tester is aware of vulnerabilities,
the next phase is to attempt to exploit those vulnerabilities and see how much data is potentially
exposed to a malicious attacker.
Now that the difference between vulnerability and penetration assessment is clarified, there are
a couple of key elements to penetration testing that must be addressed before the organisation
decides to engage in it:
Assess the impact
Type of penetration test
Penetration test report evaluation and action.
For more mature environments, depending on the risk profile, ‘black hat’ penetration testing
may be an option. Black hat penetration testing, as opposed to ‘white hat’, is where an organisation
engages a third-party to conduct penetration testing on its infrastructure (internal or external),
but without the security response team knowing the time or target of the test.
It is important to understand that without proper preparation and planning of penetration testing,
things can and will go wrong.