Special Publication 800-115

Technical Guide toInformation Security Testingand Assessment
Recommendations of the National Instituteof Standards and Technology
Karen Scarfone
Murugiah Souppaya
Amanda Cody
Angela Orebaugh

NIST Special Publication 800-115

Technical Guide to Information Security
Testing and Assessment
Recommendations of the National
Institute of Standards and Technology
Karen Scarfone
Murugiah Souppaya
Amanda Cody
Angela Orebaugh

C O M P U T E R

S E C U R I T Y

Computer Security Division
Information Technology Laboratory
National Institute of Standards and Technology
Gaithersburg, MD 20899-8930
September 2008

U.S. Department of Commerce

Carlos M. Gutierrez, Secretary
National Institute of Standards and Technology

Dr. Patrick D. Gallagher, Deputy Director

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Reports on Computer Systems Technology
The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology(NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nation’s
measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof ofconcept implementations, and technical analysis to advance the development and productive use of
information technology (IT). ITL’s responsibilities include the development of technical, physical,
administrative, and management standards and guidelines for the cost-effective security and privacy ofsensitive unclassified information in Federal computer systems. This Special Publication 800-series
reports on ITL’s research, guidance, and outreach efforts in computer security and its collaborative
activities with industry, government, and academic organizations.

National Institute of Standards and Technology Special Publication 800-115
Natl. Inst. Stand. Technol. Spec. Publ. 800-115, 80 pages (Sep. 2008)

Certain commercial entities, equipment, or materials may be identified in thisdocument in order to describe an experimental procedure or concept adequately.
Such identification is not intended to imply recommendation or endorsement by the
National Institute of Standards and Technology, nor is it intended to imply that theentities, materials, or equipment are necessarily the best available for the purpose.

ii

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Acknowledgements

The authors, Karen Scarfone and Murugiah Souppaya of the National Institute of Standards andTechnology (NIST) and Amanda Cody and Angela Orebaugh of Booz Allen Hamilton, wish to thanktheir colleagues who reviewed drafts of this document and contributed to its technical content. The
authors would like to acknowledge John Connor, Tim Grance, Blair Heiserman, Arnold Johnson, Richard
Kissel, Ron Ross, Matt Scholl, and Pat Toth of NIST and Steve Allison, Derrick Dicoi, Daniel Owens,
Victoria Thompson, Selena Tonti, Theodore Winograd, and Gregg Zepp of Booz Allen Hamilton for theirkeen and insightful assistance throughout the development of the document. The authors appreciate all
the feedback provided during the public comment period, especially by Marshall Abrams, Karen Quigg,
and others from MITRE Corporation; William Mills of SphereCom Enterprises; and representatives fromthe Financial Management Service (Department of the Treasury) and the Department of Health andHuman Services (HHS).

Trademark Information
All names are registered trademarks or trademarks of their respective companies.

iii

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Table of Contents
Executive Summary..............................................................................................................ES-1
1.

Introduction ......................................................................................................................1-1
1.1
1.2
1.3
1.4

2.

Security Testing and Examination Overview ................................................................2-1
2.1
2.2
2.3
2.4

3.

4.5

Network Discovery ...................................................................................................4-1
Network Port and Service Identification ...................................................................4-3
Vulnerability Scanning .............................................................................................4-4
Wireless Scanning ...................................................................................................4-6
4.4.1 Passive Wireless Scanning ..........................................................................4-8
4.4.2 Active Wireless Scanning .............................................................................4-9
4.4.3 Wireless Device Location Tracking ..............................................................4-9
4.4.4 Bluetooth Scanning ....................................................................................4-10
Summary................................................................................................................4-10

Target Vulnerability Validation Techniques ..................................................................5-1
5.1
5.2

5.3
5.4
6.

Documentation Review ............................................................................................3-1
Log Review ..............................................................................................................3-1
Ruleset Review ........................................................................................................3-2
System Configuration Review ..................................................................................3-3
Network Sniffing.......................................................................................................3-4
File Integrity Checking .............................................................................................3-4
Summary..................................................................................................................3-5

Target Identification and Analysis Techniques.............................................................4-1
4.1
4.2
4.3
4.4

5.

Information Security Assessment Methodology .......................................................2-1
Technical Assessment Techniques .........................................................................2-2
Comparing Tests and Examinations ........................................................................2-3
Testing Viewpoints...................................................................................................2-4
2.4.1 External and Internal ....................................................................................2-4
2.4.2 Overt and Covert ..........................................................................................2-5

Review Techniques..........................................................................................................3-1
3.1
3.2
3.3
3.4
3.5
3.6
3.7

4.

Authority...................................................................................................................1-1
Purpose and Scope .................................................................................................1-1
Audience ..................................................................................................................1-1
Document Structure .................................................................................................1-2

Password Cracking ..................................................................................................5-1
Penetration Testing..................................................................................................5-2
5.2.1 Penetration Testing Phases .........................................................................5-2
5.2.2 Penetration Testing Logistics .......................................................................5-5
Social Engineering ...................................................................................................5-6
Summary..................................................................................................................5-7

Security Assessment Planning.......................................................................................6-1
6.1
6.2
6.3

Developing a Security Assessment Policy ...............................................................6-1
Prioritizing and Scheduling Assessments ................................................................6-1
Selecting and Customizing Techniques...................................................................6-3

iv

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

6.4

6.5
6.6
6.7
7.

Security Assessment Execution.....................................................................................7-1
7.1
7.2
7.3
7.4

8.

Assessment Logistics ..............................................................................................6-4
6.4.1 Assessor Selection and Skills.......................................................................6-5
6.4.2 Location Selection ........................................................................................6-6
6.4.3 Technical Tools and Resources Selection ...................................................6-8
Assessment Plan Development .............................................................................6-10
Legal Considerations .............................................................................................6-12
Summary................................................................................................................6-12
Coordination.............................................................................................................7-1
Assessing.................................................................................................................7-2
Analysis....................................................................................................................7-3
Data Handling ..........................................................................................................7-4
7.4.1 Data Collection .............................................................................................7-5
7.4.2 Data Storage ................................................................................................7-5
7.4.3 Data Transmission........................................................................................7-6
7.4.4 Data Destruction...........................................................................................7-7

Post-Testing Activities ....................................................................................................8-1
8.1
8.2
8.3

Mitigation Recommendations...................................................................................8-1
Reporting .................................................................................................................8-1
Remediation/Mitigation ............................................................................................8-2

List of Appendices
Appendix A— Live CD Distributions for Security Testing .................................................. A-1
Appendix B— Rules of Engagement Template .................................................................... B-1
Appendix C— Application Security Testing and Examination ........................................... C-1
Appendix D— Remote Access Testing ................................................................................. D-1
Appendix E— Resources ....................................................................................................... E-1
Appendix F— Glossary .......................................................................................................... F-1
Appendix G— Acronyms and Abbreviations .......................................................................G-1

List of Tables
Table 3-1. Review Techniques .................................................................................................3-5
Table 3-2. Baseline Skill Set for Review Techniques ...............................................................3-5
Table 4-1. Target Identification and Analysis Techniques ......................................................4-10
Table 4-2. Baseline Skill Set for Target Identification and Analysis Techniques ....................4-11v

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Table 5-1. Target Vulnerability Validation Techniques .............................................................5-7
Table 5-2. Security Testing Knowledge, Skills, and Abilities ....................................................5-7
Table A-1. BackTrack Toolkit Sample...................................................................................... A-1
Table A-2. Knoppix STD Toolkit Sample ................................................................................. A-2
Table E-1. Related NIST Documents....................................................................................... E-1
Table E-2. Online Resources................................................................................................... E-1

List of Figures
Figure 5-1. Four-Stage Penetration Testing Methodology........................................................5-3
Figure 5-2. Attack Phase Steps with Loopback to Discovery Phase ........................................5-4

vi

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Executive Summary
An information security assessment is the process of determining how effectively an entity being assessed(e.g., host, system, network, procedure, person—known as the assessment object) meets specific security
objectives. Three types of assessment methods can be used to accomplish this—testing, examination, andinterviewing. Testing is the process of exercising one or more assessment objects under specified
conditions to compare actual and expected behaviors. Examination is the process of checking, inspecting,reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding,achieve clarification, or obtain evidence. Interviewing is the process of conducting discussions withindividuals or groups within an organization to facilitate understanding, achieve clarification, or identify
the location of evidence. Assessment results are used to support the determination of security control
effectiveness over time.
This document is a guide to the basic technical aspects of conducting information security assessments. It
presents technical testing and examination methods and techniques that an organization might use as partof an assessment, and offers insights to assessors on their execution and the potential impact they may
have on systems and networks. For an assessment to be successful and have a positive impact on the
security posture of a system (and ultimately the entire organization), elements beyond the execution of
testing and examination must support the technical process. Suggestions for these activities—including arobust planning process, root cause analysis, and tailored reporting—are also presented in this guide.
The processes and technical guidance presented in this document enable organizations to: Develop information security assessment policy, methodology, and individual roles andresponsibilities related to the technical aspects of assessment
 Accurately plan for a technical information security assessment by providing guidance on
determining which systems to assess and the approach for assessment, addressing logisticalconsiderations, developing an assessment plan, and ensuring legal and policy considerations are
addressed
 Safely and effectively execute a technical information security assessment using the presented
methods and techniques, and respond to any incidents that may occur during the assessment
 Appropriately handle technical data (collection, storage, transmission, and destruction)
throughout the assessment process Conduct analysis and reporting to translate technical findings into risk mitigation actions that will
improve the organization’s security posture.
The information presented in this publication is intended to be used for a variety of assessment purposes.
For example, some assessments focus on verifying that a particular security control (or controls) meetsrequirements, while others are intended to identify, validate, and assess a system’s exploitable security
weaknesses. Assessments are also performed to increase an organization’s ability to maintain a proactive
computer network defense. Assessments are not meant to take the place of implementing security
controls and maintaining system security.
To accomplish technical security assessments and ensure that technical security testing and examinations
provide maximum value, NIST recommends that organizations: Establish an information security assessment policy. This identifies the organization’s
requirements for executing assessments, and provides accountability for the appropriate

ES-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

individuals to ensure assessments are conducted in accordance with these requirements. Topics
that an assessment policy should address include the organizational requirements with whichassessments must comply, roles and responsibilities, adherence to an established assessment
methodology, assessment frequency, and documentation requirements.
 Implement a repeatable and documented assessment methodology. This provides
consistency and structure to assessments, expedites the transition of new assessment staff, andaddresses resource constraints associated with assessments. Using such a methodology enables
organizations to maximize the value of assessments while minimizing possible risks introducedby certain technical assessment techniques. These risks can range from not gathering sufficient
information on the organization’s security posture for fear of impacting system functionality toaffecting the system or network availability by executing techniques without the proper
safeguards in place. Processes that minimize risk caused by certain assessment techniques
include using skilled assessors, developing comprehensive assessment plans, logging assessor
activities, performing testing off-hours, and conducting tests on duplicates of production systems(e.g., development systems). Organizations need to determine the level of risk they are willing toaccept for each assessment, and tailor their approaches accordingly.
 Determine the objectives of each security assessment, and tailor the approach accordingly.
Security assessments have specific objectives, acceptable levels of risk, and available resources.
Because no individual technique provides a comprehensive picture of an organization’s security
when executed alone, organizations should use a combination of techniques. This also helps
organizations to limit risk and resource usage.
 Analyze findings, and develop risk mitigation techniques to address weaknesses. To ensurethat security assessments provide their ultimate value, organizations should conduct root cause
analysis upon completion of an assessment to enable the translation of findings into actionable
mitigation techniques. These results may indicate that organizations should address not only
technical weaknesses, but weaknesses in organizational processes and procedures as well.

ES-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

1.

Introduction

1.1

Authority

The National Institute of Standards and Technology (NIST) developed this document in furtherance of its
statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002,
Public Law 107-347.
NIST is responsible for developing standards and guidelines, including minimum requirements, forproviding adequate information security for all agency operations and assets; but such standards andguidelines shall not apply to national security systems. This guideline is consistent with the requirementsof the Office of Management and Budget (OMB) Circular A-130, Section 8b (3), “Securing Agency
Information Systems,” as analyzed in A-130, Appendix IV: Analysis of Key Sections. Supplemental
information is provided in A-130, Appendix III.
This guideline has been prepared for use by federal agencies. It may be used by nongovernmentalorganizations on a voluntary basis and is not subject to copyright, though attribution is desired.
Nothing in this document should be taken to contradict standards and guidelines made mandatory andbinding on federal agencies by the Secretary of Commerce under statutory authority; nor should theseguidelines be interpreted as altering or superseding the existing authorities of the Secretary of Commerce,
Director of the OMB, or any other federal official.
1.2

Purpose and Scope

The purpose of this document is to provide guidelines for organizations on planning and conducting
technical information security testing and assessments, analyzing findings, and developing mitigation
strategies. It provides practical recommendations for designing, implementing, and maintaining technicalinformation relating to security testing and assessment processes and procedures, which can be used forseveral purposes—such as finding vulnerabilities in a system or network and verifying compliance with apolicy or other requirements. This guide is not intended to present a comprehensive information security
testing or assessment program, but rather an overview of the key elements of technical security testing
and assessment with emphasis on specific techniques, their benefits and limitations, and recommendations
for their use.
This document replaces NIST Special Publication 800-42, Guideline on Network Security Testing.
1.3

Audience

This guide is intended for use by computer security staff and program managers, system and network
administrators, and other technical staff who are responsible for the technical aspects of preparing,
operating, and securing systems and network infrastructures. Managers can also use the information
presented to facilitate the technical decision-making processes associated with security testing andassessments. Material in this document is technically oriented, and assumes that readers have at least abasic understanding of system and network security.

1-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

1.4

Document Structure

The remainder of this document is organized into seven major sections: Section 2 presents an overview of information security assessments, including policies, roles andresponsibilities, methodologies, and techniques.
 Section 3 provides a detailed description of several technical examination techniques, including
documentation review, log review, network sniffing, and file integrity checking.
 Section 4 describes several techniques for identifying targets and analyzing them for potential
vulnerabilities. Examples of these techniques include network discovery and vulnerability
scanning.
 Section 5 explains techniques commonly used to validate the existence of vulnerabilities, such as
password cracking and penetration testing.
 Section 6 presents an approach and process for planning a security assessment.
 Section 7 discusses factors that are key to the execution of security assessments, including
coordination, the assessment itself, analysis, and data handling.
 Section 8 presents an approach for reporting assessment findings, and provides an overview ofremediation activities.
This guide also contains the following appendices: Appendix A describes two live operating system (OS) CD distributions that allow the user to boota computer to a CD containing a fully operational OS and testing tools.
 Appendix B provides a template for creating Rules of Engagement (ROE).
 Appendix C briefly discusses application security assessment.
 Appendix D contains recommendations for performing remote access testing.
 Appendix E offers a list of resources that may facilitate the security assessment process.
 Appendix F features a glossary of terms used throughout this document.
 Appendix G provides a list of acronyms and abbreviations.

1-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

2.

Security Testing and Examination Overview

An information security assessment is the process of determining how effectively an entity being assessed(e.g., host, system, network, procedure, person—known as the assessment object) meets specific security
objectives. Three types of assessment methods can be used to accomplish this—testing, examination, andinterviewing. Testing is the process of exercising one or more assessment objects under specified
conditions to compare actual and expected behaviors. Examination is the process of checking, inspecting,reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding,achieve clarification, or obtain evidence. Interviewing is the process of conducting discussions withindividuals or groups within an organization to facilitate understanding, achieve clarification, or identify
the location of evidence. Assessment results are used to support the determination of security control
effectiveness over time.
This publication addresses technical testing and examination techniques that can be used to identify,
validate, and assess technical vulnerabilities and assist organizations in understanding and improving thesecurity posture of their systems and networks. Security testing and examination is required by FISMA 1
and other regulations. It is not meant to take the place of implementing security controls and maintaining
system security, but to help organizations confirm that their systems are properly secured and identify anyorganization security requirements that are not met as well as other security weaknesses that should be
addressed.
This section provides an overview of information security assessment methodologies and technical testingand examination techniques.
2.1

Information Security Assessment Methodology

A repeatable and documented security assessment methodology is beneficial in that it can:
 Provide consistency and structure to security testing, which can minimize testing risks
 Expedite the transition of new assessment staff
 Address resource constraints associated with security assessments.
Because information security assessment requires resources such as time, staff, hardware, and software,resource availability is often a limiting factor in the type and frequency of security assessments.
Evaluating the types of security tests and examinations the organization will execute, developing anappropriate methodology, identifying the resources required, and structuring the assessment process to
support expected requirements can mitigate the resource challenge. This gives the organization the ability
to reuse pre-established resources such as trained staff and standardized testing platforms; decreases timerequired to conduct the assessment and the need to purchase testing equipment and software; and reduces
overall assessment costs.
A phased information security assessment methodology offers a number of advantages. The structure iseasy to follow, and provides natural breaking points for staff transition. Its methodology should containat minimum the following phases:

1

Section 3544 requires the “periodic testing and evaluation of the effectiveness of information security policies, procedures,and practices, to be performed with a frequency depending on risk, but no less than annually.” FISMA is available at
http://csrc.nist.gov/drivers/documents/FISMA-final.pdf.

2-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

 Planning. Critical to a successful security assessment, the planning phase is used to gather
information needed for assessment execution—such as the assets to be assessed, the threats ofinterest against the assets, and the security controls to be used to mitigate those threats—and todevelop the assessment approach. A security assessment should be treated as any other project,with a project management plan to address goals and objectives, scope, requirements, team roles
and responsibilities, limitations, success factors, assumptions, resources, timeline, anddeliverables. Section 6 of this guide covers planning.
 Execution. Primary goals for the execution phase are to identify vulnerabilities and validatethem when appropriate. This phase should address activities associated with the intended
assessment method and technique. Although specific activities for this phase differ byassessment type, upon completion of this phase assessors will have identified system, network,and organizational process vulnerabilities. This phase is discussed in more depth in Section 7.
 Post-Execution. The post-execution phase focuses on analyzing identified vulnerabilities todetermine root causes, establish mitigation recommendations, and develop a final report. Section
8 of this guide addresses reporting and mitigation.
Several accepted methodologies exist for conducting different types of information security assessments.
References to several of these methodologies are found in Appendix E. 2 For example, NIST has created amethodology—documented in Special Publication (SP) 800-53A, Guide for Assessing the Security
Controls in Federal Information Systems—which offers suggestions for assessing the effectiveness of the
security controls outlined in NIST SP 800-53. 3 Another widely used assessment methodology is theOpen Source Security Testing Methodology Manual (OSSTMM). 4 Because there are numerous reasons
to conduct assessments, an organization may want to use multiple methodologies. This publication offers
recommendations for technical testing and examination techniques that can be used for many assessment
methodologies and leveraged for many assessment purposes.
2.2

Technical Assessment Techniques

Dozens of technical security testing and examination techniques exist that can be used to assess the
security posture of systems and networks. The most commonly used techniques from the standpoint of
this document will be discussed in more depth later in this guide, and are grouped into the following three
categories:
 Review Techniques. These are examination techniques used to evaluate systems, applications,networks, policies, and procedures to discover vulnerabilities, and are generally conductedmanually. They include documentation, log, ruleset, and system configuration review; network
sniffing; and file integrity checking. Section 3 provides additional information on review
techniques.
 Target Identification and Analysis Techniques. These testing techniques can identify systems,
ports, services, and potential vulnerabilities, and may be performed manually but are generally
performed using automated tools. They include network discovery, network port and service
2

3

4

NIST does not endorse one methodology over another; the intent is to provide organizations with options that will allow
them to make informed decisions to adopt an existing methodology or combine several to develop a unique methodologythat suits the organization.
NIST SP 800-53A discusses the framework for development of assessment procedures, describes the process of assessing
security controls, and offers assessment procedures for each control. NIST SP 800-53A was developed to be used inconjunction with NIST SP 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems.
NIST SPs 800-53, 800-53A, and 800-37 are available at http://csrc.nist.gov/publications/PubsSPs.html.
More information on OSSTMM is available at http://www.isecom.org/osstmm/.

2-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

identification, vulnerability scanning, wireless scanning, and application security examination.
Further discussion of these techniques is presented in Section 4.
 Target Vulnerability Validation Techniques. These testing techniques corroborate theexistence of vulnerabilities, and may be performed manually or by using automatic tools,
depending on the specific technique used and the skill of the test team. Target vulnerability
validation techniques include password cracking, penetration testing, social engineering, andapplication security testing. More information on these techniques is found in Section 5.
Since no one technique can provide a complete picture of the security of a system or network,organizations should combine appropriate techniques to ensure robust security assessments. For example,
penetration testing usually relies on performing both network port/service identification and vulnerability
scanning to identify hosts and services that may be targets for future penetration. Also, multiple technicalways exist to meet an assessment requirement, such as determining whether patches have been applied
properly. This publication focuses on explaining how these different technical techniques can be
performed, and does not specify which techniques should be used for which circumstances—thus
providing organizations with the flexibility to choose the techniques that best meet their requirements.
In addition to the technical techniques described in this publication, there are many non-technicaltechniques that may be used in addition to or instead of the technical techniques. One example is physicalsecurity testing, which confirms the existence of physical security vulnerabilities by attempting tocircumvent locks, badge readers, and other physical security controls, typically to gain unauthorizedaccess to specific hosts. Another example of a non-technical technique is manual asset identification. An
organization may choose to identify assets to be assessed through asset inventories, physicalwalkthroughs of facilities, and other non-technical means, instead of relying on technical techniques for
asset identification. Details on non-technical techniques are outside the scope of this publication, but it isimportant to recognize the value of non-technical techniques and to consider when they may be more
appropriate to use than their technical counterparts.
2.3

Comparing Tests and Examinations

Examinations primarily involve the review of documents such as policies, procedures, security plans,
security requirements, standard operating procedures, architecture diagrams, engineering documentation,asset inventories, system configurations, rulesets, and system logs. They are conducted to determine
whether a system is properly documented, and to gain insight on aspects of security that are only available
through documentation. This documentation identifies the intended design, installation, configuration,
operation, and maintenance of the systems and network, and its review and cross-referencing ensuresconformance and consistency. For example, an environment’s security requirements should drive
documentation such as system security plans and standard operating procedures—so assessors should
ensure that all plans, procedures, architectures, and configurations are compliant with stated security
requirements and applicable policies. Another example is reviewing a firewall’s ruleset to ensure its
compliance with the organization’s security policies regarding Internet usage, such as the use of instantmessaging, peer-to-peer (P2P) file sharing, and other prohibited activities.
Examinations typically have no impact on the actual systems or networks in the target environment aside
from accessing necessary documentation, logs, or rulesets. 5 However, if system configuration files orlogs are to be retrieved from a given system such as a router or firewall, only system administrators and5

One passive testing technique that can potentially impact networks is network sniffing, which involves connecting a sniffer
to a hub, tap, or span port on the network. In some cases, the connection process requires reconfiguring a network device,
which could disrupt operations.

2-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

similarly trained individuals should undertake this work to ensure that settings are not inadvertently
modified or deleted.
Testing involves hands-on work with systems and networks to identify security vulnerabilities, and can be
executed across an entire enterprise or on selected systems. The use of scanning and penetration
techniques can provide valuable information on potential vulnerabilities and predict the likelihood that anadversary or intruder will be able to exploit them. Testing also allows organizations to measure levels ofcompliance in areas such as patch management, password policy, and configuration management.
Although testing can provide a more accurate picture of an organization’s security posture than what isgained through examinations, it is more intrusive and can impact systems or networks in the target
environment. The level of potential impact depends on the specific types of testing techniques used,which can interact with the target systems and networks in various ways—such as sending normal
network packets to determine open and closed ports, or sending specially crafted packets to test forvulnerabilities. Any time that a test or tester directly interacts with a system or network, the potential
exists for unexpected system halts and other denial of service conditions. Organizations should determine
their acceptable levels of intrusiveness when deciding which techniques to use. Excluding tests known tocreate denial of service conditions and other disruptions can help reduce these negative impacts.
Testing does not provide a comprehensive evaluation of the security posture of an organization, and often
has a narrow scope because of resource limitations—particularly in the area of time. Malicious attackers,
on the other hand, can take whatever time they need to exploit and penetrate a system or network. Also,
while organizations tend to avoid using testing techniques that impact systems or networks, attackers are
not bound by this constraint and use whatever techniques they feel necessary. As a result, testing is lesslikely than examinations to identify weaknesses related to security policy and configuration. In manycases, combining testing and examination techniques can provide a more accurate view of security.
2.4

Testing Viewpoints

Tests can be performed from a number of viewpoints—for example, how easily could an external attacker
or malicious insider successfully attack a system? Section 2.4.1 of this guide compares testing performed
from external and internal viewpoints. Section 2.4.2 discusses another aspect of viewpoints—namely, theprevious knowledge that assessors have of the target or target environment.
2.4.1

External and Internal

External security testing is conducted from outside the organization’s security perimeter. This offers theability to view the environment’s security posture as it appears outside the security perimeter—usually as
seen from the Internet—with the goal of revealing vulnerabilities that could be exploited by an external
attacker.
External testing often begins with reconnaissance techniques that search public registration data, Domain
Name System (DNS) server information, newsgroup postings, and other publicly available information tocollect information (e.g., system names, Internet Protocol [IP] addresses, operating systems, technicalpoints of contact) that may help the assessor to identify vulnerabilities. Next, enumeration begins byusing network discovery and scanning techniques to determine external hosts and listening services.
Since perimeter defenses such as firewalls, routers, and access control lists often limit the types of traffic
allowed into the internal network, assessors often use techniques that evade these defenses—just as
external attackers would. Depending on the protocols allowed through, initial attacks are generally
focused on commonly used and allowed application protocols such as File Transfer Protocol (FTP),
Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), and Post Office Protocol

2-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT(POP). Servers that are externally accessible are tested for vulnerabilities that might allow access tointernal servers and private information. External security testing also concentrates on discovering accessmethod vulnerabilities, such as wireless access points, modems, and portals to internal servers.
For internal security testing, assessors work from the internal network and assume the identity of a trusted
insider or an attacker who has penetrated the perimeter defenses. This kind of testing can reveal
vulnerabilities that could be exploited, and demonstrates the potential damage this type of attacker could
cause. Internal security testing also focuses on system-level security and configuration—including
application and service configuration, authentication, access control, and system hardening.
Assessors who perform internal testing are often granted some level of access to the network, normally as
general users, and are provided with information that users with similar privileges would have. This levelof temporary access depends on the goals of the test, and can be up to and including the privileges of asystem or network administrator. Working from whatever level of access they have been granted,
assessors attempt to gain additional access to the network and systems through privilege escalation—i.e.,
increasing user-level privileges to administrator-level privileges, or increasing system administrator
privileges to domain administrator privileges.
Internal testing is not as limited as external testing because it takes place behind perimeter defenses, eventhough there may be internal firewalls, routers, and switches in place that pose limitations. Examination
techniques such as network sniffing may be used in addition to testing techniques.
If both internal and external testing is to be performed, the external testing usually takes place first. Thisis particularly beneficial if the same assessors will be performing both types of testing, as it keeps themfrom acquiring insider information on network architecture or system configuration that would not be
available to an adversary—an advantage that would reduce the validity of the test.
2.4.2

Overt and Covert

Overt security testing, also known as white hat testing, involves performing external and/or internal
testing with the knowledge and consent of the organization’s IT staff, enabling comprehensive evaluationof the network or system security posture. Because the IT staff is fully aware of and involved in thetesting, it may be able to provide guidance to limit the testing’s impact. Testing may also provide a
training opportunity, with staff observing the activities and methods used by assessors to evaluate andpotentially circumvent implemented security measures. This gives context to the security requirements
implemented or maintained by the IT staff, and also may help teach IT staff how to conduct testing.
Covert security testing, also known as black hat testing, takes an adversarial approach by performing
testing without the knowledge of the organization’s IT staff but with the full knowledge and permissionof upper management. Some organizations designate a trusted third party to ensure that the target
organization does not initiate response measures associated with the attack without first verifying that an
attack is indeed underway (e.g., that the activity being detected does not originate from a test). In suchsituations, the trusted third party provides an agent for the assessors, the management, the IT staff, and the
security staff that mediates activities and facilitates communications. This type of test is useful for testing
technical security controls, IT staff response to perceived security incidents, and staff knowledge andimplementation of the organization’s security policy. Covert testing may be conducted with or without
warning.
The purpose of covert testing is to examine the damage or impact an adversary can cause—it does not
focus on identifying vulnerabilities. This type of testing does not test every security control, identify eachvulnerability, or assess all systems within an organization. Covert testing examines the organization from

2-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

an adversarial perspective, and normally identifies and exploits the most rudimentary vulnerabilities togain network access. If an organization’s goal is to mirror a specific adversary, this type of testing
requires special considerations—such as acquiring and modeling threat data. The resulting scenarios
provide an overall strategic view of the potential methods of exploit, risk, and impact of an intrusion.
Covert testing usually has defined boundaries, such as stopping testing when a certain level of access isachieved or a certain type of damage is achievable as a next step in testing. Having such boundaries
prevents damage while still showing that the damage could occur.
Besides failing to identify many vulnerabilities, covert testing is often time-consuming and costly due toits stealth requirements. To operate in a stealth environment, a test team will have to slow its scans andother actions to stay “under the radar” of the target organization’s security staff. When testing is
performed in-house, training must also be considered in terms of time and budget. In addition, an
organization may have staff trained to perform regular activities such as scanning and vulnerability
assessments, but not specialized techniques such as penetration or application security testing. Overt
testing is less expensive, carries less risk than covert testing, and is more frequently used—but covert
testing provides a better indication of the everyday security of the target organization because systemadministrators will not have heightened awareness.

2-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

3.

Review Techniques

Review techniques passively examine systems, applications, networks, policies, and procedures todiscover security vulnerabilities. 6 They also gather information to facilitate and optimize otherassessment techniques. Because review techniques are passive, they pose minimal risk to systems andnetworks. This section covers several common review techniques—documentation, log, ruleset, andsystem configuration review; network sniffing; and file integrity checking.
3.1

Documentation Review

Documentation review determines if the technical aspects of policies and procedures are current andcomprehensive. These documents provide the foundation for an organization’s security posture, but areoften overlooked during technical assessments. Security groups within the organization should provide
assessors with appropriate documentation to ensure a comprehensive review. Documents to review fortechnical accuracy and completeness include security policies, architectures, and requirements; standard
operating procedures; system security plans and authorization agreements; memoranda of understanding
and agreement for system interconnections; and incident response plans.
Documentation review can discover gaps and weaknesses that could lead to missing or improperly
implemented security controls. Assessors typically verify that the organization’s documentation iscompliant with standards and regulations such as FISMA, and look for policies that are deficient or
outdated. Common documentation weaknesses include OS security procedures or protocols that are nolonger used, and failure to include a new OS and its protocols. Documentation review does not ensurethat security controls are implemented properly—only that the direction and guidance exist to support
security infrastructure.
Results of documentation review can be used to fine-tune other testing and examination techniques. For
example, if a password management policy has specific requirements for minimum password length andcomplexity, this information can be used to configure password-cracking tools for more efficientperformance.
3.2

Log Review

Log review determines if security controls are logging the proper information, and if the organization isadhering to its log management policies. 7 As a source of historical information, audit logs can be used tohelp validate that the system is operating in accordance with established policies. For example, if thelogging policy states that all authentication attempts to critical servers must be logged, the log review will
determine if this information is being collected and shows the appropriate level of detail. Log review may
also reveal problems such as misconfigured services and security controls, unauthorized accesses, andattempted intrusions. For example, if an intrusion detection system (IDS) sensor is placed behind afirewall, its logs can be used to examine communications that the firewall allows into the network. If thesensor registers activities that should be blocked, it indicates that the firewall is not configured securely.

6

7This publication discusses reviews strictly from the aspect of assessment. Reviews should also be conducted periodically as
part of regular system monitoring and maintenance, such as to identify operational problems, security misconfigurations,malicious activity, and other types of security events. Organizations can choose to use findings from operational reviews fortheir assessments.
NIST SP 800-92, Guide to Security Log Management, provides more information on security log management methods andtechniques, including log review. It is available at http://csrc.nist.gov/publications/PubsSPs.html.

3-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Examples of log information that may be useful when conducting technical security assessments include: Authentication server or system logs may include successful and failed authentication attempts.
 System logs may include system and service startup and shutdown information, installation ofunauthorized software, file accesses, security policy changes, account changes (e.g., accountcreation and deletion, account privilege assignment), and privilege use.
 Intrusion detection and prevention system logs may include malicious activity and inappropriateuse.
 Firewall and router logs may include outbound connections that indicate compromised internal
devices (e.g., rootkits, bots, Trojan horses, spyware).
 Firewall logs may include unauthorized connection attempts and inappropriate use.
 Application logs may include unauthorized connection attempts, account changes, use ofprivileges, and application or database usage information.
 Antivirus logs may include update failures and other indications of outdated signatures andsoftware.
 Security logs, in particular patch management and some IDS and intrusion prevention system(IPS) products, may record information on known vulnerable services and applications.
Manually reviewing logs can be extremely time-consuming and cumbersome. Automated audit tools areavailable that can significantly reduce review time and generate predefined and customized reports thatsummarize log contents and track them to a set of specific activities. Assessors can also use these
automated tools to facilitate log analysis by converting logs in different formats to a single, standard
format for analysis. In addition, if assessors are reviewing a specific action—such as the number of failed
logon attempts in an organization—they can use these tools to filter logs based on the activity being
checked.
3.3

Ruleset Review

A ruleset is a collection of rules or signatures that network traffic or system activity is compared against
to determine what action to take—for example, forwarding or rejecting a packet, creating an alert, or
allowing a system event. Review of these rulesets is done to ensure comprehensiveness and identify gaps
and weaknesses on security devices and throughout layered defenses such as network vulnerabilities,
policy violations, and unintended or vulnerable communication paths. A review can also uncoverinefficiencies that negatively impact a ruleset’s performance.
Rulesets to review include network- and host-based firewall and IDS/IPS rulesets, and router access
control lists. The following list provides examples of the types of checks most commonly performed inruleset reviews:
 For router access control lists–

Each rule is still required (for example, rules that were added for temporary purposes are
removed as soon as they are no longer needed)–

Only traffic that is authorized per policy is permitted, and all other traffic is denied by default

 For firewall rulesets

3-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT–

Each rule is still required–

Rules enforce least privilege access, such as specifying only required IP addresses and ports–

More specific rules are triggered before general rules–

There are no unnecessary open ports that could be closed to tighten the perimeter security–

The ruleset does not allow traffic to bypass other security defenses–

For host-based firewall rulesets, the rules do not indicate the presence of backdoors, spyware
activity, or prohibited applications such as peer-to-peer file sharing programs

 For IDS/IPS rulesets

3.4–

Unnecessary signatures have been disabled or removed to eliminate false positives andimprove performance–

Necessary signatures are enabled and have been fine-tuned and properly maintained.

System Configuration Review

System configuration review is the process of identifying weaknesses in security configuration controls,
such as systems not being hardened or configured according to security policies. For example, this typeof review will reveal unnecessary services and applications, improper user account and password settings,
and improper logging and backup settings. Examples of security configuration files that may be reviewed
are Windows security policy settings and Unix security configuration files such as those in /etc.
Assessors using manual review techniques rely on security configuration guides or checklists to verify
that system settings are configured to minimize security risks. 8 To perform a manual system
configuration review, assessors access various security settings on the device being evaluated andcompare them with recommended settings from the checklist. Settings that do not meet minimum
security standards are flagged and reported.
The Security Content Automation Protocol (SCAP) is a method for using specific standards to enable
automated vulnerability management, measurement, and policy compliance evaluation. 9 NIST SCAP
files are written for FISMA compliance and NIST SP 800-53A security control testing. Other tools canbe used to retrieve and report security settings and provide remediation guidance. Automated tools areoften executed directly on the device being assessed, but can also be executed on a system with network
access to the device being assessed. While automated system configuration reviews are faster thanmanual methods, there may still be settings that must be checked manually. Both manual and automatedmethods require root or administrator privileges to view selected security settings.
Generally it is preferable to use automated checks instead of manual checks whenever feasible.
Automated checks can be done very quickly and provide consistent, repeatable results. Having a person
manually checking hundreds or thousands of settings is tedious and error-prone.

89

NIST maintains a repository of security configuration checklists for IT products at http://checklists.nist.gov/.
More information on SCAP is located at http://scap.nist.gov/.

3-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

3.5

Network Sniffing

Network sniffing is a passive technique 10 that monitors network communication, decodes protocols, andexamines headers and payloads to flag information of interest. Besides being used as a review technique,network sniffing can also be used as a target identification and analysis technique (see Section 4.1).
Reasons for using network sniffing include the following: Capturing and replaying network traffic Performing passive network discovery (e.g., identifying active devices on the network) Identifying operating systems, applications, services, and protocols, including unsecured (e.g.,telnet) and unauthorized (e.g., peer-to-peer file sharing) protocols
 Identifying unauthorized and inappropriate activities, such as the unencrypted transmission ofsensitive information
 Collecting information, such as unencrypted usernames and passwords.
Network sniffing has little impact on systems and networks, with the most noticeable impact being on
bandwidth or computing power utilization. The sniffer—the tool used to conduct network sniffing—requires a means to connect to the network, such as a hub, tap, or switch with port spanning. Port
spanning is the process of copying the traffic transmitted on all other ports to the port where the sniffer isinstalled. Organizations can deploy network sniffers in a number of locations within an environment.
These commonly include the following: At the perimeter, to assess traffic entering and exiting the network
 Behind firewalls, to assess that rulesets are accurately filtering traffic
 Behind IDSs/IPSs, to determine if signatures are triggering and being responded to appropriately
 In front of a critical system or application to assess activity
 On a specific network segment, to validate encrypted protocols.
One limitation to network sniffing is the use of encryption. Many attackers take advantage of encryption
to hide their activities—while assessors can see that communication is taking place, they are unable toview the contents. Another limitation is that a network sniffer is only able to sniff the traffic of the local
segment where it is installed. This requires the assessor to move it from segment to segment, install
multiple sniffers throughout the network, and/or use port spanning. Assessors may also find it
challenging to locate an open physical network port for scanning on each segment. In addition, network
sniffing is a fairly labor-intensive activity that requires a high degree of human involvement to interpret
network traffic.
3.6

File Integrity Checking

File integrity checkers provide a way to identify that system files have been changed computing andstoring a checksum for every guarded file, and establishing a file checksum database. Stored checksums
are later recomputed to compare their current value with the stored value, which identifies file10

Sniffers may perform domain name lookups for the traffic they collect, during which they generate network traffic. Domainname lookups can be disabled for stealthy network sniffing.

3-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

modifications. A file integrity checker capability is usually included with any commercial host-based
IDS, and is also available as a standalone utility.
Although an integrity checker does not require a high degree of human interaction, it must be usedcarefully to ensure its effectiveness. File integrity checking is most effective when system files arecompared with a reference database created using a system known to be secure—this helps ensure that thereference database was not built with compromised files. The reference database should be stored offline
to prevent attackers from compromising the system and covering their tracks by modifying the database.
In addition, because patches and other updates change files, the checksum database should be kept up-todate.
For file integrity checking, strong cryptographic checksums such as Secure Hash Algorithm 1 (SHA-1)
should be used to ensure the integrity of data stored in the checksum database. Federal agencies are
required by Federal Information Processing Standard (FIPS) PUB 140-2, Security Requirements for
Cryptographic Modules 11 , to use SHA (e.g., SHA-1, SHA-256).
3.7

Summary

Table 3-1 summarizes the major capabilities of review techniques discussed in Section 3.
Table 3-1. Review Techniques
Technique•

Capabilities
Evaluates policies and procedures for technical accuracy and completeness

Ruleset Review•••

Provides historical information on system use, configuration, and modification
Could reveal potential problems and policy deviations
Reveals holes in ruleset-based security controls

System Configuration
Review••

Evaluates the strength of system configuration
Validates that systems are configured in accordance with hardening policy•

Monitors network traffic on the local segment to capture information such as active
systems, operating systems, communication protocols, services, and applications
Verifies encryption of communications
Identifies changes to important files; can also identify certain forms of unwanted
files, such as well-known attacker tools

Documentation Review
Log Review

Network Sniffing
File Integrity Checking••

Risks are associated with each technique and their combinations. To ensure that all are executed safely
and accurately, each assessor should have a certain baseline skill set. Table 3-2 provides guidelines forthe minimum skill set needed for each technique presented in Section 3.
Table 3-2. Baseline Skill Set for Review Techniques
Technique11

Baseline Skill Set

Documentation Review

General knowledge of security from a policy perspective

Log Review

Knowledge of log formats and ability to interpret and analyze log data; ability to use
automated log analysis and log correlation tools

Ruleset Review

Knowledge of ruleset formats and structures; ability to correlate and analyze rulesets
from a variety of devices

System Configuration
Review

Knowledge of secure system configuration, including OS hardening and security policy
configuration for a variety of operating systems; ability to use automated security
configuration testing tools

FIPS PUB 140-2 is available at http://csrc.nist.gov/publications/PubsFIPS.html.

3-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Technique

Baseline Skill Set

Network Sniffing

General Transmission Control Protocol/Internet Protocol (TCP/IP) and networking
knowledge; ability to interpret and analyze network traffic; ability to deploy and use
network sniffing tools

File Integrity Checking

General file system knowledge; ability to use automated file integrity checking tools
and interpret the results

3-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

4.

Target Identification and Analysis TechniquesThis section addresses technical target identification and analysis techniques, which focus on identifying
active devices and their associated ports and services, and analyzing them for potential vulnerabilities.
The assessor uses this information to continue to explore devices that will validate existence of thevulnerabilities. Organizations often use non-technical techniques in addition or instead of technicaltechniques to identify the assets to be analyzed. For example, organizations may have existing asset
inventories or other lists of assets to be targeted; another example is assessors performing a walkthroughof a facility to identify assets that were not found by technical techniques, such as hosts that were shut off
or disconnected from the network when the technical techniques were used.
Target identification and analysis techniques for application security examination are briefly discussed inAppendix C.
4.1

Network Discovery

Network discovery uses a number of methods to discover active and responding hosts on a network,
identify weaknesses, and learn how the network operates. Both passive (examination) and active (testing)techniques exist for discovering devices on a network. Passive techniques use a network sniffer tomonitor network traffic and record the IP addresses of the active hosts, and can report which ports are in
use and which operating systems have been discovered on the network. Passive discovery can also
identify the relationships between hosts—including which hosts communicate with each other, howfrequently their communication occurs, and the type of traffic that is taking place—and is usually
performed from a host on the internal network where it can monitor host communications. This is done
without sending out a single probing packet. Passive discovery takes more time to gather information
than does active discovery, and hosts that do not send or receive traffic during the monitoring period
might not be reported.
Active techniques send various types of network packets, such as Internet Control Message Protocol(ICMP) pings, to solicit responses from network hosts, generally through the use of an automated tool.
One activity, known as OS fingerprinting, enables the assessor to determine the system’s OS by sending it
a mix of normal, abnormal, and illegal network traffic. Another activity involves sending packets tocommon port numbers to generate responses that indicate the ports are active. The tool analyzes the
responses from these activities, and compares them with known traits of packets from specific operating
systems and network services—enabling it to identify hosts, the operating systems they run, their ports,
and the state of those ports. This information can be used for purposes that include gathering information
on targets for penetration testing, generating topology maps, determining firewall and IDS configurations,and discovering vulnerabilities in systems and network configurations.
Network discovery tools have many ways to acquire information through scanning. Enterprise firewalls
and intrusion detection systems can identify many instances of scans, particularly those that use the most
suspicious packets (e.g., SYN/FIN scan, NULL scan). Assessors who plan on performing discovery
through firewalls and intrusion detection systems should consider which types of scans are most likely toprovide results without drawing the attention of security administrators, and how scans can be conductedin a more stealthy manner (such as more slowly or from a variety of source IP addresses) to improve theirchances of success. Assessors should also be cautious when selecting types of scans to use against older
systems, particularly those known to have weak security, because some scans can cause system failures.
Typically, the closer the scan is to normal activity, the less likely it is to cause operational problems.
Network discovery may also detect unauthorized or rogue devices operating on a network. For example,
an organization that uses only a few operating systems could quickly identify rogue devices that utilize

4-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

different ones. Once a wired rogue device is identified, 12 it can be located by using existing network
maps and information already collected on the device’s network activity to identify the switch to which it
is connected. It may be necessary to generate additional network activity with the rogue device—such aspings—to find the correct switch. The next step is to identify the switch port on the switch associatedwith the rogue device, and to physically trace the cable connecting that switch port to the rogue device.
A number of tools exist for use in network discovery, and it should be noted that many active discovery
tools can be used for passive network sniffing and port scanning as well. Most offer a graphical userinterface (GUI), and some also offer a command-line interface. Command-line interfaces may takelonger to learn than GUIs because of the number of commands and switches that specify what tests thetool should perform and which an assessor must learn to use the tool effectively. Also, developers havewritten a number of modules for open source tools that allow assessors to easily parse tool output. For
example, combining a tool’s Extensible Markup Language (XML) output capabilities, a little scripting,and a database creates a more powerful tool that can monitor the network for unauthorized services andmachines. Learning what the many commands do and how to combine them is best achieved with thehelp of an experienced security engineer. Most experienced IT professionals, including systemadministrators and other network engineers, should be able to interpret results, but working with thediscovery tools themselves is more efficiently handled by an engineer.
Some of the advantages of active discovery, as compared to passive discovery, are that an assessment canbe conducted from a different network and usually requires little time to gather information. In passive
discovery, ensuring that all hosts are captured requires traffic to hit all points, which can be timeconsuming—especially in larger enterprise networks.
A disadvantage to active discovery is that it tends to generate network noise, which sometimes results in
network latency. Since active discovery sends out queries to receive responses, this additional network
activity could slow down traffic or cause packets to be dropped in poorly configured networks if
performed at high volume. Active discovery can also trigger IDS alerts, since unlike passive discovery it
reveals its origination point. The ability to successfully discover all network systems can be affected byenvironments with protected network segments and perimeter security devices and techniques. For
example, an environment using network address translation (NAT)—which allows organizations to haveinternal, non-publicly routed IP addresses that are translated to a different set of public IP addresses forexternal traffic—may not be accurately discovered from points external to the network or from protected
segments. Personal and host-based firewalls on target devices may also block discovery traffic.
Misinformation may be received as a result of trying to instigate activity from devices. Active discovery
presents information from which conclusions must be drawn about settings on the target network.
For both passive and active discovery, the information received is seldom completely accurate. To
illustrate, only hosts that are on and connected during active discovery will be identified—if systems or asegment of the network are offline during the assessment, there is potential for a large gap in discovering
devices. Although passive discovery will only find devices that transmit or receive communications
during the discovery period, products such as network management software can provide continuous
discovery capabilities and automatically generate alerts when a new device is present on the network.
Continuous discovery can scan IP address ranges for new addresses or monitor new IP address requests.
Also, many discovery tools can be scheduled to run regularly, such as once every set amount of days at aparticular time. This provides more accurate results than running these tools sporadically.

12

See Section 4.4 for information on locating wireless rogue devices.

4-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

4.2

Network Port and Service Identification

Network port and service identification involves using a port scanner to identify network ports andservices operating on active hosts—such as FTP and HTTP—and the application that is running eachidentified service, such as Microsoft Internet Information Server (IIS) or Apache for the HTTP service.
Organizations should conduct network port and service identification to identify hosts if this has notalready been done by other means (e.g., network discovery), and flag potentially vulnerable services.
This information can be used to determine targets for penetration testing.
All basic scanners can identify active hosts and open ports, but some scanners are also able to provide
additional information on the scanned hosts. Information gathered during an open port scan can assist in
identifying the target operating system through a process called OS fingerprinting. For example, if a hosthas TCP ports 135, 139, and 445 open, it is probably a Windows host, or possibly a Unix host running
Samba. Other items—such as the TCP packet sequence number generation and responses to packets—
also provide a clue to identifying the OS. But OS fingerprinting is not foolproof. For example, firewalls
block certain ports and types of traffic, and system administrators can configure their systems to respondin nonstandard ways to camouflage the true OS.
Some scanners can help identify the application running on a particular port through a process called
service identification. Many scanners use a services file that lists common port numbers and typicalassociated services—for example, a scanner that identifies that TCP port 80 is open on a host may reportthat a web server is listening at that port—but additional steps are needed before this can be confirmed.
Some scanners can initiate communications with an observed port and analyze its communications todetermine what service is there, often by comparing the observed activity to a repository of informationon common services and service implementations. These techniques may also be used to identify theservice application and application version, such as which Web server software is in use—this process isknown as version scanning. A well-known form of version scanning, called banner grabbing, involves
capturing banner information transmitted by the remote port when a connection is initiated. This
information can include the application type, application version, and even OS type and version. Version
scanning is not foolproof, because a security-conscious administrator can alter the transmitted banners orother characteristics in hopes of concealing the service’s true nature. However, version scanning is far
more accurate than simply relying on a scanner’s services file.
Scanner models support the various scanning methods with strengths and weaknesses that are normallyexplained in their documentation. For example, some scanners work best scanning through firewalls,
while others are better suited for scans inside the firewall. Results will differ depending on the portscanner used. Some scanners respond with a simple open or closed response for each port, while others
offer additional detail (e.g., filtered or unfiltered) that can assist the assessor in determining what othertypes of scans would be helpful to gain additional information.
Network port and service identification often uses the IP address results of network discovery as thedevices to scan. Port scans can also be run independently on entire blocks of IP addresses—here, portscanning performs network discovery by default through identifying the active hosts on the network. The
result of network discovery and network port and service identification is a list of all active devices
operating in the address space that responded to the port scanning tool, along with responding ports.
Additional active devices could exist that did not respond to scanning, such as those that are shielded by
firewalls or turned off. Assessors can try to find these devices by scanning the devices themselves,

4-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

placing the scanner on a segment that can access the devices, or attempting to evade the firewall throughthe use of alternate scan types (e.g., SYN/FIN or Xmas scan). 13
It is recommended that if both external and internal scanning are to be used and the assessors areintentionally performing the testing “blind,” that external scanning be performed first. Done in this order,logs can be reviewed and compared before and during internal testing. When performing externalscanning, assessors may use any existing stealth techniques to get packets through firewalls while evading
detection by IDS and IPS. 14 Tools that use fragmentation, duplication, overlap, out-of-order, and timing
techniques to alter packets so that they blend into and appear more like normal traffic are recommended.
Internal testing tends to use less aggressive scanning methods because these scans are blocked less often
than external scans. Using more aggressive scans internally significantly increases the changes of
disrupting operations without necessarily improving scan results. Being able to scan a network withcustomized packets also works well for internal testing, because checking for specific vulnerabilities
requires highly customized packets. Tools with packet-builder ability are helpful with this process. Once
built, packets can be sent through a second scanning program that will collect the results. Because
customized packets can trigger a denial of service (DoS) attack, this type of test should be conductedduring periods of low network traffic—such as overnight or on the weekend.
Although port scanners identify active hosts, operating systems, ports, services, and applications, they do
not identify vulnerabilities. Additional investigation is needed to confirm the presence of insecure
protocols (e.g., Trivial File Transfer Protocol [TFTP], telnet), malware, unauthorized applications, andvulnerable services. To identify vulnerable services, the assessor compares identified version numbers ofservices with a list of known vulnerable versions, or perform automated vulnerability scanning as
discussed in Section 4.3. With port scanners, the scanning process is highly automated but interpretationof the scanned data is not.
Although port scanning can disrupt network operations by consuming bandwidth and slowing network
response times, it enables an organization to ensure that its hosts are configured to run only approved
network services. Scanning software should be carefully selected to minimize disruptions to operations.
Port scanning can also be conducted after hours to cause minimal impact to operations.
4.3

Vulnerability Scanning

Like network port and service identification, vulnerability scanning identifies hosts and host attributes(e.g., operating systems, applications, open ports), but it also attempts to identify vulnerabilities rather
than relying on human interpretation of the scanning results. Many vulnerability scanners are equipped toaccept results from network discovery and network port and service identification, which reduces theamount of work needed for vulnerability scanning. Also, some scanners can perform their own network
discovery and network port and service identification. Vulnerability scanning can help identify outdated
software versions, missing patches, and misconfigurations, and validate compliance with or deviations
from an organization’s security policy. This is done by identifying the operating systems and major
software applications running on the hosts and matching them with information on known vulnerabilities
stored in the scanners’ vulnerability databases.
Vulnerability scanners can: Check compliance with host application usage and security policies
13

14

Many firewalls can recognize and block various alternate scan types, so testers may not be able to use them to evadefirewalls in many environments.
This can be particularly helpful in improving the tuning and configuration of IDSs and IPSs.

4-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

 Provide information on targets for penetration testing Provide information on how to mitigate discovered vulnerabilities.
Vulnerability scanners can be run against a host either locally or from the network. Some network-based
scanners have administrator-level credentials on individual hosts and can extract vulnerability informationfrom hosts using those credentials. Other network-based scanners do not have such credentials and must
rely on conducting scanning of networks to locate hosts and then scan those hosts for vulnerabilities. In
such cases, network-based scanning is primarily used to perform network discovery and identify open
ports and related vulnerabilities—in most cases, it is not limited by the OS of the targeted systems.
Network-based scanning without host credentials can be performed both internally and externally—and
although internal scanning usually uncovers more vulnerabilities than external scanning, testing from both
viewpoints is important. External scanning must contend with perimeter security devices that block
traffic, limiting assessors to scanning only the ports authorized to pass traffic.
Assessors performing external scanning may find challenges similar to those faced with network
discovery, such as the use of NAT or personal and host-based firewalls. To overcome the challenges of
NAT and conduct successful network-based scanning, assessors can ask the firewall administrator toenable port forwarding on specific IP addresses or groups of addresses if this is supported by the firewall,or request network access behind the device performing NAT. Assessors can also request that personal or
host-based firewalls be configured to permit traffic from test system IP addresses during the assessment
period. These steps will give assessors increased insight into the network, but do not accurately reflect
the capabilities of an external attacker—although they may offer a better indication of the capabilities
available to a malicious insider or an external attacker with access to another host on the internal network.
Assessors can also perform scanning on individual hosts.
For local vulnerability scanning, a scanner is installed on each host to be scanned. This is done primarily
to identify host OS and application misconfigurations and vulnerabilities—both network-exploitable andlocally exploitable. Local scanning is able to detect vulnerabilities with a higher level of detail than
network-based scanning because local scanning usually requires both host (local) access and a root oradministrative account. Some scanners also offer the capability of repairing local misconfigurations.
A vulnerability scanner is a relatively fast and easy way to quantify an organization's exposure to surface
vulnerabilities. A surface vulnerability is a weakness that exists in isolation, independent from othervulnerabilities. The system’s behaviors and outputs in response to attack patterns submitted by thescanner are compared against those that characterize the signatures of known vulnerabilities, and the tool
reports any matches that are found. Besides signature-based scanning, some vulnerability scanners
attempt to simulate the reconnaissance attack patterns used to probe for exposed, exploitable
vulnerabilities, and report the vulnerabilities found when these techniques are successful.
One difficulty in identifying the risk level of vulnerabilities is that they rarely exist in isolation. For
example, there could be several low-risk vulnerabilities that present a higher risk when combined.
Scanners are unable to detect vulnerabilities that are revealed only as the result of potentially unending
combinations of attack patterns. The tool may assign a low risk to each vulnerability, leaving the assessor
falsely confident in the security measures in place. A more reliable way of identifying the risk ofvulnerabilities in aggregate is through penetration testing, which is discussed in Section 5.2.
Another problem with identifying the risk level of vulnerabilities is that vulnerability scanners often use
their own proprietary methods for defining the levels. For example, one scanner might use the levels low,
medium, and high, while another scanner might use the levels informational, low, medium, high, andcritical. This makes it difficult to compare findings among multiple scanners. Also, the risk levels
assigned by a scanner may not reflect the actual risk to the organization—for example, a scanner might
4-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

label an FTP server as a moderate risk because it transmits passwords in cleartext, but if the organization
only uses the FTP server as an anonymous public server that does not use passwords, then the actual risk
might be considerably lower. Assessors should determine the appropriate risk level for each vulnerability
and not simply accept the risk levels assigned by vulnerability scanners.
Network-based vulnerability scanning has some significant weaknesses. As with network sniffing anddiscovery, this type of scanning uncovers vulnerabilities only for active systems. This generally covers
surface vulnerabilities, and is unable to address the overall risk level of a scanned network. Although theprocess itself is highly automated, vulnerability scanners can have a high false positive error rate (i.e.,reporting vulnerabilities when none exist). An individual with expertise in networking and OS security
should interpret the results. And because network-based vulnerability scanning requires more
information than port scanning to reliably identify the vulnerabilities on a host, it tends to generatesignificantly more network traffic than port scanning. This may have a negative impact on the hosts or
network being scanned, or on network segments through which scanning traffic is traversing. Many
vulnerability scanners also include network-based tests for DoS attacks that, in the hands of an
inexperienced assessor, can have a marked negative impact on scanned hosts. Scanners often allow all
DoS attack tests to be suppressed so as to reduce the risk of impacting hosts through testing.
Another significant limitation of vulnerability scanners is that, like virus scanners and IDSs, they rely on
a repository of signatures. This requires the assessors to update these signatures frequently to enable thescanner to recognize the latest vulnerabilities. Before running any scanner, an assessor should install thelatest updates to its vulnerability database. Some vulnerability scanner databases are updated moreregularly than others—this update frequency should be a major consideration when selecting avulnerability scanner.
Most vulnerability scanners allow the assessor to perform different levels of scanning that vary in termsof thoroughness. While more comprehensive scanning may detect a greater number of vulnerabilities, it
can slow the overall scanning process. Less comprehensive scanning can take less time, but identifies
only well-known vulnerabilities. It is generally recommended that assessors conduct a thoroughvulnerability scan if resources permit.
Vulnerability scanning is a somewhat labor-intensive activity that requires a high degree of human
involvement to interpret results. It may also disrupt network operations by taking up bandwidth andslowing response times. Nevertheless, vulnerability scanning is extremely important in ensuring thatvulnerabilities are mitigated before they are discovered and exploited by adversaries.
As with all pattern-matching and signature-based tools, application vulnerability scanners typically havehigh false positive rates. Assessors should configure and calibrate their scanners to minimize both false
positives and false negatives to the greatest possible extent, and meaningfully interpret results to identify
the real vulnerabilities. Scanners also suffer from the high false negative rates that characterize othersignature-based tools—but vulnerabilities that go undetected by automated scanners can potentially becaught using multiple vulnerability scanners or additional forms of testing. A common practice is to use
multiple scanners—this provides assessors with a way to compare results.
4.4

Wireless Scanning

Wireless technologies, in their simplest sense, enable one or more devices to communicate without theneed for physical connections such as network or peripheral cables. They range from simple technologies
like wireless keyboards and mice to complex cell phone networks and enterprise wireless local area
networks (WLAN). As the number and availability of wireless-enabled devices continues to increase, it

4-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

is important for organizations to actively test and secure their enterprise wireless environments. 15
Wireless scans can help organizations determine corrective actions to mitigate risks posed by wirelessenabled technologies.
The following factors in the organization’s environment should be taken into consideration whenplanning technical wireless security assessments:
 The location of the facility being scanned, because the physical proximity of a building to a
public area (e.g., streets and public common areas) or its location in a busy metropolitan area may
increase the risk of wireless threats The security level of the data to be transmitted using wireless technologies
 How often wireless devices connect to and disconnect from the environment, and the typicaltraffic levels for wireless devices (e.g., occasional activity or fairly constant activity)—this isbecause only active wireless devices are discoverable during a wireless scan Existing deployments of wireless intrusion detection and prevention systems (WIDPS 16 ), which
may already collect most of the information that would be gathered by testing.
Wireless scanning should be conducted using a mobile device with wireless analyzer software installed
and configured—such as a laptop, handheld device, or specialty device. The scanning software or tool
should allow the operator to configure the device for specific scans, and to scan in both passive and activemodes. The scanning software should also be configurable by the operator to identify deviations from theorganization’s wireless security configuration requirements.
The wireless scanning tool should be capable of scanning all Institute of Electrical and Electronics
Engineers (IEEE) 802.11a/b/g/n channels, whether domestic or international. In some cases, the device
should also be fitted with an external antenna to provide an additional level of radio frequency (RF)capturing capability. Support for other wireless technologies, such as Bluetooth, will help evaluate thepresence of additional wireless threats and vulnerabilities. Note that devices using nonstandard
technology or frequencies outside of the scanning tool’s RF range will not be detected or properly
recognized by the scanning tool. A tool such as an RF spectrum analyzer will assist organizations in
identifying transmissions that occur within the frequency range of the spectrum analyzer. Spectrum
analyzers generally analyze a large frequency range (e.g., 3 to 18 GHz) —and although these devices do
not analyze traffic, they enable an assessor to determine wireless activity within a specific frequency
range and tailor additional testing and examination accordingly.
Some devices also support mapping and physical location plotting through use of a mapping tool, and insome cases support Global Positioning System (GPS)-based mapping. Sophisticated wireless scanning
tools allow the user to import a floor plan or map to assist in plotting the physical location of discovered
devices. (It is important to note that GPS has limited capabilities indoors.)
Individuals with a strong understanding of wireless networking—especially IEEE 802.11a/b/g/n
technologies—should operate wireless scanning tools. These operators should be trained on thefunctionality and capability of the scanning tools and software to better understand the captured
information and be more apt to identify potential threats or malicious activity. Individuals with similar
15

16

For proper measures to secure IEEE 802.11-based WLANs, please refer to NIST SP 800-97, Establishing Wireless Robust
Security Networks: A Guide to IEEE 802.11i, and NIST SP 800-48 Revision 1, Guide to Securing Legacy IEEE 802.11
Wireless Networks, available at http://csrc.nist.gov/publications/PubsSPs.html.
For more information, see NIST SP 800-94, Guide to Intrusion Detection and Prevention Systems (IDPS), which is available
at http://csrc.nist.gov/publications/PubsSPs.html.

4-7

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

skills should be employed to analyze the data and results acquired from wireless scans. Scanning tool
operators should be aware of other RF signals authorized for use within the area being scanned.
4.4.1

Passive Wireless Scanning

Passive scanning should be conducted regularly to supplement wireless security measures already inplace, such as WIDPSs. 17 Wireless scanning tools used to conduct completely passive scans transmit nodata, nor do the tools in any way affect the operation of deployed wireless devices. By not transmitting
data, a passive scanning tool remains undetected by malicious users and other devices. This reduces thelikelihood of individuals avoiding detection by disconnecting or disabling unauthorized wireless devices.
Passive scanning tools capture wireless traffic being transmitted within the range of the tool’s antenna.
Most tools provide several key attributes regarding discovered wireless devices, including service set
identifier (SSID), device type, channel, media access control (MAC) address, signal strength, and numberof packets being transmitted. This information can be used to evaluate the security of the wirelessenvironment, and to identify potential rogue devices and unauthorized ad hoc networks discovered within
range of the scanning device. The wireless scanning tool should also be able to assess the captured
packets to determine if any operational anomalies or threats exist.
Wireless scanning tools scan each IEEE 802.11a/b/g/n channel/frequency separately, often for only
several hundred milliseconds at a time. The passive scanning tool may not receive all transmissions on a
specific channel. For example, the tool may have been scanning channel 1 at the precise moment when awireless device transmitted a packet on channel 5. This makes it important to set the dwell time of thetool to be long enough to capture packets, yet short enough to efficiently scan each channel. Dwell time
configurations will depend on the device or tool used to conduct the wireless scans. In addition, security
personnel conducting the scans should slowly move through the area being scanned to reduce the numberof devices that go undetected.
Rogue devices can be identified in several ways through passive scanning: The MAC address of a discovered wireless device indicates the vendor of the device’s wirelessinterface. If an organization only deploys wireless interfaces from vendors A and B, the presenceof interfaces from any other vendor indicates potential rogue devices.
 If an organization has accurate records of its deployed wireless devices, assessors can compare
the MAC addresses of discovered devices with the MAC addresses of authorized devices. Most
scanning tools allow assessors to enter a list of authorized devices. Because MAC addresses canbe spoofed, assessors should not assume that the MAC addresses of discovered devices areaccurate—but checking MAC addresses can identify rogue devices that do not use spoofing.
 Rogue devices may use SSIDs that are not authorized by the organization.
 Some rogue devices may use SSIDs that are authorized by the organization but do not adhere to
its wireless security configuration requirements.
The signal strength of potential rogue devices should be reviewed to determine whether the devices arelocated within the confines of the facility or in the area being scanned. Devices operating outside an

17

In some environments, the WIDPS implementation might be performing most of the same functions as passive wirelessscanning. Some WIDPS products offer mobile sensors similar to the wireless scanning device setup described in Section
4.4. Organizations with WIDPS implementations should use the wireless scanning techniques described in this publication
to supplement, not duplicate, WIDPS functionality.

4-8

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

organization’s confines might still pose significant risks because the organization’s devices might
inadvertently associate to them.
4.4.2

Active Wireless Scanning

Organizations can move beyond passive wireless scanning to conduct active scanning. This builds on theinformation collected during passive scans, and attempts to attach to discovered devices and conduct
penetration or vulnerability-related testing. For example, organizations can conduct active wirelessscanning on their authorized wireless devices to ensure that they meet wireless security configuration
requirements—including authentication mechanisms, data encryption, and administration access if thisinformation is not already available through other means.
Organizations should be cautious in conducting active scans to make sure they do not inadvertently scan
devices owned or operated by neighboring organizations that are within range. It is important to evaluate
the physical location of devices before actively scanning them. Organizations should also be cautious inperforming active scans of rogue devices that appear to be operating within the organization’s facility.
Such devices could belong to a visitor to the organization who inadvertently has wireless access enabled,or to a neighboring organization with a device that is close to, but not within, the organization’s facility.
Generally, organizations should focus on identifying and locating potential rogue devices rather thanperforming active scans of such devices.
Organizations may use active scanning when conducting penetration testing on their own wirelessdevices. Tools are available that employ scripted attacks and functions, attempt to circumvent
implemented security measures, and evaluate the security level of devices. For example, tools used to
conduct wireless penetration testing attempt to connect to access points (AP) through various methods tocircumvent security configurations. If the tool can gain access to the AP, it can obtain information andidentify the wired networks and wireless devices to which the AP is connected. Some active tools may
also identify vulnerabilities discovered on the wireless client devices, or conduct wired network
vulnerability tests as outlined in Section 4.
While active scanning is being performed, the organization’s WIDPSs can be monitored to evaluate theircapabilities and performance. Depending on assessment goals, assessors conducting these scans mayneed to inform the WIDPS administrators and wireless network administrators of pending scanning toprepare them for possible alarms and alerts. In addition, some WIDPSs can be configured to ignore
alarms and alerts triggered by a specific device—such as one used to perform scanning.
Tools and processes to identify unauthorized devices and vulnerabilities on wired networks can also be
used to identify rogue and misconfigured wireless devices. Wired-side scanning is another process thatcan be conducted to discover, and possibly locate, rogue wireless devices. Sections 3.5 and 4.1 discuss
wired scanning.
4.4.3

Wireless Device Location Tracking

Security personnel who operate the wireless scanning tool should attempt to locate suspicious devices.
RF signals propagate in a manner relative to the environment, which makes it important for the operator
to understand how wireless technology supports this process. Mapping capabilities are useful here, but
the main factors needed to support this capability are a knowledgeable operator and an appropriate
wireless antenna.
If rogue devices are discovered and physically located during the wireless scan, security personnel should
ensure that specific policies and processes are followed on how the rogue device is handled—such as

4-9

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

shutting it down, reconfiguring it to comply with the organization’s policies, or removing the device
completely. If the device is to be removed, security personnel should evaluate the activity of the rogue
device before it is confiscated. This can be done through monitoring transmissions and attempting toaccess the device.
If discovered wireless devices cannot be located during the scan, security personnel should attempt to use
a WIDPS to support the location of discovered devices. This requires the WIDPS to locate a specific
MAC address that was discovered during the scan. Properly deployed WIDPSs should have the ability toassist security personnel in locating these devices, and usually involves the use of multiple WIDPS
sensors to increase location identification granularity. Because the WIDPS will only be able to locate adevice within several feet, a wireless scanning tool may still be needed to pinpoint the location of thedevice.
4.4.4

Bluetooth Scanning

For organizations that want to confirm compliance with their Bluetooth security requirements, passive
scanning for Bluetooth-enabled wireless devices should be conducted to evaluate potential presence andactivity. Because Bluetooth has a very short range (on average 9 meters [30 feet], with some devices
having ranges of as little as 1 meter [3 feet]), scanning for devices can be difficult and time-consuming.
Assessors should take range limitations into consideration when scoping this type of scanning.
Organizations may want to perform scanning only in areas of their facilities that are accessible by the
public—to see if attackers could gain access to devices via Bluetooth—or to perform scanning in asampling of physical locations rather than throughout the entire facility. Because many Bluetoothenabled devices (such as cell phones and personal digital assistants [PDA]) are mobile, conducting
passive scanning several times over a period of time may be necessary. Organizations should also scan
any Bluetooth infrastructure, such as access points, that they deploy. If rogue access points are
discovered, the organization should handle them in accordance with established policies and processes.
A number of tools are available for actively testing the security and operation of Bluetooth devices.
These tools attempt to connect to discovered devices and perform attacks to surreptitiously gain accessand connectivity to Bluetooth-enabled devices. Assessors should be extremely cautious of performing
active scanning because of the likelihood of inadvertently scanning personal Bluetooth devices, which arefound in many environments. As a general rule, assessors should use active scanning only when they arecertain that the devices being scanned belong to the organization. Active scanning can be used toevaluate the security mode in which a Bluetooth device is operating, and the strength of Bluetooth
password identification numbers (PIN). Active scanning can also be used to verify that these devices are
set to the lowest possible operational power setting to minimize their range. As with IEEE 802.11a/b/g
rogue devices, rogue Bluetooth devices should be dealt with in accordance with policies and guidance.
4.5

Summary

Table 4-1 summarizes the major capabilities of the target identification and analysis techniques discussed
in Section 4.
Table 4-1. Target Identification and Analysis Techniques
Technique
Network Discovery••

Network Port andService Identification••

Capabilities
Discovers active devices
Identifies communication paths and facilitates determination of networkarchitectures
Discovers active devices
Discovers open ports and associated services/ applications

4-10

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Technique
Vulnerability Scanning

Wireless Scanning

Capabilities
•
•
•
•
•
•

Identifies hosts and open ports
Identifies known vulnerabilities (note: has high false positive rates)
Often provides advice on mitigating discovered vulnerabilities
Identifies unauthorized wireless devices within range of the scanners
Discovers wireless signals outside of an organization’s perimeter
Detects potential backdoors and other security violations

There are risks associated with each technique and combination of techniques. To ensure that all are
executed safely and accurately, each assessor should have a certain baseline skill set. Table 4-2 provides
guidelines for the minimum skill set needed for each technique presented in Section 4.
Table 4-2. Baseline Skill Set for Target Identification and Analysis Techniques
Technique

Baseline Skill Set

Network Discovery

General TCP/IP and networking knowledge; ability to use both passive and active
network discovery tools

Network Port andService Identification

General TCP/IP and networking knowledge; knowledge of ports and protocols for avariety of operating systems; ability to use port scanning tools; ability to interpret
results from tools

Vulnerability Scanning

General TCP/IP and networking knowledge; knowledge of ports, protocols, services,and vulnerabilities for a variety of operating systems; ability to use automatedvulnerability scanning tools and interpret/analyze the results

Wireless Scanning

General knowledge of computing and radio transmissions in addition to specificknowledge of wireless protocols, services, and architectures; ability to use automatedwireless scanning and sniffing tools

4-11

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

5.

Target Vulnerability Validation TechniquesThis section addresses target vulnerability validation techniques, which use information produced fromtarget identification and analysis to further explore the existence of potential vulnerabilities. The
objective is to prove that a vulnerability exists, and to demonstrate the security exposures that occur when
it is exploited. Target vulnerability validation involves the greatest amount of risk in assessments, sincethese techniques have more potential to impact the target system or network than other techniques.
Target vulnerability validation techniques for application security testing are briefly discussed inAppendix C.
5.1

Password Cracking

When a user enters a password, a hash of the entered password is generated and compared with a storedhash of the user’s actual password. If the hashes match, the user is authenticated. Password cracking is
the process of recovering passwords from password hashes stored in a computer system or transmitted
over networks. It is usually performed during assessments to identify accounts with weak passwords.
Password cracking is performed on hashes that are either intercepted by a network sniffer while beingtransmitted across a network, or retrieved from the target system, which generally requires administrativelevel access on, or physical access to, the target system. Once these hashes are obtained, an automatedpassword cracker rapidly generates additional hashes until a match is found or the assessor halts thecracking attempt.
One method for generating hashes is a dictionary attack, which uses all words in a dictionary or text file.
There are numerous dictionaries available on the Internet that encompass major and minor languages,
names, popular television shows, etc. Another cracking method is known as a hybrid attack, which builds
on the dictionary method by adding numeric and symbolic characters to dictionary words. Depending on
the password cracker being used, this type of attack can try a number of variations, such as using common
substitutions of characters and numbers for letters (e.g., p@ssword and h4ckme). Some will also try
adding characters and numbers to the beginning and end of dictionary words (e.g., password99,
password$%).
Yet another password-cracking method is called the brute force method. This generates all possible
passwords up to a certain length and their associated hashes. Since there are so many possibilities, it cantake months to crack a password. Although brute force can take a long time, it usually takes far less timethan most password policies specify for password changing. Consequently, passwords found during brute
force attacks are still too weak. Theoretically, all passwords can be cracked by a brute force attack, given
enough time and processing power, although it could take many years and require serious computing
power. Assessors and attackers often have multiple machines over which they can spread the task ofcracking passwords, which greatly shortens the time involved.
Password cracking can also be performed with rainbow tables, which are lookup tables with precomputed password hashes. For example, a rainbow table can be created that contains every possible
password for a given character set up to a certain character length. Assessors may then search the table
for the password hashes that they are trying to crack. Rainbow tables require large amounts of storage
space and can take a long time to generate, but their primary shortcoming is that they may be ineffective
against password hashing that uses salting. Salting is the inclusion of a random piece of information in
the password hashing process that decreases the likelihood of identical passwords returning the samehash. Rainbow tables will not produce correct results without taking salting into account—but thisdramatically increases the amount of storage space that the tables require. Many operating systems use

5-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

salted password hashing mechanisms to reduce the effectiveness of rainbow tables and other forms ofpassword cracking.
Password crackers can be run during an assessment to ensure policy compliance by verifying acceptablepassword composition. For example, if the organization has a password expiration policy, then password
crackers can be run at intervals that coincide with the intended password lifetime. Password cracking thatis performed offline produces little or no impact on the system or network, and the benefits of thisoperation include validating the organization’s password policy and verifying policy compliance.
5.2

Penetration Testing

Penetration testing is security testing in which assessors mimic real-world attacks to identify methods forcircumventing the security features of an application, system, or network. It often involves launching real
attacks on real systems and data that use tools and techniques commonly used by attackers. Most
penetration tests involve looking for combinations of vulnerabilities on one or more systems that can beused to gain more access than could be achieved through a single vulnerability. Penetration testing canalso be useful for determining: How well the system tolerates real world-style attack patterns
 The likely level of sophistication an attacker needs to successfully compromise the system Additional countermeasures that could mitigate threats against the system
 Defenders’ ability to detect attacks and respond appropriately.
Penetration testing can be invaluable, but it is labor-intensive and requires great expertise to minimize therisk to targeted systems. Systems may be damaged or otherwise rendered inoperable during the course ofpenetration testing, even though the organization benefits in knowing how a system could be rendered
inoperable by an intruder. Although experienced penetration testers can mitigate this risk, it can never befully eliminated. Penetration testing should be performed only after careful consideration, notification,and planning.
Penetration testing often includes non-technical methods of attack. For example, a penetration tester
could breach physical security controls and procedures to connect to a network, steal equipment, capturesensitive information (possibly by installing keylogging devices), or disrupt communications. Caution
should be exercised when performing physical security testing—security guards should be made aware of
how to verify the validity of tester activity, such as via a point of contact or documentation. Another nontechnical means of attack is the use of social engineering, such as posing as a help desk agent and calling
to request a user’s passwords, or calling the help desk posing as a user and asking for a password to bereset. Additional information on physical security testing, social engineering techniques, and other nontechnical means of attack included in penetration testing lies outside the scope of this publication.
5.2.1

Penetration Testing Phases

Figure 5-1 represents the four phases of penetration testing. 18 In the planning phase, rules are identified,management approval is finalized and documented, and testing goals are set. The planning phase sets thegroundwork for a successful penetration test. No actual testing occurs in this phase.

18This is an example of how the penetration process can be divided into phases. There are many acceptable ways of grouping
the actions involved in performing penetration testing.

5-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Additional Discovery

Planning

Discovery

AttackReportingFigure 5-1. Four-Stage Penetration Testing Methodology

The discovery phase of penetration testing includes two parts. The first part is the start of actual testing,and covers information gathering and scanning. Network port and service identification, described in
Section 4.2, is conducted to identify potential targets. In addition to port and service identification, othertechniques are used to gather information on the targeted network: Host name and IP address information can be gathered through many methods, including DNSinterrogation, InterNIC (WHOIS) queries, and network sniffing (generally only during internal
tests)
 Employee names and contact information can be obtained by searching the organization’s Web
servers or directory servers
 System information, such as names and shares can be found through methods such as
NetBIOS enumeration (generally only during internal tests) and Network Information System(NIS) (generally only during internal tests)
 Application and service information, such as version numbers, can be recorded through banner
grabbing.
In some cases, techniques such as dumpster diving and physical walkthroughs of facilities may be used tocollect additional information on the targeted network, and may also uncover additional information to be
used during the penetration tests, such as passwords written on paper.
The second part of the discovery phase is vulnerability analysis, which involves comparing the services,
applications, and operating systems of scanned hosts against vulnerability databases (a process that is
automatic for vulnerability scanners) and the testers’ own knowledge of vulnerabilities. Human testers
can use their own databases—or public databases such as the National Vulnerability Database (NVD) —to identify vulnerabilities manually. Appendix E has more information on these publicly available
vulnerability databases. Manual processes can identify new or obscure vulnerabilities that automatedscanners may miss, but are much slower than an automated scanner.
Executing an attack is at the heart of any penetration test. Figure 5-2 represents the individual steps of theattack phase—the process of verifying previously identified potential vulnerabilities by attempting to
exploit them. If an attack is successful, the vulnerability is verified and safeguards are identified tomitigate the associated security exposure. In many cases, exploits 19 that are executed do not grant the19

Exploit programs or scripts are specialized tools for exploiting specific vulnerabilities. The same cautions that apply tofreeware tools apply to exploit programs and scripts. Some vulnerability databases, including Bugtraq (available at
http://www.securityfocus.com/) provide exploit instructions or code for many identified vulnerabilities.

5-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

maximum level of potential access to an attacker. They may instead result in the testers learning more
about the targeted network and its potential vulnerabilities, or induce a change in the state of the targeted
network’s security. Some exploits enable testers to escalate their privileges on the system or network to
gain access to additional resources. If this occurs, additional analysis and testing are required todetermine the true level of risk for the network, such as identifying the types of information that can begleaned, changed, or removed from the system. In the event an attack on a specific vulnerability proves
impossible, the tester should attempt to exploit another discovered vulnerability. If testers are able toexploit a vulnerability, they can install more tools on the target system or network to facilitate the testingprocess. These tools are used to gain access to additional systems or resources on the network, and obtain
access to information about the network or organization. Testing and analysis on multiple systems should
be conducted during a penetration test to determine the level of access an adversary could gain. This
process is represented in the feedback loop in Figure 5-1 between the attack and discovery phase of apenetration test.

Figure 5-2. Attack Phase Steps with Loopback to Discovery Phase

While vulnerability scanners check only for the possible existence of a vulnerability, the attack phase of apenetration test exploits the vulnerability to confirm its existence. Most vulnerabilities exploited bypenetration testing fall into the following categories: Misconfigurations. Misconfigured security settings, particularly insecure default settings, areusually easily exploitable.
 Kernel Flaws. Kernel code is the core of an OS, and enforces the overall security model for thesystem—so any security flaw in the kernel puts the entire system in danger.
 Buffer Overflows. A buffer overflow occurs when programs do not adequately check input for
appropriate length. When this occurs, arbitrary code can be introduced into the system andexecuted with the privileges—often at the administrative level—of the running program.

5-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

 Insufficient Input Validation. Many applications fail to fully validate the input they receivefrom users. An example is a Web application that embeds a value from a user in a database
query. If the user enters SQL commands instead of or in addition to the requested value, and the
Web application does not filter the SQL commands, the query may be run with malicious changes
that the user requested—causing what is known as a SQL injection attack.
 Symbolic Links. A symbolic link (symlink) is a file that points to another file. Operating
systems include programs that can change the permissions granted to a file. If these programs run
with privileged permissions, a user could strategically create symlinks to trick these programs
into modifying or listing critical system files.
 File Descriptor Attacks. File descriptors are numbers used by the system to keep track of files
in lieu of filenames. Specific types of file descriptors have implied uses. When a privileged
program assigns an inappropriate file descriptor, it exposes that file to compromise.
 Race Conditions. Race conditions can occur during the time a program or process has enteredinto a privileged mode. A user can time an attack to take advantage of elevated privileges while
the program or process is still in the privileged mode.
 Incorrect File and Directory Permissions. File and directory permissions control the accessassigned to users and processes. Poor permissions could allow many types of attacks, including
the reading or writing of password files or additions to the list of trusted remote hosts.
The reporting phase occurs simultaneously with the other three phases of the penetration test (see Figure5-1). In the planning phase, the assessment plan—or ROE—is developed. In the discovery and attack
phases, written logs are usually kept and periodic reports are made to system administrators and/or
management. At the conclusion of the test, a report is generally developed to describe identified
vulnerabilities, present a risk rating, and give guidance on how to mitigate the discovered weaknesses.
Section 8 discusses post-testing activities such as reporting in more detail.
5.2.2

Penetration Testing Logistics

Penetration test scenarios should focus on locating and targeting exploitable defects in the design andimplementation of an application, system, or network. Tests should reproduce both the most likely andmost damaging attack patterns—including worst-case scenarios such as malicious actions byadministrators. Since a penetration test scenario can be designed to simulate an inside attack, an outside
attack, or both, external and internal security testing methods are considered. If both internal and external
testing is to be performed, the external testing usually occurs first.
Outsider scenarios simulate the outsider-attacker who has little or no specific knowledge of the target and
who works entirely from assumptions. To simulate an external attack, testers are provided with no real
information about the target environment other than targeted IP addresses or address ranges, 20 and
perform open source research by collecting information on the targets from public Web pages,
newsgroups, and similar sites. Port scanners and vulnerability scanners are then used to identify target
hosts. Since the testers’ traffic usually goes through a firewall, the amount of information obtained fromscanning is far less than if the test were undertaken from an insider perspective. After identifying hosts
on the network that can be reached from outside, testers attempt to compromise one of the hosts. If
successful, this access may then be used to compromise other hosts that are not generally accessible from20

If given a list of authorized IP addresses to use as targets, assessors should verify that all public addresses (i.e., not private,unroutable addresses) are under the organization’s purview before testing begins. Web sites that provide domain nameregistration information (e.g., WHOIS) can be used to determine owners of address spaces.

5-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

outside the network. Penetration testing is an iterative process that leverages minimal access to gain
greater access.
Insider scenarios simulate the actions of a malicious insider. An internal penetration test is similar to an
external test, except that the testers are on the internal network (i.e., behind the firewall) and have been
granted some level of access to the network or specific network systems. Using this access, thepenetration testers try to gain a greater level of access to the network and its systems through privilege
escalation. Testers are provided with network information that someone with their level of access would
normally have—generally as a standard employee, although depending on the goals of the test it could
instead be information that a system or network administrator might possess.
Penetration testing is important for determining the vulnerability of an organization’s network and thelevel of damage that can occur if the network is compromised. It is important to be aware that dependingon an organization’s policies, testers may be prohibited from using particular tools or techniques or may
be limited to using them only during certain times of the day or days of the week. Penetration testing also
poses a high risk to the organization’s networks and systems because it uses real exploits and attacks
against production systems and data. Because of its high cost and potential impact, penetration testing ofan organization’s network and systems on an annual basis may be sufficient. Also, penetration testing canbe designed to stop when the tester reaches a point when an additional action will cause damage. The
results of penetration testing should be taken seriously, and any vulnerabilities discovered should be
mitigated. Results, when available, should be presented to the organization’s managers. Organizations
should consider conducting less labor-intensive testing activities on a regular basis to ensure that they are
maintaining their required security posture. A well-designed program of regularly scheduled network andvulnerability scanning, interspersed with periodic penetration testing, can help prevent many types ofattacks and reduce the potential impact of successful ones.
5.3

Social EngineeringSocial engineering is an attempt to trick someone into revealing information (e.g., a password) that can beused to attack systems or networks. It is used to test the human element and user awareness of security,and can reveal weaknesses in user behavior—such as failing to follow standard procedures. Social
engineering can be performed through many means, including analog (e.g., conversations conducted inperson or over the telephone) and digital (e.g., e-mail, instant messaging). One form of digital social
engineering is known as phishing, where attackers attempt to steal information such as credit card
numbers, Social Security numbers, user IDs, and passwords. Phishing uses authentic-looking emails torequest information or direct users to a bogus Web site to collect information. Other examples of digitalsocial engineering include crafting fraudulent e-mails and sending attachments that could mimic worm
activity.
Social engineering may be used to target specific high-value individuals or groups in the organization,such as executives, or may have a broad target set. Specific targets may be identified when theorganization knows of an existing threat or feels that the loss of information from a person or specificgroup of persons could have a significant impact. For example, phishing attacks can be targeted based on
publicly available information about specific individuals (e.g., titles, areas of interest). Individual
targeting can lead to embarrassment for those individuals if testers successfully elicit information or gain
access. It is important that the results of social engineering testing are used to improve the security of theorganization and not to single out individuals. Testers should produce a detailed final report that
identifies both successful and unsuccessful tactics used. This level of detail will help organizations totailor their security awareness training programs.

5-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

5.4

Summary

Each information security testing technique has its own strengths and weaknesses. Table 5-1 compares
the range of testing techniques discussed in Section 5.
Table 5-1. Target Vulnerability Validation Techniques
Technique
Password Cracking
Penetration Testing
Social Engineering•

Capabilities
Identifies weak passwords and password policies

•
•
•
•

Tests security using the same methodologies and tools that attackers employ
Verifies vulnerabilities
Demonstrates how vulnerabilities can be exploited iteratively to gain greater access
Allows testing of both procedures and the human element (user awareness)

Risks are associated with all techniques and technique combinations. To ensure that each technique is
executed safely and accurately, testers should have a specific baseline skill set. Table 5-2 provides
guidance on the minimum skill sets needed for testing techniques presented in this guide.
Table 5-2. Security Testing Knowledge, Skills, and Abilities
Technique

Baseline Skill Set

Password Cracking

Knowledge of secure password composition and password storage for operating systems;
ability to use automated cracking tools

Penetration Testing

Extensive TCP/IP, networking, and OS knowledge; advanced knowledge of network andsystem vulnerabilities and exploits; knowledge of techniques to evade security detection

Social Engineering

Ability to influence and persuade people; ability to remain composed under pressure

5-7

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

6.

Security Assessment Planning

Proper planning is critical to a successful security assessment. This section provides guidance on creating
an assessment policy, prioritizing and scheduling assessments, selecting the appropriate assessment
approach, and addressing logistical considerations. It also provides recommendations for developing anassessment plan and outlines assessment-related legal considerations that organizations may need toaddress.
6.1

Developing a Security Assessment Policy

Organizations should develop an information security assessment policy to provide direction andguidance for their security assessments. This policy should identify security assessment requirements,and hold accountable those individuals responsible for ensuring that assessments comply with therequirements. It should address:
 Organizational requirements with which assessments must comply
 Appropriate roles and responsibilities (at a minimum, for those individuals approving andexecuting assessments)
 Adherence to established methodology
 Assessment frequency
 Documentation requirements, such as assessment plans and assessment results.
Once developed and approved by the appropriate senior officials, the policy should be disseminated to theappropriate staff—which might include the offices of the Chief Information Officer (CIO), Chief
Information Security Officer (CISO), and Chief Technology Officer (CTO). Leadership should also
communicate the policy to any third parties who are to conduct assessments.
It is recommended that organizations review their assessment policy at least annually, and whenever there
are new assessment-related requirements. These reviews will determine the policy’s continued
applicability, address any necessary modifications, and provide opportunities for incorporating lessons
learned.
6.2

Prioritizing and Scheduling Assessments

As part of planning, organizations should decide which systems should undergo technical security
assessments and how often these assessments should be done. This prioritization is based on system
categorization, expected benefits, scheduling requirements, and applicable regulations where assessment
is a requirement. A good starting point is to evaluate system categorization and associated requirements
for security assessment. Here, an evaluation of the system’s impact rating (e.g., low, moderate, high) 21
and security assessment status (e.g., when was an assessment last conducted) is necessary to determine aschedule for moving forward. For instance, organizations should generally assess a high-impact systembefore a moderate-impact system—but a moderate-impact system that is overdue may need to beevaluated before a high-impact system whose last security assessment is still within the acceptable

21

FIPS PUB 199, Standards for Security Categorization of Federal Information and Information Systems, provides standards
for determining the security category of an organization’s information systems which can be helpful in developing a priority
ranking of those systems for testing purposes. FIPS PUB 199 is available for download fromhttp://csrc.nist.gov/publications/PubsFIPS.html.

6-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

timeframe. As part of continuous monitoring, 22 a number of NIST SP 800-53 security controls must also
be constantly tested. 23
Assessment frequency is often driven by an organization’s requirements to demonstrate compliance withspecific regulations or policies. For example, FISMA requires periodic testing depending on risk, to bedone at least annually. NIST SPs 800-53 and 800-53A provide organizations with recommendations
regarding the frequency of conducting security assessments. Since an assessment provides a snapshot of
security at a given point in time, organizations may choose to require more frequent assessments.
Important technical considerations can also help determine testing frequency. For example, if a system isbelieved to have several weaknesses, testing might be conducted sooner to confirm the presence of the
weaknesses—or delayed until the weaknesses have been mitigated, to confirm they have been resolved.
The timing used depends on the testing objective. Another consideration is whether any system or
network activities required by the testing may impact the functionality or security of the environment—
for example, if a major upgrade is about to be conducted, testing might be delayed until the upgrade has
been completed. Another example of a technical consideration is when an organization wants to identify
rogue devices on wired networks. This could be accomplished using one or more techniques, such as
performing network discovery through passive sniffing or active scanning, or reviewing data collected by
network management software, network intrusion detection sensors, or other devices that routinely
monitor network activity. If these monitoring devices are able to generate alerts as soon as a new,potentially rogue device is observed on the network, there may be little or no need to perform periodic
testing for rogue devices because effective testing is continuously being performed.
Organizations also need to carefully consider resource availability. Resources should first be identified
for high-priority systems, after which lower-priority systems may be tested with less frequency and indescending order. If a gap exists between required and available resources, the organization may need toallocate additional resources and consider reducing the scope of its planned assessments. Examples ofscoping elements that may be relevant include: The size of what is being assessed, in terms of number of components (e.g., single database, alluser systems, or entire architecture) and network size (e.g., Local Area Network [LAN] or WideArea Network [WAN], number of network locations that a tester will need to physically plug into
for testing).
 The complexity of what is being assessed. More heterogeneous environments generally requirelarger amounts of resources because more diverse skill sets and tools are needed.
 The feasibility of using a sample for assessment, along with the sample size and its makeup. For
example, it may be much more efficient—and nearly as effective—to port scan a small sample ofhosts rather than thousands of hosts, especially if the hosts are managed and similarly configured.
 The level of resources needed to conduct specific testing or examination techniques. For
example, it could take many hours for a skilled assessor to review a system’s complete security
documentation.

22

23

NIST SP 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, Section 3.4,
provides guidance on the continuous monitoring phase of the accreditation process. See
http://csrc.nist.gov/publications/PubsSPs.html.
Continuous monitoring activities include configuration management and control of information system components, security
impact analyses of changes to the system, ongoing assessment of security controls, and status reporting. NIST SP 800-53
http://csrc.nist.gov/publications/PubsSPs.html provides additional guidance.

6-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

 The level of human interaction required. For instance, if assessors are to work in tandem with IT
staff, this may serve as a form of training for the IT staff but will likely increase the time needed
to complete the assessment when compared to the time needed by assessors and IT staff working
independently.
6.3

Selecting and Customizing Techniques

There are many factors to consider when determining which technical testing and examination techniquesshould be used for a particular assessment. An organization should first determine its assessment
objectives, such as focusing on verifying compliance with a particular mandate, verifying a system’s
security as part of certification and accreditation (C&A) activities, identifying exploitable vulnerabilities
in a group of systems, or evaluating intrusion detection system and incident handling procedure
performance. Next, the organization should select the classes of techniques (e.g., review, target
identification and analysis, target vulnerability validation) to be used to obtain information that supports
those objectives, and specific techniques within each selected class. For some testing techniques, theorganization must also determine the assessors’ viewpoint (e.g., internal versus external, covert versus
overt) and select corresponding techniques.
Since in most cases more than one technique can be used to meet an assessment objective, organizations
need to determine which techniques are best for each case. As discussed in Section 6.2, one importantconsideration is resources—some techniques may cost substantially more than others to use because of
the types of tools required and the number of hours of staff time needed. Some techniques may also take
too long to perform—if there is a short timeframe for conducting an assessment, less extensive or
resource-intensive techniques may be needed, such as performing vulnerability scanning rather than apenetration test. Skills are another important factor in technique selection—for example, an organization
may not have assessors on staff with the appropriate skill sets to use certain specialized techniques.
Organizations should also carefully consider risk when selecting testing techniques. Some techniques,
such as penetration testing, could lead to loss of system availability or exposure of sensitive data. In
some cases, organizations should consider whether testing should be performed on production systems orsimilarly configured non-production systems, if such alternate systems are available, or restrict the use ofcertain techniques to off-hours so as to minimize impact to operations. Factors to evaluate when making
such decisions include: The possible impact to the production systems. For example, if a particular test technique islikely to cause a denial of service, it should probably be used against a non-production system.
 The presence of sensitive personally identifiable information (PII). If testing could expose
sensitive PII—such as Social Security numbers (SSN) or credit card information—to individuals
who are not authorized to have access, organizations should consider performing their testing on
a non-production system with a false version of the PII (e.g., test data instead of actual PII).
 How similarly the production and non-production systems can be configured. In practice, there
are usually inconsistencies between the test and production environments, which can result in
missed vulnerabilities if non-production systems are used.
Organizations often use a combination of techniques to achieve an in-depth security assessment while
maintaining an acceptable level of risk to systems and networks. As mentioned in Section 2, nontechnical techniques may be used instead of or in addition to technical techniques; many assessments use
a combination of non-technical and technical techniques.

6-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

The following examples show how multiple technical techniques can complement one another and how
selection of techniques can relate to risk concerns. These examples are intended as illustrations rather
than as recommended combinations of techniques for organizations’ assessments. Each case is different,
and organizations should evaluate the requirements and objectives of each assessment when determining
an appropriate combination of techniques.
 Identify technical weaknesses in a system’s security architecture and security configuration while
minimizing risk from the assessment itself.

–

Step 1. Documentation Review. Identify policy and procedure weaknesses and security
architecture flaws.

–

Step 2. Ruleset and Security Configuration Review. Identify deviations fromorganizational security policies in the forms of the system’s network security architecture andsystem security flaws.

–

Step 3. Wireless Scanning. Identify rogue wireless devices within proximity of the system,
and additional security architecture weaknesses related to the wireless networks used by thesystem.

–Step 4. Network Discovery and Vulnerability Scanning. Identify all active hosts within
the system and their known vulnerabilities.

 Identify and validate technical weaknesses in a system’s security architecture and security
configuration—validation will include attempts to exploit selected vulnerabilities.

–

Step 1. Ruleset and Security Configuration Review. Identify deviations fromorganizational security policies in the forms of the system’s network security architecture andsystem security flaws.

–

Step 2. Network Discovery and Vulnerability Scanning. Identify all active hosts within
the system and their known vulnerabilities.

–

Step 3. Penetration Test with Social Engineering. Validate vulnerabilities in the system.

 Identify and validate technical weaknesses in a system’s security architecture and security
configuration from an external attacker’s viewpoint—validation will include attempting to exploit
some or all vulnerabilities. Evaluate the effectiveness of the organization’s audit capabilities for
attacks against the system.

6.4–

Step 1. External Penetration Testing. Perform external network discovery, port scanning,vulnerability scanning, and attacks to identify and validate system vulnerabilities.

–

Step 2. Log Review. Review security control audit logs for the system to determine theireffectiveness in capturing information relating to external penetration testing activities.

Assessment Logistics

Addressing logistics for technical assessments includes identifying all resources required for conducting
the assessment; the environment from which to test; and required hardware and software testing tools.
These are addressed in the subsections below.
In addition to the standard logistical requirements discussed below, it is equally important to identify
logistical requirements for each test during the planning phase. Depending on the scope and the

6-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

environment, individual tests may have additional logistical requirements such as submitting a visit
request for an external test team, shipping equipment to a facility to enable testing, and planning for local
or long-distance travel. These needs should be addressed on a case-by-case basis during the planning
process.
6.4.1

Assessor Selection and Skills

Assessors conduct examinations and tests using technical methods and techniques, such as thosedescribed in this guide. Organizations should take care when selecting assessors, because properly vetted,
skilled, and experienced assessors will lower the risks involved in conducting security tests. Because
assessors may also require access to sensitive information on network architecture, security posture, andweaknesses, some organizations may require background checks or security clearances. Organizations
should also be mindful of possible conflicts of interest, such as a single individual conducting a formal
assessment and being responsible for addressing the findings of that assessment.
Many organizations have dedicated internal assessment teams. Depending on an organization’s structure,size, location, and available resources, these teams may be divided by geographical location or centralized
and deployed to various sites to conduct their assessments. Some teams address specific technicalcompetencies, such as wireless security testing, while other teams can address many areas of security invarying levels of depth. For instance, a team may have among its members some individuals who arecapable of reviewing a system configuration, others who can use automated assessment tools to identify
known vulnerabilities, and still others who are able to actively exploit vulnerabilities to demonstrateineffective security measures.
Assessors should have significant security and networking knowledge, including expertise in network
security, firewalls, intrusion detection systems, operating systems, programming, and networking
protocols (such as TCP/IP). A wide range of technical skill sets is required to conduct testing in an
effective and efficient manner while ensuring minimal risk. Assessors should also be skilled in thespecific types of techniques being executed, such as vulnerability identification and verification, security
configuration, vulnerability management, and penetration testing. Operational experience is preferred toclassroom or laboratory training. Allowing inexperienced or untrained staff to conduct technical tests cannegatively affect an organization’s systems and networks, potentially hindering its mission and damaging
the credibility of its security program management office and assessors. It is also beneficial to have atechnical writer or other individual on the team with strong technical writing skills. This helps the teamto effectively convey the results of the assessment, particularly to less technical readers.
When assessments are performed by a team, the team leader facilitates the assessment process;demonstrates an understanding of the organization’s environment and requirements; and (if applicable)
eases communication between the assessors and the organization’s security group. The team’s leader
should be selected based on overall technical knowledge and experience with the type of techniques being
executed, and knowledge of the assets being assessed. Team leaders should also have strong
communication, organization, planning, and conflict resolution skills.
The skills possessed by an assessment team should be balanced to provide a well-rounded view of theorganization’s security posture. For example, having an individual that specializes in perimeter defense ishelpful, but having a team full of people that specialize in perimeter defense is likely to be redundantunless the testing’s sole focus is to determine the perimeter’s security posture. Ideally, a team isassembled based on the individual requirements of the examinations and tests being conducted. Systemcharacteristics may also be important—for instance, supervisory control and data acquisition (SCADA)
systems have a number of unique components with which a traditional security assessor may not befamiliar, reducing the assessor’s ability to safely and adequately test the security posture of those systems.

6-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

In this type of case, one or more subject matter experts (SME) may be needed to augment the regularassessors. The SME may be an experienced security tester and system expert, or may be skilled only inthe system being tested. Regardless, SMEs should be educated on the goals, objectives, approach, andprocess of the assessment—and should also be included in the planning process whenever possible
because they may have critical knowledge to contribute.
Assessors need to remain abreast of new technology and the latest means by which an adversary may
attack that technology. They should periodically refresh their knowledge base, reassess their
methodology-updating techniques as appropriate, and update their tool kits. For example, attending
technical training courses, performing hands-on testing in a test environment, or researching the latest
vulnerabilities and exploits are just a few activities in which assessors should regularly engage. Assessors
should also perform technical hands-on tests in operational environments on a regular basis to maintainand improve their skills.
Responsibilities of assessors include: Informing the appropriate parties—such as security officers, management, system administrators,
and users—of security assessment activities
 Developing assessment plans with system managers, the Information Systems Security Officer(ISSO), and the CISO
 Executing examinations and tests, and collecting all relevant data
 Analyzing collected data and developing mitigation recommendations
 Conducting additional examinations and tests when needed to validate mitigation actions.
In some cases, engaging third parties (e.g., auditors, contractor support staff) to conduct the assessment
offers an independent view and approach that internal assessors may not be able to provide.
Organizations may also use third parties to provide specific subject matter expertise that is not available
internally. While it can be beneficial to gain an external perspective on the security posture, giving
outsiders access to an organization’s systems can introduce additional risk. External entities should beproperly vetted to ensure that they possess the necessary skills, experience, and integrity, and should be
asked to assume some of the risk associated with the security assessment in that they may be responsiblefor damages incurred by the organization being assessed. External entities should also understand andcomply with the organization’s applicable policies and operational and security requirements.
In addition to those listed above, the responsibilities for external assessors include: Coordinating and communicating with the organization being assessed Ensuring that proper authority is granted, and maintaining a signed copy of the assessment plan toensure all updates are documented
 Signing and abiding by any required nondisclosure agreements
 Properly protecting data in accordance with the organization’s regulations, including handling,transmission, storage, and deletion of all collected data and resulting reports.
6.4.2

Location Selection

The environment in which assessors operate differs according to the techniques being used. For many
types of tests, assessors can operate either onsite or offsite, with onsite testing defined as testing executed

6-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENTat the organization’s location. Placing assessors offsite, however, may make the test more realistic (e.g.,when applying the covert testing approach). For examinations, assessors are generally located onsite sothey can easily access the organization’s security documentation, logs, and other information. For
assessments performed by third parties, the organization will need to determine the appropriate level ofphysical access (e.g., unrestricted, escorted). For technical assessments conducted from within the
network—such as security configuration reviews and vulnerability scanning—assessors should be
provided network access either onsite, through an encrypted virtual private network (VPN) tunnel, or via
a dedicated connection from a trusted environment such as an approved test lab. 24
Assessors may require different levels of access to the network depending on the tools that they use.
Some tools require network or domain administrator privileges—if this is the case, organizations should
create new administrator accounts for use during assessments. Each assessor should have his or her own
account—administrator accounts should not be shared for any reason. This approach allows theorganization to monitor these accounts, which will be disabled or deleted at the assessment’s conclusion.
Technical assessments conducted from outside the network’s perimeter can be executed following a
number of scenarios, of which the most common are discussed here. The assessors’ systems can beconnected directly to a perimeter device (e.g., border router), which keeps the assessors within theorganization’s logical and physical boundaries. However, use of this location does not provide a true
evaluation of the organization’s security posture from an adversarial viewpoint. External tests can also be
executed from a test lab with an Internet connection that is independent from the network of theorganization being tested—and, if applicable, the organization conducting the testing (e.g., third-party
assessors conducting the tests from their own facility). 25 Organizations conducting external tests may
also choose to rent a server and an independent Internet connection. These services are provided by avariety of vendors, typically for a monthly fee. If a rented server is used, assessors should securely delete
the data on the system and rebuild it before conducting a security test. Once testing is complete, the teamshould follow the guidelines provided in Section 7.4 for data handling.
When selecting a location for assessment activities, organizations should consider the inherent risks ofusing external locations. These typically offer less control over physical and logical access to external
locations than internal locations, and may place assessment systems and data at a greater risk ofcompromise. Network traffic between the external location and the organization’s facilities is also atgreater risk of being monitored by unauthorized parties, which could expose security weaknesses detectedby tests. There may also be issues with performing certain types of testing, such as penetration testing,over third-party networks—such tests may appear malicious in nature to security staff monitoring
network usage, and may even violate the security policies of the network provider.
As previously discussed in Section 5, the location of the assessment systems may affect the results ofcertain types of tests. For example, if vulnerability scanning network traffic passes through a firewall,that firewall might inadvertently block portions of the traffic and prevent certain vulnerabilities frombeing detected. Also, intrusion detection and prevention systems and other security controls might block
network traffic perceived as malicious in nature, such as certain types of tests. These problems areexacerbated when tests are run from an external location over a third-party network, in which case neither
assessors nor the organization may have knowledge of or control over the security features interferingwith test activities.
24

25

Systems being tested may not be located on a production network, in which case the test team may need to be provided
access to the non-production network used by those systems.
Using an independent network is particularly advantageous if covert testing is being conducted. This can make it moredifficult for the security staff to identify the source of the activity (i.e., the IP addresses are not associated with a test team ororganization). Also, it prevents an inadvertent denial of service against legitimate users, which could occur if the security
staff blocked access from the testers’ IP address range in response to the testing activity.

6-7

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

6.4.3

Technical Tools and Resources Selection

Information systems built to execute a security assessment should meet the requirements of the specific
type of assessment and its expected tools. For example, systems for document review should haveapplications installed to read documents, track vulnerabilities, and compose reports. Systems designed to
execute tests such as vulnerability assessments and penetration testing are more complex in terms ofsystem requirements and software tools. Systems for technical assessments can include servers,
workstations, or laptops. Laptops are generally used by traveling assessors, and servers or workstations
may be used if assessors are in a test lab or an onsite location. Assessors may also establish a networkfrom which to execute techniques—this enables an environment that supports the centralized logging ofactivities and servers dedicated to activities that require increased processing power.
The requirements of test systems vary. A system that can handle the processing and memory
requirements of all tools, operating systems, and virtual machines 26 (VM) should be used to lessen thelikelihood of the system crashing during a test. A crash could cause that component of the test to need tobe redone, data to be lost, and test systems to be rebuilt. Processing power and memory requirements are
driven by both the tools used and the speed with which the test team expects to process certain
components. For example, password cracking generally requires increased processing power andmemory, so test teams may wish to have a dedicated password-cracking server. A dedicated system willallow the team to execute other test objectives during the password-cracking process. Hard drive
requirements will depend on the expected amount of data collected during a test. In the event that longterm storage of the data is required, a storage method (e.g., independent system or removable media)
should be identified and procured as appropriate.
Tools used by the test team will vary depending on the individual test scope, but the team should have acore set of tools that it uses and keeps up to date. Depending on the engagement and organization, a team
may use a combination of tools developed in-house, open source tools, and/or commercial or government
off-the-shelf (GOTS) tools. Tools should be obtained from well-established sources. Some organizations
may also have specific tools they require or encourage teams to use—for example, an organization may
purchase a license for a product that all its test teams can use. Many freeware tools are available as well.
Appendix A lists common tools, and describes the purpose of and how it can be obtained. Organizations
should take care to evaluate each tool before using it in a test—this process could range fromdownloading the tool from a trusted site to conducting an in-depth code review to ensure that the tool
does not contain malicious code.
Often, tools will determine the operating system required to execute the testing—including the need for
multiple operating systems. Systems may be configured a variety of ways, including single OS, single
OS with VM images, and dual-boot systems. An example of a dual-boot system is a system that can bebooted to either a version of Microsoft Windows or a version of Linux such as Red Hat, Mandrake, or
SuSE. A dual-boot system allows a tester to use two operating systems from a single machine, but thiscan be inconvenient because the tester needs to reboot the system to switch between each OS and its
tools.
Another more popular and functional option is to use VMs. Many testing tools require a specificoperating system, and VMs allow testers to use a wider variety of tools more easily because they allow
testers to switch from one OS to another without rebooting the system—enabling them to run multiple
operating systems simultaneously. This has several possible benefits, including logging, documentation
26

A virtual machine (VM) is software that allows a single host to run one or more guest operating systems. These operating
systems do not interact and are not aware of each other. A virtual machine monitor is the piece of software that controls
communication between the physical hardware and the individual VMs.

6-8

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

capabilities, and executing simultaneous tests. Since the system hosting the VM supports two or more
operating systems at once, test systems running VMs require greater processing power and memory.
Testers should be knowledgeable, experienced, and comfortable using all operating systems found on thetest system because system modifications are frequently required to operate specific tools or systemcapabilities successfully. For example, if the test team is using Red Hat Linux to conduct a wirelesssecurity test, the team will need to be familiar with installing and configuring wireless network cards
because the steps for doing so may not be obvious to a Red Hat Linux novice.
Regardless of the system installation method used, organizations conducting security tests should develop
and maintain a baseline image from which to conduct their tests. An image provides a standardized
toolkit for the team to use, and enables rapid deployment of a team. The baseline image should consist of
the operating system, drivers, requisite system and security configurations, applications, and tools to
conduct testing, including mechanisms for automatically logging assessor actions (e.g., commands
issued). Full system images are often hardware-dependent, so installing an image on another system withdifferent hardware (e.g., video cards) requires the test team to modify the image—which involves specificskills and is time-consuming. VM images are more versatile and do not carry the same hardware
restrictions as full system images, making them a more favorable option for test teams. Multifunction
teams—such as those with the skills to conduct wireless scans, application testing, vulnerability
assessments, and penetration tests—may have one image that contains the tools required to execute all
test types or multiple images for various techniques. Using one image is generally preferable, as retaining
multiple images requires additional maintenance.
The VM image should be updated periodically to ensure that only the latest tools and versions are being
used. During this update period, the team should confirm tool functionality and identify—with
documentation as appropriate—any changes in the functionality or use. Updating tools that discovervulnerabilities (e.g., vulnerability scanners) before each test helps ensure that recently discovered
vulnerabilities are part of the testing. In addition to maintaining their existing toolset, the team should
periodically assess its toolkit to identify obsolete tools to be removed and new tools that should be added.
Before using test systems in a security test, the test team should apply the latest security patches andenable only the services needed for connectivity and testing. This recommendation applies to all
operating systems that may be used for testing, including those in VMs. The organization’s security
group may validate that test systems are compliant with the organization’s security requirements andapproved for testing before connecting these systems to the network. Validation can be done via the same
systems used for technical tests such as vulnerability scans. Test systems may not meet all of theorganization’s security requirements because of the requirements of the tools used for testing—for
example, some security controls may interfere with tool operation because they attempt to stop scans or
attacks performed using those tools. In such cases, assessors may need to disable these security controls
when the tools are in use.
Traveling teams should maintain a flyaway kit that includes systems, images, additional tools, cables,
projectors, and other equipment that a team may need when performing testing at other locations. If anorganization uses an external test team, this team should not use the organization’s resources unlessrequired to do so. If the organization does not authorize external systems to be connected to its network,
the external test team will need to either install all required tools onto an approved client system or bringa bootable system emulation capability such as a live CD. 27 Appendix A provides examples of two live

27

A live CD is a fully functioning operating system environment that is contained on a bootable CD. This technology does not
require the user to load anything (e.g., software, drivers, etc) onto the system.

6-9

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

CD distributions. If tools are directly installed onto a client system, the test team should ensure that thetools and any files that they generate are removed from the system when testing is done.
6.5

Assessment Plan Development

An assessment plan provides structure and accountability by documenting the activities planned for anassessment, along with other related information. NIST SP 800-53A provides additional information on
assessment plans, and addresses several distinct steps that assessors should consider in developing a plan.
These steps are: (i) determining the type of security control assessment; (ii) determining the security
controls and control enhancements to be included in the assessment; (iii) selecting the appropriate
assessment procedures to be used during the assessment based on security controls and control
enhancements in the system security plan; (iv) tailoring the selected assessment procedures for the
information system impact level and organization’s operating environment; (v) developing additional
assessment procedures, if necessary, to address other security controls and control enhancements; (vi)developing a strategy to apply the extended assessment procedure; (vii) optimizing assessment procedures
to reduce duplication of effort and provide cost-effective assessment solutions; and (vi) finalizing theassessment plan and obtaining the approvals needed for its execution..
Each assessment should be addressed in an assessment plan, regardless of the scope, level of
intrusiveness, or party performing the test (i.e., internal, third party). 28 This plan provides the rules andboundaries to which assessors must adhere, and protects the organization by reducing the risk of anincident such as accidental system disruption or the inadvertent disclosure of sensitive information.
Assessment plans also protect the test team by ensuring that the organization’s management understands
and agrees to the assessment’s scope, activities, and limitations. Development of the assessment planshould be a collaborative process between the assessors and key members of the organization’s security
group.
The assessment plan should answer these basic questions: What is the scope of the assessment?
 Who is authorized to conduct the assessment?
 What are the assessment’s logistics?
 How should sensitive data be handled?
 What should occur in the event of an incident?
The assessment plan should identify which systems and networks are authorized to be examined andtested. This can be done by providing the number of systems and the IP addresses or address ranges thatthey use. The plan should also list specific systems—at a minimum by IP address and preferably also bysystem name—that are not authorized to be examined or tested. For example, if an organization’s payroll
database is deemed too mission-critical for a particular type of testing, the system name and IP address
should be included in the assessment plan’s exclusion list. If the organization does not control part or allof its network, such as having a portion of its systems housed on a third party’s network, the owner of theother network usually must also consent in writing to the assessment plan. A similar situation involves
28

In addition to an assessment plan, it may be useful to develop a shorter document (a one- or two-page memorandum) thatassessors can present to parties in the organization (e.g., users or system owners) as authorization to gain access to particularsystems. The document should describe allowable and unallowable activities, authorized and unauthorized systems, theacceptable level of cooperation to be provided by users, and a point of contact in the organization’s security group that users
can contact for more information.

6-10

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

systems that are shared by organizations, such as a system using virtual machine technology to provide
services to multiple organizations. By signing the assessment plan, all parties acknowledge and approveof the assessment.
Besides determining which systems are authorized for assessment, the assessment plan should also detailthe type and level of the testing permitted. For example, if the organization desires a vulnerability
assessment, the assessment plan should provide information on activities authorized to be performed on
the target network—such as port and service identification, vulnerability scanning, security configuration
review, and password cracking—with enough detail included to describe the type of testing, approach,and tools. For example, if password cracking will be used, the method through which the passwords will
be obtained (e.g., sniffed off the network or copied from the OS password file) should be included in theassessment plan. The plan should also explicitly state any activities that are prohibited—for example, filecreation and modification—in a way that leaves no room for interpretation. If questions regarding scopeand level of authorization arise during the course of an assessment, the assessors and the organization’s
identified point of contact should meet to discuss them.
The plan should also address the logistical details of the engagement—including the hours of operation
for assessors; the clearance or background check level required; a call plan with current contact
information, network and security operations centers, and the organization’s main point of contact for theassessment; the physical location where assessment activities will originate; and the equipment and toolsthat will be used to conduct the assessment. Any requirements to inform parent organizations, law
enforcement, and a computer incident response team (CIRT) should be identified in the assessment plan.
In addition, the person responsible for informing the organizations of the pending security assessment
should be identified. In the case of covert or other unannounced testing, the assessment plan should also
define how test activity detected and reported by the organization’s security staff, CIRT, and others
should be handled—including as the escalation processes to be followed. The primary purpose for this is
to ensure that assessment activity does not trigger reporting of security breaches to external parties, suchas external incident response teams.
IP addresses of the machines from which assessment activities will be conducted should be identified in
the assessment plan to enable administrators to differentiate assessment activities such as penetration
testing attacks from actual malicious attacks. If appropriate for the goals of the assessment, security
administrators can configure intrusion detection systems and other security monitoring devices to ignore
activity generated by these IP addresses during testing.
Data handling requirements should be addressed in the assessment plan, including: Storage of organizational data during the assessment on the assessors’ systems, including
physical security of the systems, passwords, and data encryption
 Data storage upon conclusion of the assessment, to meet long-term storage requirements orvulnerability tracking
 Transmission of data during or after the assessment across internal or external networks (e.g., theInternet)
 Removal of data from systems upon conclusion of the assessment—in particular, for third-party
assessments that include references to specific requirements set forth by the governingorganization’s policies or procedures.
Finally, the assessment plan should provide specific guidance on incident handling in the event thatassessors cause or uncover an incident during the course of the assessment. This section of the plan

6-11

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

should define the term incident and provide guidelines for determining whether or not an incident has
occurred. The plan should identify specific primary and alternate points of contact for the assessors,
generally the assessment team leader and assistant team leader, and the organization’s security group.
Guidelines should be included that clearly state actions to be taken by both the assessors and theorganization’s security group upon determination that an incident has occurred. For example, if theassessors discover an actual intruder or an intruder’s footprints within the network, should testing stop? If
so, when can testing recommence—and by whose authority? The assessment plan should provide clearcut instructions on what actions assessors should take in these situations.
Some assessments use ROE in addition to or instead of an assessment plan. The ROE contains the same
information in an assessment plan, and also addresses testing activities that are usually prohibited by theorganization. For example, some activities that are often performed during penetration testing, such as
issuing attacks to compromise systems, are usually prohibited by an organization’s policies. The ROE
provides authorization for the assessors to conduct such activities as part of the assessment process.
Appendix B provides a sample template for an ROE.
Each organization should determine when assessment plans and/or ROEs should be used. Organizations
should also consider developing central assessment plans, or ROE templates or partial drafts, andrequiring their use to promote consistency.
6.6

Legal Considerations

An evaluation of potential legal concerns for an assessment should be addressed before the assessment
begins. While the involvement of legal advisors is at the discretion of the organization, it isrecommended that they always be involved for intrusive tests such as penetration testing. If anorganization authorizes an external entity to conduct an assessment, the legal departments of eachorganization may be involved. These departments may assist in reviewing the assessment plan andproviding indemnity or limitation of liability clauses into contracts that govern security assessments—
particularly for types of tests that are deemed intrusive. The legal department may also require external
entities to sign nondisclosure agreements that prohibit assessors from disclosing any sensitive,
proprietary, or otherwise restricted information to unapproved entities.
The legal department should also address any privacy concerns that the organization may have. Most
organizations have warning banners or signed user agreements that disclose their systems are monitored,
warning that individuals consent to monitoring by their use of the system. However, not all organizations
have these in place, and the legal department should address potential privacy violations before theassessment begins. In addition, captured data may include sensitive data that does not belong to theorganization—or personal employee data, which may create privacy concerns. Assessors should beaware of these risks and conduct packet captures that follow any requirements set forth by the legal
department. The legal department may also determine data handling requirements to ensure data
confidentiality (e.g., vulnerabilities).
6.7

Summary

Information security assessment is a complex activity because of organizational requirements, the number
and type of systems within an organization, the technical techniques to be used, and the logistics
associated with assessments. Security assessments can be simplified and associated risks reduced throughan established, repeatable planning process. Accurate and timely planning of a security assessment canalso ensure that all factors necessary for assessment success are taken into account.
The core activities involved in planning for an assessment include:

6-12

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT Developing a security assessment policy. Organizations should develop an information security
assessment policy to provide direction and guidance for their security assessments. This policy
should identify security assessment requirements and hold accountable those individuals
responsible for ensuring that assessments comply with the requirements. The approved policy
should be disseminated to the appropriate staff, as well as third parties who are to conduct
assessments for the organization. The policy should be reviewed at least annually and whenever
there are new assessment-related requirements.
 Prioritizing and scheduling assessments. Organizations should decide which systems should
undergo assessments and how often these assessments should be done. This prioritization isbased on system categorization, expected benefits, scheduling requirements, applicableregulations where assessment is a requirement, and resource availability. Technicalconsiderations can also help determine assessment frequency, such as waiting until known
weaknesses are corrected or a planned upgrade to the system is performed before conducting
testing.
 Selecting and customizing technical testing and examination techniques. There are manyfactors for organizations to consider when determining which techniques should be used for aparticular assessment. Factors include the assessment objectives, the classes of techniques thatcan obtain information to support those objectives, and the appropriate techniques within eachclass. Some techniques also require the organization to determine the assessors’ viewpoint (e.g.,internal versus external) so that corresponding techniques can be selected.
 Determining the logistics of the assessment. This includes identifying all required resources,
including the assessment team; selecting environments and locations from which to perform theassessment; and acquiring and configuring all necessary technical tools.
 Developing the assessment plan. The assessment plan documents the activities planned for anassessment and other related information. A plan should be developed for every assessment toprovide the rules and boundaries to which assessors must adhere. The plan should identify thesystems and networks to be assessed, the type and level of testing permitted, logistical details of
the assessment, data handling requirements, and guidance for incident handling.
 Addressing any legal considerations. Organizations should evaluate potential legal concerns
before commencing an assessment, particularly if the assessment involves intrusive tests (e.g.,
penetration testing) or if the assessment is to be performed by an external entity. Legal
departments may review the assessment plan, address privacy concerns, and perform otherfunctions in support of assessment planning.

6-13

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

7.

Security Assessment Execution

During execution of the security assessment, vulnerabilities are identified by the methods and techniques
decided upon in the planning phase and identified in the assessment plan or ROE. It is critical that theassessment be conducted in accordance with the plan or ROE—and the purpose of this section is tohighlight key points for assessors to consider throughout the execution phase. For example, proper
coordination throughout the assessment facilitates the assessment process and reduces the possibility of
associated risks. Key considerations such as incident handling and the challenges organizations facewhen conducting assessments are also highlighted. This section also discusses the analysis process, andprovides recommendations for the collection, storage, transmission, and destruction of assessment-related
data.
7.1

Coordination

Throughout an assessment, it is critical for assessors to coordinate with various entities in theorganization. Coordination requirements are determined by the assessment plan or ROE and should be
followed accordingly. Proper coordination helps to ensure that: Stakeholders are aware of the assessment schedule, activities, and potential impacts theassessment may have
 The assessment does not take place during upgrades, new technology integration, or other times
when the system security is being altered (e.g., testing occurs during maintenance windows orperiods of low utilization)
 Assessors are provided with required levels of access to the facility and systems, as appropriate
 Appropriate personnel such as the CIO, CISO, and ISSO are informed of any critical high-impact
vulnerabilities as soon as they are discovered
 Appropriate individuals are informed (e.g., assessors, incident response team, senior
management) in the event of an incident. Should this occur, it is recommended that activities
cease until the incident is addressed and the assessors are given approval to resume their activities
in accordance with the assessment plan or ROE. The extent to which assessment activities should
be suspended varies based on the organization and the type of incident, but in many cases the
only activities suspended are those involving the systems directly involved in the incident.
The level of coordination between assessors and the organization are driven primarily by the system andthe assessment being conducted. Critical systems generally require more coordination to ensure system
availability throughout the engagement, and assessment techniques pose varying levels of risk to thetarget system during execution. Techniques that fall in the review category have minimal risk; target
identification and analysis category have moderate risk; and a high risk is associated with the target
vulnerability validation category. For instance, a critical system undergoing penetration testing generally
requires more coordination than would a document review of a critical system or a penetration test of anoncritical system. However, organizations may encounter circumstances where the reverse is true, andin such cases the level of coordination should be commensurate with requirements and organizational
considerations. Assessors and other stakeholders—such as system owners—should remain vigilant
during the execution of assessments. The level of access required by assessors will also drive
coordination to ensure they have appropriate physical and system access (e.g., when testing the insider
threat).

7-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Assessors should be proactive in their communication with the appropriate parties in the organization.
This communication can be maintained through periodic status meetings and daily or weekly reports.
Meeting attendees and report recipients should be identified in the assessment plan or ROE, and may
include the assessors, ISSO, CISO, and CIO. The frequency of status meetings and reports will be driven
by the assessment’s length and complexity. For example, for a one-month penetration test, status
meetings may be held weekly with daily reports provided during the active testing phase (i.e., the period
during which systems are being exploited). Meetings and reports should address activities completed todate, success rate, problems encountered, and critical findings/recommended remediation.
7.2

Assessing

As discussed in Section 6, the assessment plan or ROE provides guidelines for conducting the assessment.
The plan or ROE should be followed unless specific permission to deviate has been obtained, normally in
writing, from the original signatory or individual in command. It is critical that all assessors read andunderstand the plan or ROE. It is recommended that assessors periodically review the plan or ROE
during the assessment—particularly in the case of activities in the target vulnerability validation category.
During an assessment, the organization’s incident response team may detect an incident. This could becaused by the assessors’ actions—or by a real adversary that happens to perform an attack while theassessment is in progress. Regardless, the incident response team or individual discovering the incident
should follow the organization’s normal escalation procedures, and assessors should follow the guidelines
set forth by the assessment plan or ROE unless instructed otherwise. If the presence of an adversary isfound during the assessment, it should immediately be reported to the appropriate individual andassessors should follow the protocol identified in the assessment plan or ROE. It is recommended thatassessors stop assessing the systems involved in the incident while the organization carries out its
response.
In addition to encountering new incidents or uncovering existing ones, assessors may face other technical,
operational, and political challenges during an assessment. These can include: Resistance. Resistance to assessments can come from many sources within an organization,
including system and network administrators and end users. Reasons may include fear of losing
system or network availability, fear of being reprimanded, inconvenience, and resistance tochange. Obtaining upper management approval and support will help resolve problems related toresistance, and incorporating security assessments into the organization’s overall security policy
will help establish a process that does not surprise administrators and users.
 Lack of Realism. In preparing for an assessment, users and administrators sometimes modify
settings to make their systems more secure, resistant to attack, or more compliant with policies
and other requirements. While this can be viewed as positive, changes made under thesecircumstances are generally only maintained for the duration of the assessment, after which thesystems are returned to their previous configurations. Providing no advance notice ofassessments to users and administrators helps to address this challenge. Many organizations
perform occasional unannounced assessments to supplement their announced assessments.
 Immediate Mitigation. As security weaknesses are identified during an assessment,
administrators may want to take immediate steps to mitigate them and expect assessors to quickly
re-assess the system to confirm that the problems have been resolved. Although this desire forquick mitigation is admirable, assessors should communicate the importance of following theorganization’s change management policies and procedures.

7-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

 Time. Security assessment is often incorporated into development or deployment with little
notice and narrow timeframes when it should actually be made a regular part of the development
or deployment cycle. Time is also a challenge when testing critical systems and networks that are
in production—if testing techniques have the potential to cause loss of availability or otherproblems, systems and networks may need to be tested off-hours. Assessors are often restricted
to testing timeframes, while real attackers are not limited to such constraints.
 Resources. Security assessment faces the continual challenge of obtaining and maintaining
adequate resources (e.g., a skilled test team and up-to-date hardware and software). It issuggested that organizations designate security assessment equipment—such as laptops andwireless cards—to be used solely for assessments. 29 If commercial assessment software is used,
the purchase of continuous licenses and support contracts should be considered. Assessors shouldschedule time before the assessment begins to ensure that all assessment software is properly
patched and up to date. If internal assessors are not available or do not meet assessment
requirements, it may be a challenge to find dependable, trustworthy outside assessors.
Organizations should seek a firm with an established methodology, proven processes, comparable
and sufficient past performance, and experienced personnel. If an organization is using internal
assessors, it should continue to recruit and train skilled assessors and offer other challenging
opportunities within the organization where assessors can become involved to avoid burnout.
 Evolving Technology. Assessors need to stay up to date on tools and testing techniques.
Budgets should allow for annual training classes and conferences where assessors can update andrefresh their skills.
 Operational Impact. Although assessments are planned to prevent or limit operational impact,
there is always a chance of accidental or unexpected complications. Every test conducted should
be recorded with a timestamp, test type, tool used, commands, the IP address of testing
equipment, etc. It is recommended that a logging script be used to capture all commands andkeystrokes used during the testing process. Terminal and GUI tools exist that can record atester’s actions, and this type of recording can also assist in countering accusations that testing
has negatively impacted operations and system performance. Because of the risk of operational
impact, it is recommended that an established incident response plan be in place during testing.
7.3

Analysis

Although some analysis may be performed after an assessment has been completed (see Section 8.1),
most analysis occurs during the assessment itself. The primary goals in conducting analysis are to
identify false positives, categorize vulnerabilities, and determine the vulnerabilities’ causes. Automatedtools can produce a significant number of findings, but these findings often need to be validated to isolate
false positives. Assessors may validate vulnerabilities by manually examining the vulnerable system orby using a second automated tool and comparing the results. Although this can be done quickly, thesecomparison tools can often produce similar results—including the same false positives. Manual
examination typically provides more accurate results than comparing results from multiple tools, but it
also has the potential to be time-consuming.
Organizations may choose to categorize their findings according to the security controls and control
families in NIST SP 800-53, which organizes controls into families such as incident response and access
control. This categorization may facilitate vulnerability analysis, remediation, and documentation.

29

Organizations may want to disconnect their dedicated test equipment from networks when testing is not taking place.

7-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

While individual vulnerabilities need to be identified and resolved, identifying the root cause ofvulnerabilities is key to improving the organization’s overall security posture because a root cause canoften be traced to program-level weaknesses. Some common root causes include: Insufficient patch management, such as failing to apply patches in a timely fashion or failing toapply patches to all vulnerable systems
 Insufficient threat management, including outdated antivirus signatures, ineffective spam
filtering, and firewall rulesets that do not enforce the organization’s security policy
 Lack of security baselines, such as inconsistent security configuration settings on similar systems
 Poor integration of security into the system development life cycle, such as missing or unsatisfied
security requirements and vulnerabilities in organization-developed application code Security architecture weaknesses, such as security technologies not being properly integrated into
the infrastructure (e.g., poor placement, insufficient coverage, or outdated technologies), or poor
placement of systems that increases their risk of compromise Inadequate incident response procedures, such as delayed responses to penetration testing
activities
 Inadequate training, both for end users (e.g., failure to recognize social engineering and phishingattacks, deployment of rogue wireless access points) and for network and system administrators(e.g., deployment of weakly secured systems, poor security maintenance) Lack of security policies or policy enforcement (e.g., open ports, active services, unsecured
protocols, rogue hosts, weak passwords).
A useful resource to reference throughout the analysis phase is the NIST National Vulnerability Database(NVD) 30 . NVD is a database that contains information on Common Vulnerabilities and Exposures(CVE), a list of standardized names for known vulnerabilities. The NVD scores vulnerabilities with theCommon Vulnerability Scoring System (CVSS) and provides additional information regarding thevulnerability and additional resources to reference for mitigation recommendations (e.g., vendor Web
sites).
Another goal of analysis is to identify throughout the assessment any critical vulnerabilities that theorganization needs to immediately address. For instance, if penetration testing exploits a vulnerability
that allows assessors to gain administrator rights on a critical system, assessors should immediately notify
the person identified in the assessment plan or ROE.
7.4

Data Handling

The method by which an organization’s data is handled throughout the assessment is critical to ensuring
protection of sensitive information—including system architecture, security configurations, and systemvulnerabilities. Organizations should ensure proper documentation of requirements for data handling in
the assessment plan or ROE, and adhere to their governing policies regarding the handling of systemvulnerabilities. This section offers suggested methods for collecting, storing, and transmitting assessment
data during an engagement, as well as for storing and destroying data once an assessment is complete.

30

The NVD website is http://nvd.nist.gov/.

7-4

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

7.4.1

Data Collection

Relevant information should be collected by the team throughout the assessment. This includes
information related to the architecture and configuration of the networks being assessed, as well as
information on assessor activities. Because this data is sensitive, it is important to handle it appropriately.
Types of information the assessors might collect include: Architecture and Configuration Data. Assessment type and desired outcome will drive thedata collected by the team, which may include but not be limited to system names, IP addresses,
OS, physical and logical network positions, security configurations, and vulnerabilities.
 Assessor Activities. Assessors should keep a log that includes assessment system information
and a step-by-step record of their activities. This provides an audit trail, and allows theorganization to distinguish between the actions of assessors and true adversaries. The activity log
can also be useful in developing the assessment results report.
Use of a keystroke logger on an assessor’s system can create a step-by-step log of many tester actions,although it will not capture mouse clicks and certain other actions. 31 For automated tools, assessors canmaintain the audit logs from each tool that is used. While assessors may choose to dump the output of
the keystroke logger or tool audit log onto a separate system to create a centralized storage and auditing
capability, an alternate manual approach is an activities log that tracks each command executed byassessors on the network. This approach is time-consuming for the assessors, and leaves room for error.
If an activities log is used, it should include at a minimum the following information—date and time,assessor’s name, assessment system identifier (i.e., IP or MAC), target system identifier (i.e., IP or
MAC), tool used, command executed, and comments.
7.4.2

Data Storage

Secure storage of data collected during the assessment, including vulnerabilities, analysis results, andmitigation recommendations, is the assessors’ responsibility. Inappropriate release of this information
can damage the organization’s reputation and increase the likelihood of exploitation. At a minimum,
assessors should store the following information to be used for identifying, analyzing, and reporting on
the security posture of the organization, and provide an audit trail of testing activities:
 Assessment plans and ROEs
 Documentation on system security configuration and network architecture Results from automated tools and other findings
 Assessment results report Corrective action plan or Plan of Action and Milestones (POA&M).
Many options exist for storing information on discovered vulnerabilities, such as keeping the findings in
the format output by the tool that was used, or importing the findings into a database. 32 Most vulnerability
scanning tools have report formats that list the system, vulnerabilities, and recommended mitigation
31

32

A keystroke logger records every keystroke made by the user of the system, and places it into a log. This level of recording
provides assessors with a method to track each action on the network—and allows the organization being assessed to see
exactly what the assessors executed on the network, when it occurred, and which system conducted the test. In addition, thistype of recording provides assessors with documentation that they were not the cause of malfunctioning or compromise of a
network system.
Storing vulnerability information can also be helpful for performing historical comparisons.

7-5

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

techniques. This may be an acceptable approach if the assessment is small in scope (e.g., only uses onetool). For more in-depth assessments, larger organizations, or assessments that use multiple tools orapproaches, a more robust and collaborative storage method—such as a spreadsheet or database—can be
developed. Although functionality is limited, a spreadsheet may be appropriate for individual
examinations or tests, as it is easy to use, usually quick to develop, and can accommodate a number oftools that can output findings in a compatible format. For complex examinations or tests with multiple
technical approaches, assessment actions that regularly recur, or situations with a need to correlate data
easily, developing a database may be beneficial.
Organizations should ensure the secure storage of all sensitive assessment data, such as the assessment
plan or ROE, raw vulnerability data, and assessment reports. In the hands of an adversary, information
regarding network architecture, system configuration, security controls, and specific systemvulnerabilities would provide a blueprint and roadmap for exploiting the organization’s information
systems. Organizations may choose to store this data on removable media, or on an information systemthat could be accessed as needed. The removable media or system designed to store this information
should be isolated physically or logically from day-to-day network resources. Access to this system andthe information it contains should be limited to those individuals whose access is needed to fulfill roles
and responsibilities. This data is also recommended to be encrypted in compliance with FIPS 140-2 toensure that it remains secure.
Retention requirements for security assessments data vary and may not be explicitly stated for an
organization, in which case retention requirements for the assessment should be specified in theassessment plan or ROE. Maintaining accurate records for an assessment provides an organization with
an audit trail of its vulnerabilities and the remediation actions it has taken to mitigate identified risks. An
audit trail maintained over time may allow organizations to evaluate the effectiveness of their information
security program by conducting trend analyses of metrics involving vulnerability type, frequency ofoccurrence, mean time to remediation, etc.
Assessment systems—such as servers, laptops, or other mobile devices—should not be left unattended
when storing sensitive data without the proper physical and logical security safeguards in place. For
example, mobile systems should not be left in unlocked vehicles or in plain sight in locked vehicles, andmobile devices in hotel rooms should be secured by a cable lock, stored in a room safe, or physicallysecured by other means. In addition to these physical safeguards, assessors should ensure that the system
is configured in a way that deters adversaries from compromising it. Assessors should take appropriate
measures to ensure the integrity and confidentiality of data a system contains, and protect the system at aminimum with a strong password—and it is suggested that organizations consider using two-factor
authentication. 33 In addition, all sensitive data on the system should be encrypted, 34 and an authentication
mechanism separate from the system authentication should be used to restrict access to the encrypted
information.
7.4.3

Data Transmission

It may be necessary to transmit assessment data, such as system configurations and vulnerabilities, overthe network or Internet, and it is important to ensure the security of the data being transmitted to protect itfrom compromise. The assessment plan or ROE should address the requirements of, and process for,transmitting sensitive system information across the network or Internet. Secure data transmission
methods include encrypting individual files containing sensitive information, encrypting communication
33

34

Two-factor authentication provides additional security by requiring two of the following three factors—something you know
(e.g., password), something you have (e.g., security token), and something you are (e.g., retinal scan).
Such data should be encrypted in compliance with FIPS 140-2 to ensure that it remains secure.

7-6

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

channels using FIPS-compliant encryption (e.g., VPNs, Secure Sockets Layer [SSL] protocol), andproviding information through delivered or mailed hard or soft copies.
7.4.4

Data Destruction

When assessment data is no longer needed, the assessment systems, hard copy documentation, and media
should be appropriately sanitized. NIST SP 800-88, Guidelines for Media Sanitization 35 divides media
sanitization into four categories:
 Disposal: the act of discarding media with no other sanitization considerations. This is mostoften done by recycling paper that contains nonconfidential information, but may also include
other media.
 Clearing: a level of media sanitization that would protect information confidentiality against arobust keyboard attack. Simple deletion of items does not suffice for clearing. Clearing must
keep information from being retrieved by data, disk, or file recovery utilities, and must be
resistant to keystroke recovery attempts executed from standard input devices and data
scavenging tools. Overwriting is an example of an acceptable method for clearing media.
 Purging: a media sanitization process that protects information confidentiality against alaboratory attack. 36 For some media, clearing media does not suffice for purging. Examples ofalternatives to clearing media are executing the firmware Secure Erase command (for Advanced
Technology Attachment [ATA] drives only) and degaussing 37 .
 Destruction: physical obliteration of media to render it no longer usable for its intended purposeand making the data it contains no longer retrievable. Physical destruction is possible through avariety of methods, including disintegration, incineration, pulverizing, shredding, and melting.
Organizations should maintain a policy on their sanitization requirements for assessment systems. NIST
SP 800-88 presents a decision-flow diagram to assist organizations in determining which sanitization
method is most applicable for their circumstances. An assessment plan or ROE may also specify
destruction requirements for particular tests.
Third-party assessors should ensure that they understand the organization’s requirements for sanitization,
as policy may differ between organizations and possibly among divisions within the same organization.
For example, some organizations prohibit third-party assessors from having any access to assessment data
once their final reports have been submitted. In such cases, a qualified individual from the organization
being assessed should verify that appropriate sanitization measures have been carried out.

35
36

37

NIST SP 800-88 is available at http://csrc.nist.gov/publications/PubsSPs.html.
A laboratory attack would involve an attacker with the resources and knowledge to use nonstandard systems to conduct data
recovery attempts on media outside the normal operating environment. This type of attack involves using signal processing
equipment and specially trained personnel.
Degaussing is exposing the magnetic media to a strong magnetic field to disrupt the recorded magnetic domains.

7-7

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

8.

Post-Testing Activities

Following the execution phase—whose findings are expressed in terms of vulnerabilities—theorganization should take steps to address the vulnerabilities that have been identified. This section
presents ways that organizations can translate their findings into actions that will improve security. First,
final analysis of the findings should be performed, and mitigation actions developed. Second, a report
should be developed to present the recommendations. Lastly, the mitigation activities should be carriedout. Many of the actions presented in this section may occur outside of the testing process itself—for
example, as part of a risk assessment that utilizes testing results.
8.1

Mitigation Recommendations

As described in Section 7.3, most analysis occurs during the testing process. Final analysis, such as thedevelopment of overall conclusions, usually takes place after all testing activities have been completed
and involves the development of mitigation recommendations. While identifying and categorizing
vulnerabilities is important, a security test is much more valuable if it also results in a mitigation strategy
being developed and implemented. Mitigation recommendations, including the outcome of the root cause
analysis, should be developed for each finding. There may be both technical recommendations (e.g.,
applying a particular patch) and nontechnical recommendations that address the organization’s processes(e.g., updating the patch management process). Examples of mitigation actions include policy, process,and procedure modifications; security architecture changes; deployment of new security technologies; anddeployment of OS and application patches.
NIST SP 800-53 suggests mitigation recommendations for each security control. Organizations should
compare potential mitigation actions against operational requirements to determine the actions that bestbalance functionality and security. Section 8.3 discusses the implementation of mitigation
recommendations.
8.2

ReportingUpon completion of analysis, a report should be generated that identifies system, network, andorganizational vulnerabilities and their recommended mitigation actions. Security testing results can beused in the following ways:
 As a reference point for corrective action
 In defining mitigation activities to address identified vulnerabilities As a benchmark for tracking an organization’s progress in meeting security requirements
 To assess the implementation status of system security requirements
 To conduct cost/benefit analysis for improvements to system security
 To enhance other life cycle activities, such as risk assessments, C&A, and process improvement
efforts
 To meet reporting requirements, such as those of FISMA.
Security testing results should be documented and made available to the appropriate staff, which may
include the CIO, CISO, and ISSO as well as appropriate program managers or system owners. Because areport may have multiple audiences, multiple report formats may be required to ensure that all areappropriately addressed. For example, organizations developing reports for FISMA compliance need to

8-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

address FISMA requirements such as reporting on findings from evaluations, compliance with NIST
standards, significant deficiencies, and planned remediation activities. Reports that will remain within theorganization can be tailored for the appropriate audiences, such as program management, information
management, security engineers, configuration management, or technical staff. Internal reports should
include test methodology, test results, analysis, and POA&M. 38 A POA&M will ensure that individual
vulnerabilities are addressed with specific, measurable, attainable, realistic, and tangible actions.
8.3

Remediation/Mitigation

The POA&M provides the program management office with the details and required actions needed toappropriately and acceptably mitigate risk. As a complement to the POA&M, organizations may consider
developing a strategy or process for implementing the plan. Organizations should follow at least the four
steps outlined below during their remediation implementation process—these will provide consistency
and structure for security personnel and program managers.
The first step in the process is testing the remediation recommendation. Before implementing technicalmodifications to a production asset, testing should be done on test systems in an environment that
replicates the network in which the mitigation action would be implemented. For example, before being
pushed to the enterprise, patches should be installed on comparable systems in the test environment todetermine if there are any negative implications. Such testing significantly reduces, but does not
eliminate, the risk of a system reacting adversely to a technical modification.
Second, the POA&M should be coordinated through an organization’s configuration control or
configuration management board because the POA&M likely proposes changes to existing systems,
networks, policy, or processes. Communicating POA&M changes both before deployment and uponcompletion ensures that the appropriate individuals are aware of the pending changes and their impact onenvironment, mission, and operations. At a minimum, the program manager or system owner should becontacted before executing any POA&M actions and should provide approval of the planned mitigation
actions before they are implemented.
Obtaining management approval can be challenging. It may be beneficial to identify why it is needed(i.e., whether it is driven by policy or technology) and the positive impact that will be realized with the
mitigation action (i.e., increased security posture or compliance). A cost/benefit analysis may also
provide managers with a quantitative analysis of the increased savings to be realized by implementing the
POA&M items. Additional benefits that may be communicated to senior management include decreased
exposure, increased control of assets, decreased vulnerabilities, a proactive approach to security, andmaintenance of compliance.
Third, mitigation actions are implemented and verified to ensure their appropriate and accurate
implementation. Verification can take place by conducting an audit of the system, retesting the system
and its components, and holding personnel accountable through documentation. A system audit provides
technical verification of the changes that have been implemented on the system, and can be conducted byonsite security personnel or an external security test team. The audit team may use the mitigation strategy
as a checklist for ensuring that each action is accomplished—also, retesting the system will validate thatthe mitigation actions have been completed. It is important to note that the test team will be able to verify
its implementation only if a mirror copy of the original test is performed. As technology evolves,
38

NIST SP 800-37 notes that a POA&M “describes the measures that have been implemented or planned: (i) to correct any
deficiencies noted during the assessment of the security controls; and (ii) to reduce or eliminate known vulnerabilities in theinformation system. The plan of actions and milestones document identifies: (i) the tasks needing to be accomplished; (ii)the resources required to accomplish the elements of the plan; (iii) any milestones in meeting the tasks; and (iv) scheduled
completion dates for the milestones.”

8-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

additional vulnerabilities may be uncovered during follow-up security tests. An organization may alsochoose to verify implementation of the mitigation strategy through nontechnical means such as
documentation. For example, it may be appropriate and cost-effective to hold the security personnel
responsible for implementing the mitigation strategy accountable by requesting that they sign a document
describing all of the accomplished actions. While this method is more cost-effective in the short term foran organization, there are risks posed by not technically verifying that changes have been implemented.
Last, as part of the implementation strategy, it is important to continuously update POA&Ms to identify
activities that have been accomplished, partially accomplished, or are pending action by anotherindividual or system. Ensuring that the POA&M is integrated into the organization’s configuration
management process will facilitate centralized tracking and management of changes to systems, policies,
processes, and procedures, as well as provide an oversight mechanism that will address compliancerequirements.

8-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix A—Live CD Distributions for Security Testing
Live distribution CDs focused on security testing are available to the public at no charge, and provide
security testers with a live distribution OS that contains tools for security testing. 39 The OS distribution isloaded onto a CD-ROM, Universal Serial Bus (USB) drive, or other peripheral device. It is not installed
onto a system, but is run directly from the device on which it is loaded—hence its designation as a “live”
distribution. Two such distributions are BackTrack and Knoppix Security Tool Distribution (STD).
BackTrack 40 features a collection of over 300 security tools for network discovery, scanning and sniffing,
password cracking, remote access testing, Bluetooth testing, computer forensics, and penetration testing.
It offers user modularity, meaning that the user can customize distribution to include personal scripts oradditional tools. BackTrack also includes tools to analyze Voice over Internet (VoIP) protocols such as
the Session Initiation Protocol (SIP); tools such as Cisco Global Exploiter (CGE) and Cisco Torch thatspecifically target Cisco systems; and Metasploit, a vulnerability assessment tool. Recognizing thegrowing importance of application security testing, it also includes tools such as Peach, Fuzzer, and theJava tool, Paros Proxy. Table A-1 provides a sample of the tools available in BackTrack. 41
Table A-1. BackTrack Toolkit Sample
Security Testing Technique

Security Testing Tool

Review
Network Sniffing

Dsniff, Ettercap, Kismet, Mailsnarf, Msgsnarf, Ntop, Phoss, SinFP, SMB Sniffer,and Wireshark

File Integrity Checking

Autopsy, Foremost, RootkitHunter, and Sleuthkit

Target Identification and Analysis
Application Security Testing

CIRT Fuzzer, Fuzzer 1.2, NetSed, Paros Proxy, and Peach

Network Discovery

Autonomous System Scanner, Ettercap, Firewalk, Netdiscover, Netenum,
Netmask, Nmap, P0f, Tctrace, and Umit

Network Port and Service
Identification

Amap, AutoScan, Netdiscover, Nmap, P0f, Umit, and UnicornScan

Vulnerability Scanning

Firewalk, GFI LANguard, Hydra, Metasploit, Nmap, Paros Proxy, Snort, andSuperScan

Wireless Scanning

Airsnarf, Airsnort, BdAddr, Bluesnarfer, Btscanner, FakeAP, GFI LANguard,
Kismet, and WifiTAP

Target Vulnerability Validation
Password Cracking

Hydra, John the Ripper, RainbowCrack, Rcrack, SIPcrack, SIPdump, TFTPBrute, THC PPTP, VNCrack, and WebCrack

Remote Access Testing

IKEProbe, IKE-Scan, PSK-Crack, and VNC_bypauth

Penetration Testing

Driftnet, Dsniff, Ettercap, Kismet, Metasploit, Nmap, Ntop, SinFP, SMB Sniffer,and Wireshark

39

40

41

Such toolkits do not necessarily include all the tools that would be needed for a particular test—in many cases, toolkits willneed to be supplemented with additional tools.
BackTrack is derived from two separate Linux live security-based distributions, WHAX and the Auditor Security
Collection. Both were popular for their abundance of security tools and ease of use. Shortly after the creators ofeach distribution began to collaborate, they released the first non-beta version, renamed BackTrack, in May 2006.
BackTrack quickly became and remains a favorite toolset among security professionals. BackTrack 3.0 is the version
referenced for this publication.
Many of the tools listed in Tables A-1 and A-2 could be listed for additional techniques, but for brevity they are not.

A-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

An older Linux live OS distribution and open source security toolset is Knoppix STD, which is based on
Knoppix Linux. It was created by a security professional to assist with teaching security techniques toothers. Knoppix STD was first released in May 2004 as Knoppix-STD 0.1 and has not been updated
since. The lack of a newer version is due to its creator leaving the project. Version 0.1 is the version
referred to for this publication. Before BackTrack, Knoppix STD was the benchmark security toolset andit remains widely used.
Similar to BackTrack, Knoppix STD enables network discovery, port and service identification, network
sniffing, password cracking, forensics, and remote access testing. While there is some overlap between
the distributions, there are some differences as well. Knoppix contains some tools that BackTrack does
not, such as Netcat and Nessus; addresses technology areas such as cryptography; and offers more tools
for computer forensics and sniffing. It does not provide Metasploit, and compared to BackTrack is weakon wireless security tools. Table A-2 provides a sample of the tools available on the Knoppix STD
distribution.
Table A-2. Knoppix STD Toolkit Sample
Security Testing Technique

Security Testing Tool

Review
Network Sniffing

Dsniff, Ettercap, Ethereal, Filesnarf, Kismet, Mailsnarf, Msgsnarf,
Ngrep, Ntop, TCPdump, and Webspy

File Integrity Checking

Autopsy, Biew, Bsed, Coreography, Foremost, Hashdig, Rifiuti, andSleuthkit

Target Identification and Analysis
Application Security Testing

NetSedNetwork Discovery

Cryptcat, Ettercap, Firewalk, Netcat, Nmap, and P0f

Network Port and Service Identification

Amap, Netcat, Nmap, and P0f

Vulnerability Scanning

Exodus, Firewalk, Nmap, and Snort

Wireless Scanning

Airsnarf, Airsnort, GPSdrive, Kismet, and MACchanger

Target Vulnerability Validation
Password Cracking

Allwords2, chntpw, Cisilia, Djohn, Hydra, John the Ripper, andRcrackRemote Access Testing

Apache Server, IKE-Scan, Net-SNMP, SSHD, TFTPD, and VNC
ServerPenetration Testing

Driftnet, Dsniff, Ethereal, Ettercap, Kismet, Nessus, Netcat, Ngrep,
Nmap, Ntop, and TCPdump

A-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix B—Rules of Engagement Template
This template provides organizations with a starting point for developing their ROE. 42 Individualorganizations may find it necessary to include information to supplement what is outlined here.
1.

Introduction

1.1.

PurposeIdentifies the purpose of the document as well as the organization being tested, the group conducting thetesting (or, if an external entity, the organization engaged to conduct the testing), and the purpose of the
security test.
1.2.

Scope

Identifies test boundaries in terms of actions and expected outcomes.
1.3.

Assumptions and Limitations

Identifies any assumptions made by the organization and the test team. These may relate to any aspect of
the test to include the test team, installation of appropriate safeguards for test systems, etc.
1.4.

Risks

Inherent risks exist when conducting information security tests—particularly in the case of intrusive tests.
This section should identify these risks, as well as mitigation techniques and actions to be employed by
the test team to reduce them.
1.5.

Document Structure

Outlines the ROE’s structure, and describes the content of each section.
2.

Logistics

2.1.

Personnel

Identifies by name all personnel assigned to the security testing task, as well as key personnel from theorganization being tested. Should include a table with all points of contact for the test team, appropriate
management personnel, and the incident response team. If applicable, security clearances or comparable
background check details should also be provided.
2.2.

Test Schedule

Details the schedule of testing, and includes information such as critical tests and milestones. Thissection should also address hours during which the testing will take place—for example, it may be
prudent to conduct technical testing of an operational site during evening hours rather than during peakbusiness periods.

42

The structure of this template is intended to be illustrative. Organizations should organize their ROEs in whatever manner
they choose.

B-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

2.3.

Test Site

Identifies the location or locations from which testing is authorized. If testing will occur on theorganization’s site, building and equipment access should be discussed. Physical access should coverrequirements such as badges, escorts, and security personnel that the testers may encounter. Equipment
access should address areas such as level of access (user or administrator) to the systems and/or network,and physical access to computer rooms or specific racks that these rooms contain. Areas to which the test
team will not be given access should be identified here as well.
If testing will be conducted from a remote location such as a rented server farm or test lab, details of thetest site architecture should be included in this section.
2.4.

Test Equipment

Identifies equipment that the test team will use to conduct the information security tests. This section
should also identify the method of differentiating between the organization’s systems and the systems
conducting the testing—for example, if the test team’s systems are identified by MAC, keeping track oftest systems could be handled through use of network discovery software. In addition to hardware, tools
authorized for use on the network should be identified. It would also be appropriate to include a write-upof each tool in an appendix.
3.

Communication Strategy

3.1.

General Communication

Discusses frequency and methods of communication. For example, identify meeting schedule, locations,and conference call information if appropriate.
3.2.

Incident Handling and Response

This section is critical in the event that an incident occurs on the network while testing is in progress.
Criteria for halting the information security testing should be provided, as should details on the test
team’s course of action in the event that a test procedure negatively impacts the network or an adversary
attacks the organization while testing is underway. The organization’s incident response call tree/chain ofcommand should be provided in a quick-reference format. A process for reinstating the test team andresuming testing should also be provided.
4.

Target System/Network

Identifies the systems and/or networks to be tested throughout the information security testing process.
Information should include authorized and unauthorized IP addresses or other distinguishing identifiers, if
appropriate, for the systems (servers, workstations, firewalls, routers, etc.), operating systems, and any
applications to be tested. It is also crucial to identify any system not authorized for testing—this isreferred to as the “exclude list.”
5.

Testing Execution

This section is specific to test type and scope, but should detail allowable and unallowable activities andinclude a description of the information security testing methodology. If necessary, an assessment planshould be developed that complements the ROE—this could be either an appendix or a separate
document.

B-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

5.1.

Nontechnical Test Components

Identifies nontechnical test activities that will take place, and includes information to help identify thetypes of policies, procedures, and other documents that should be reviewed. If interviews or site surveys
are to be conducted, guidelines should be established for advance approval of the interview list andquestions. If physical security of information systems is in the scope of the testing, procedures should bedetermined and a form—with appropriate signatures and contact information—generated for the test teamto show to law enforcement or onsite security personnel in the event that they are questioned.
5.2.

Technical Test Components

Includes the type of technical testing to be conducted (e.g., network scanning, discovery, penetration
testing); discusses whether files are authorized to be installed, created, modified, and/or executed tofacilitate testing; and explains the required actions for those files once testing is completed. Any
additional information regarding the technical testing of the organization’s systems and networks should
also be included in this section. Significant detail should be included on what activities will occur on thetarget network to ensure that all parties are aware of what is authorized and to be expected as a result of
the testing.
5.3.

Data Handling

Identifies guidelines for gathering, storing, transmitting, and destroying test data, and establishes detailed,unambiguous requirements for data handling. Keep in mind that data results from any type of
information security test will identify vulnerabilities that an adversary can exploit, and should beconsidered sensitive.
6.

Reporting

Details reporting requirements and the report deliverables expected to be provided throughout the testingprocess and at its conclusion. Minimum information to be provided in each report (e.g., vulnerabilities
and recommended mitigation techniques) and the frequency with which the reports will be delivered (e.g.,
daily status reports) should be included. A template may be provided as an appendix to the ROE todemonstrate report format and content.
7.

Signature Page

Designed to identify accountable parties and ensure that they know and understand their responsibilities
throughout the testing process. At a minimum, the test team leader and the organization’s senior
management (CSO, CISO, CIO, etc.) should sign the ROE stating that they understand the test’s scopeand boundaries.

B-3

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix C—Application Security Testing and Examination
Application security testing and examination help an organization determine whether its custom
application software—for example, Web applications—contains vulnerabilities that can be exploited, andwhether the software behaves and interacts securely with its users, other applications (such as databases),and its execution environment. Application security can be assessed in a number of ways, ranging fromsource code review to penetration testing of the implemented application. 43 Many application security
tests subject the application to known attack patterns typical for that application’s type. These patterns
may directly target the application itself, or may attempt to attack indirectly by targeting the execution
environment or security infrastructure. Examples of attack patterns are information leakage (e.g.,reconnaissance, exposure of sensitive information), authentication exploits, session management exploits,
subversion (e.g., spoofing, impersonation, command injections), and denial of service attacks.
Application security assessment should be integrated into the software development life cycle of theapplication to ensure that it is performed throughout the life cycle. For example, code reviews can be
performed as code is being implemented, rather than waiting until the entire application is ready fortesting. Tests should also be performed periodically once an application has gone into production; whensignificant patches, updates, or other modifications are made; or when significant changes occur in thethreat environment where the application operates.
Many application security testing and examination techniques are available. They can be divided into
white box techniques, which involve direct analysis of the application’s source code, and black box
techniques, which are performed against the application’s binary executable without source codeknowledge. 44 Most assessments of custom applications are performed with white box techniques, since
source code is usually available—however, these techniques cannot detect security defects in interfaces
between components, nor can they identify security problems caused during compilation, linking, or
installation-time configuration of the application. White box techniques still tend to be more efficient andcost-effective for finding security defects in custom applications than black box techniques. Black box
techniques should be used primarily to assess the security of individual high-risk compiled components;
interactions between components; and interactions between the entire application or application systemwith its users, other systems, and the external environment. Black box techniques should also be used todetermine how effectively an application or application system can handle threats. Many tests use bothwhite box and black box techniques—this combination is known as gray box testing.
Assessors performing application security assessments should have a certain baseline skill set.
Guidelines for the minimum skill set include knowledge of specific programming languages andprotocols; knowledge of application development and secure coding practices; understanding of thevulnerabilities introduced by poor coding practices; the ability to use automated software code review andother application security test tools; and knowledge of common application vulnerabilities.

43

44

Some elements of application security testing, such as penetration testing an application, are target vulnerability validation
techniques, not target identification and analysis techniques. Application security testing is discussed only in this section forbrevity.
Some applications, such as many web applications, do not have compiled (binary) executables, so black box techniques may
not be applicable to analyzing their code.

C-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Application security continues to grow in importance as attackers increasingly focus on application-layer
attacks. Because application security assessment is a complex topic with dozens of commonly usedtechniques, it is outside the scope of this publication to provide specific information on techniques orrecommendations for their use. 45 Appendix E provides references with additional information.

45

In the future, NIST may release a separate publication on application security testing and examination.

C-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix D—Remote Access Testing
Remote access testing assesses remote access methods for vulnerabilities, and covers technologies such as
terminal servers, VPNs, secure shell (SSH) tunnels, remote desktop applications, and dial-up modems.
This testing is intended to discover alternative methods of entry into the network that circumvent
perimeter defenses. Remote access testing is often conducted as part of penetration testing, but can also
be performed separately to better focus on remote access implementations. Testing techniques vary
according to the type of remote access being tested and the specific goals of the test. Examples ofcommonly used techniques include: Discovering unauthorized remote access services. Port scanning may be used to locate open
ports that are often associated with remote access services. Systems may be manually checked
for remote access services by viewing running processes and installed applications.
 Reviewing rulesets to find unintended remote access paths. Remote access rulesets, such as
those on VPN gateways, should be reviewed for holes or misconfigurations that could permit
unwanted access.
 Testing remote access authentication mechanisms. Since remote access methods normallyrequire authentication, testers should first verify that they are required to authenticate before theyattempt to gain access. Testers can try default accounts and passwords (e.g., guest accounts,
maintenance accounts) and brute-force attacks—and social engineering can also be used toattempt to get a password reset or to gain access without an authentication token (e.g., byclaiming the token is lost). Testers can also attempt to gain access through self-service
authentication programs that allow passwords to be reset by answering user-specific questions—this also may involve social engineering.
 Monitoring remote access communications. Testers can monitor remote access
communications with a network sniffer. If communications are not protected, testers may be able
to use them as sources for remote access authentication information and other data sent andreceived by remote access users.
Active or intrusive remote access testing should be performed during times of low demand to limit
potential disruption to employees and the remote access systems.
Another aspect of remote access testing is assessing an organization’s phone systems for vulnerabilities
that permit unauthorized or unsecured access. NIST SP 800-24, PBX Vulnerability Analysis 46 , provides
information on elements and approaches to private branch exchange (PBX) security testing. In the area of
remote access, the primary target of phone system testing is modems—and although their use has
decreased due to the wide availability of wired and wireless network access, successful attacks continue
to be launched through unauthorized modems. For example, there are users who still use modems on
their work computers for remote access, and some organizations use older technologies—such as building
operations controllers and switches—that have maintenance modems enabled. A single compromise via amodem could allow an attacker direct, undetected access to a network that avoids perimeter security.
Several available software packages allow network administrators—and attackers—to dial large blocks oftelephone numbers to search for available modems. This process is called war dialing. A computer withfour modems can dial 10,000 numbers in a matter of days. War dialers provide reports on numbers withmodems, and some dialers have the capacity to attempt limited automatic attacks when a modem is
discovered. Organizations should conduct war dialing at least once per year to identify their unauthorized46

See http://csrc.nist.gov/publications/PubsSPs.html for additional information on PBX security.

D-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

modems, with testing conducted after normal business hours to limit potential disruption to employees
and the organization’s phone system. (It should be considered, however, that many unauthorized modems
may be turned off after hours and might go undetected.) War dialing may also be used to detect fax
equipment. Testing should include all numbers that belong to an organization, except those that could be
impacted by receiving a large number of calls (e.g., 24-hour operation centers and emergency numbers). 47
Skills needed to conduct remote access testing include TCP/IP and networking knowledge; knowledge of
remote access technologies and protocols; knowledge of authentication and access control methods;
general knowledge of telecommunications systems and modem/PBX operations; and the ability to use
scanning and security testing tools such as war dialers.

47

Most types of war dialing software allow testers to exempt specific numbers from the calling list.

D-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix E—Resources
This appendix lists a wide range of additional resources for use with technical security testing andexamination. Table E-1 contains a list of NIST documents that complement this guide, and Table E-2
provides a list of online resources that organizations may reference for additional information.

Table E-1. Related NIST Documents 48
NIST Document

URL

SP 800-30, Risk Management Guide forInformation Technology Systems

http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf

SP 800-40 Version 2.0, Creating a
Patch and Vulnerability Management
Program

http://csrc.nist.gov/publications/nistpubs/800-40-Ver2/SP800-40v2.pdf

SP 800-53 Revision 2, Recommended
Security Controls for Federal
Information Systems

http://csrc.nist.gov/publications/nistpubs/800-53-Rev2/sp800-53-rev2final.pdf

SP 800-53A, Guide for Assessing the
Security Controls in Federal Information
Systems

http://csrc.nist.gov/publications/nistpubs/800-53A/SP800-53A-finalsz.pdf

SP 800-64 Revision 1, Security
Considerations in the Information
System Development Life Cycle

http://csrc.nist.gov/publications/nistpubs/800-64/NIST-SP800-64.pdf

SP 800-84, Guide to Test, Training, andExercise Programs for IT Plans andCapabilities

http://csrc.nist.gov/publications/nistpubs/800-84/SP800-84.pdf

SP 800-92, Guide to Computer Security
Log Management

http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf

SP 800-94, Guide to Intrusion Detection
and Prevention Systems (IDPS)

http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

Table E-2. Online Resources
Resource

URL

Methodologies
Information Design Assurance Red Team (IDART)

http://www.idart.sandia.gov/

NIST SP 800-53A, Guide for Assessing the Security
Controls in Federal Information Systems

http://csrc.nist.gov/publications/PubsSPs.html

National Security Agency (NSA) Information
Assessment Methodology (IAM)

http://www.nsa.gov/ia/industry/education/iam.cfm?Menu
ID=10.2.4.2

Open Source Security Testing Methodology Manual(OSSTMM)http://www.isecom.org/osstmm/

Open Web Application Security Project (OWASP)
Testing Project

http://www.owasp.org/index.php/Category:OWASP_Tes
ting_Project

Tools
BackTrack (Linux live distribution)48

http://www.remote-exploit.org/backtrack.html

The base URL for all the NIST SPs is http://csrc.nist.gov/publications/PubsSPs.html.

E-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Resource

URL

Extra – Knoppix (Linux live distribution)

http://www.knopper.net/knoppix-mirrors/index-en.html

F.I.R.E. (Linux live distribution)

http://fire.dmzs.com/

Helix (Linux live distribution)

http://www.e-fense.com/helix/

INSERT Rescue Security Toolkit (Linux live distribution)

http://www.inside-security.de/insert_en.html

Knoppix Security Tools Distribution (STD) (Linux live
distribution)

http://s-t-d.org/download.html

nUbuntu (Linux live distribution)

http://www.nubuntu.org/downloads.php

Operator (Linux live distribution)

http://www.ussysadmin.com/operator/

PHLAK (Linux live distribution)

http://sourceforge.net/projects/phlakproject/

Top 100 Network Security Tools

http://sectools.org/

Vulnerability Information
Common Configuration Enumeration (CCE)

http://cce.mitre.org/

Common Vulnerabilities and Exposures (CVE)

http://cve.mitre.org/

Common Weakness Enumeration (CWE)

http://cwe.mitre.org/

Default Password List

[Outdated link removed]

French Security Incident Response Team (FrSIRT)

http://www.frsirt.com/english/

iDefense Lab’s Public Advisories List

http://labs.idefense.com/intelligence/vulnerabilities/

milw0rm

http://www.milw0rm.com/

National Vulnerability Database (NVD)

http://nvd.nist.gov/

Neohapsis Archives

http://archives.neohapsis.com/

Open Source Vulnerability Database

http://www.osvdb.org/

Open Web Application Security Project (OWASP)
Vulnerabilities

http://www.owasp.org/index.php/Category:Vulnerability

Secunia Advisories

http://secunia.com/advisories/

SecurityFocus Vulnerabilities

http://www.securityfocus.com/vulnerabilities

SecurityTracker

http://www.securitytracker.com/

Secwatch’s Vulnerability Archive

http://secwatch.org/advisories/

The Hacker’s Choice (THC)

http://freeworld.thc.org/

United States Computer Emergency Readiness Team(US-CERT) Vulnerability Notes Database

http://www.kb.cert.org/vuls

Wireless Vulnerabilities and Exploits (WVE)

http://www.wirelessve.org/

E-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix F—Glossary
Selected terms used in the publication are defined below.
Active Security Testing: Security testing that involves direct interaction with a target, such as sending
packets to a target.
Banner Grabbing: The process of capturing banner information—such as application type and version—that is transmitted by a remote port when a connection is initiated.
Covert Testing: Testing performed using covert methods and without the knowledge of theorganization’s IT staff, but with full knowledge and permission of upper management.
External Security Testing: Security testing conducted from outside the organization’s security
perimeter.
False Positive: An alert that incorrectly indicates that a vulnerability is present.
File Integrity Checking: Software that generates, stores, and compares message digests for files to detectchanges made to the files.
Information Security Testing: The process of validating the effective implementation of security
controls for information systems and networks, based on the organization’s security requirements.
Internal Security Testing: Security testing conducted from inside the organization’s security perimeter.
Network Discovery: The process of discovering active and responding hosts on a network, identifying
weaknesses, and learning how the network operates.
Network Sniffing: A passive technique that monitors network communication, decodes protocols, andexamines headers and payloads for information of interest. It is both a review technique and a target
identification and analysis technique.
Operating System (OS) Fingerprinting: Analyzing characteristics of packets sent by a target, such as
packet headers or listening ports, to identify the operating system in use on the target.
Overt Testing: Security testing performed with the knowledge and consent of the organization’s IT staff.
Passive Security Testing: Security testing that does not involve any direct interaction with the targets,
such as sending packets to a target.
Password Cracking: The process of recovering secret passwords stored in a computer system ortransmitted over a network.
Penetration Testing: Security testing in which evaluators mimic real-world attacks in an attempt toidentify ways to circumvent the security features of an application, system, or network. Penetration
testing often involves issuing real attacks on real systems and data, using the same tools and techniques
used by actual attackers. Most penetration tests involve looking for combinations of vulnerabilities on asingle system or multiple systems that can be used to gain more access than could be achieved through asingle vulnerability.

F-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Phishing: A digital form of social engineering that uses authentic-looking—but bogus—e-mails torequest information from users or direct them to a fake Web site that requests information.
Plan of Actions and Milestones (POA&M): A document that identifies tasks needing to beaccomplished. It details resources required to accomplish the elements of the plan, any milestones formeeting the tasks, and scheduled milestone completion dates.
Port Scanner: A program that can remotely determine which ports on a system are open (e.g., whether
systems allow connections through those ports).
Review Techniques: Passive information security testing techniques, generally conducted manually, thatare used to evaluate systems, applications, networks, policies, and procedures to discover vulnerabilities.
They include documentation, log, ruleset, and system configuration review; network sniffing; and file
integrity checking.
Rogue Device: An unauthorized node on a network.
Rules of Engagement (ROE): Detailed guidelines and constraints regarding the execution of information
security testing. The ROE is established before the start of a security test, and gives the test teamauthority to conduct defined activities without the need for additional permissions.
Ruleset: A collection of rules or signatures that network traffic or system activity is compared against to
determine an action to take—such as forwarding or rejecting a packet, creating an alert, or allowing asystem event.
Social Engineering: The process of attempting to trick someone into revealing information (e.g., apassword).
Target Identification and Analysis Techniques: Information security testing techniques, mostly active
and generally conducted using automated tools, that are used to identify systems, ports, services, andpotential vulnerabilities. Target identification and analysis techniques include network discovery,
network port and service identification, vulnerability scanning, wireless scanning, and application security
testing.
Target Vulnerability Validation Techniques: Active information security testing techniques that
corroborate the existence of vulnerabilities. They include password cracking, remote access testing,penetration testing, social engineering, and physical security testing.
Version Scanning: The process of identifying the service application and application version currently
in use.
Virtual Machine (VM): Software that allows a single host to run one or more guest operating systems.
Vulnerability: Weakness in an information system, or in system security procedures, internal controls, or
implementation, that could be exploited or triggered by a threat source.
Vulnerability Scanning: A technique used to identify hosts/host attributes and associated vulnerabilities.

F-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

Appendix G—Acronyms and Abbreviations
Selected acronyms and abbreviations used in this publication are defined below.
ARP
ATA

Address Resolution Protocol
Advanced Technology Attachment

C&A
CCE
CGE
CIO
CIRT
CISO
CTO
CVE
CVSS
CWE

Certification and Accreditation
Common Configuration Enumeration
Cisco Global Exploiter
Chief Information Officer
Computer Incident Response Team
Chief Information Security Officer
Chief Technology Officer
Common Vulnerabilities and Exposures
Common Vulnerability Scoring System
Common Weakness Enumeration

DNS
DoS
DSL

Domain Name System
Denial of Service
Digital Subscriber Line

FIPS
FISMA
FrSIRT
FTP

Federal Information Processing Standards
Federal Information Security Management Act
French Security Incident Response Team
File Transfer Protocol

GOTS
GPS
GUI

Government Off-the-Shelf
Global Positioning System
Graphical User Interface

HHS
HTTP

Department of Health and Human Services
Hypertext Transfer Protocol

IAM
ICMP
IDART
IDPS
IDS
IEEE
IIS
IP
IPS
ISSO
IT
ITL

Information Assessment Methodology
Internet Control Message Protocol
Information Design Assurance Red Team
Intrusion Detection and Prevention System
Intrusion Detection System
Institute of Electrical and Electronics Engineers
Internet Information ServerInternet Protocol
Intrusion Prevention System
Information Systems Security Officer
Information Technology
Information Technology Laboratory

LAN

Local Area Network

MAC

Media Access Control

G-1

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

NAT
NIS
NIST
NSA
NVD

Network Address Translation
Network Information System
National Institute of Standards and Technology
National Security Agency
National Vulnerability Database

OMB
OS
OSSTMM
OWASP

Office of Management and Budget
Operating System
Open Source Security Testing Methodology Manual
Open Web Application Security Project

P2P
PBX
PDA
PII
PIN
POA&M
POP

Peer-to-Peer
Private Branch Exchange
Personal Digital Assistant
Personally Identifiable Information
Personal Identification Number
Plan of Action and Milestones
Post Office Protocol

RF
ROE

Radio Frequency
Rules of Engagement

SCADA
SCAP
SHA
SIP
SME
SMTP
SP
SSH
SSID
SSL
SSN
STD

Supervisory Control and Data Acquisition
Security Content Automation Protocol
Secure Hash Algorithm
Session Initiation Protocol
Subject Matter Expert
Simple Mail Transfer Protocol
Special Publication
Secure Shell
Service Set Identifier
Secure Sockets Layer
Social Security Number
Security Tool Distribution

TCP
TCP/IP
TCP/UDP
TFTP
THC

Transmission Control Protocol
Transmission Control Protocol/Internet Protocol
Transmission Control Protocol/User Datagram Protocol
Trivial File Transfer Protocol
The Hacker’s Choice

UDP
URL
US-CERT
USB

User Datagram Protocol
Uniform Resource Locator
United States Computer Emergency Readiness Team
Universal Serial Bus

VM
VoIP
VPN

Virtual Machine
Voice Over Internet Protocol
Virtual Private Network

WAN

Wide Area Network

G-2

TECHNICAL GUIDE TO INFORMATION SECURITY TESTING AND ASSESSMENT

WIDPS
WLAN
WVE

Wireless Intrusion Detection and Prevention System
Wireless Local Area Network
Wireless Vulnerabilities and Exploits

XML

Extensible Markup Language

G-3