Penetration Test Report (National Cybersecurity Assessments and Technical Services (NCATS)): PCA Sample Report 508 Compliant

XXXXX
September 1, 2018
Phishing Campaign Assessment Summary
Office of Example
Phishing Campaign Assessment Details
Customer Name OFFICE OF EXAMPLE (EXAMPLE)
Customer POC John Doe, Email@EXAMPLE.gov
NCATS Team Lead Federal Lead, Email@hq.dhs.gov
Dates June 4, 2018 to July 13, 2018
Test Location DHS/NCATS Lab
Scope 1000 users within the following domain:
@EXAMPLE.gov
Services Phishing Campaign Assessment
NCATS ID XXXXX
Revision History
Version Date Author Notes
1.0 06/04/2018 Federal Lead, DHS First draft
1.1 07/02/2018 John Doe Agency Review and Comment
2.0 07/16/2018 Federal Lead, DHS Final Report
2
Table of Contents
Executive Report ........................................................................................................................... 4
EXAMPLE Methodology Specifics .............................................................................................. 5
Summary of Testing Activities and Results ............................................................................... 5
Click Rate vs. Report Rate ....................................................................................................... 6
Click Time vs. Report Time ...................................................................................................... 8
Open-Source Intelligence Gathering ...................................................................................... 9
Closing ...................................................................................................................................... 10
Appendix A: Methodology .......................................................................................................... 11
Appendix B: Detailed Results .................................................................................................... 11
Appendix C: Templates .............................................................................................................. 13
Appendix C2: Engagement Specific Templates .................................................................. 14
Appendix D: Landing/Redirect Page ........................................................................................ 15
Appendix E: Acronyms ................................................................................................................ 16
3
Executive Report
This report provides the results for the Department of Homeland Security (DHS) National
Cybersecurity Assessments and Technical Services- (NCATS) led Phishing Campaign Assessment
(PCA) for OFFICE OF EXAMPLE (EXAMPLE). The PCA is a practical exercise intended to support and
measure the effectiveness of security awareness training for information system users. The results
of this PCA show the susceptibility of EXAMPLE personnel to social engineering attacks—specifically
email phishing attacks—in which an adversary tricks an email user into clicking a malicious link to
gain unauthorized network access. In the scenario that this PCA tested, the phishing attack bypassed
technical controls that would normally detect and/or block malicious emails and links. The PCA,
therefore, measured EXAMPLE’s level of vulnerability to a successful phishing attack by targeted
user click rates, click times, response rates, and response times, as shown in Table 1.
This report aims to enhance EXAMPLE’s understanding of their information system users’
cybersecurity behavior and to promote a more secure and resilient workforce.
Table 1: Targeted User Measurements
User Activity Metrics Results
Total users targeted for phishing 1000
# of emails (phishing attempts) sent overall 2000 (2 per user)
# of clicked emails (successful phishing attempts) overall1 219 (10.95% click rate)
# of phished users overall2 203 (20.3% of target population)
# of user reports sent to helpdesk overall3 148 (7.4% report rate)
Ratio of reports-to-clicks .68
Average time to first click4,5 0 hours 52 minutes 41 seconds
Average time to first report6 0 hours 30 minutes 25 seconds
Level 6 “Updated Paycheck
Most successful phishing template
System Policy”
NCATS has provided this PCA to EXAMPLE at no cost and coordinated all activities—including
planning and testing–with EXAMPLE’s point of contact. EXAMPLE maintained control over the testing,
including providing target email addresses, approving phishing email templates, approving testing
timeframes, and adjusting mail security setting to ensure inbox access. This PCA was not intended
to, and did not, test technical controls or electronic protections designed to block phishing attempts.
This PCA spanned a six-week period and aimed to capture behavior-based metrics of EXAMPLE
information system users’ reaction to phishing emails of multiple complexity levels.
1 Click rate is the total number of phishing emails that users clicked on divided by the total number of emails
sent. Users had to click on the “malicious” link in the email in order to be counted as “phished.”
2 There were 203 users who clicked a “malicious” email link at least once by the end of the PCA out of the
1000 that were phished. There were 16 users who clicked in more than one campaign.
3 Reporting rate is total number of user reports to the helpdesk divided by total phishing emails sent.
4 Average time to first click and first report is calculated with geometric mean to compensate for a small
sample size that is sensitive to being skewed by outliers.
5 Click time is the time NCATS sent the emails minus the time user clicked the link within the email.
6 Report time is the time NCATS sent the emails minus the time user alerts security office or helpdesk.
4
NCATS has established this voluntary service to help organizations
• support security awareness training efforts and
• decrease information system user vulnerability to phishing attempts.
Based on the assessment data, NCATS recommends implementing the following:
Table 2: Recommendations
Summary Recommendations
[Recommendation 1]
[Recommendation 2]
[Recommendation 3]
The remainder of this report provides findings and metrics of the NCATS phishing service for
EXAMPLE.
EXAMPLE Methodology Specifics
NCATS used the methodology documented in Appendix A: Methodology to perform testing EXAMPLE.
Specific phishing templates can be reviewed in Appendix C: Templates with corresponding
complexity calculations. NCATS divided the 1000 email addresses that EXAMPLE provided into three
groups. Group 1 had 333 addresses and received levels 1 and 4. Group 2 had 333 addresses and
received levels 2 and 5. Group 3 had 334 addresses and received levels 3 and 6. See Table 3 for
descriptions of email levels 1 – 6.
The assessment’s goal was to capture the behavior-based responses of EXAMPLE to email phishing
attempts; and the assessment operated under the scenario that no technical controls were able to
detect, report, or stop the email phishing attempts. NCATS, as an external body conducting this PCA,
did not have direct knowledge whether or not emails successfully arrived in targeted EXAMPLE user
inboxes. To allow for a clear determination of click rates (email links clicked divided by emails sent),
EXAMPLE or NCATS performed the following:
• EXAMPLE whitelisted the NCATS domain and IP address during the planning stage.
• EXAMPLE created specific mail receiving rules to permit the NCATS emails to land in user
inboxes
Summary of Testing Activities and Results
This PCA concentrated on how phishing email complexity affected EXAMPLE user click behavior and
response behavior. NCATS expects that a more complex and deceptive phishing email has a higher
likelihood of being clicked and a lower likelihood of being reported. This assessment also gathered
organizational emails through open-source intelligence techniques to determine EXAMPLE’s
5
potentially attackable online presence. See Appendix B: Detailed Results for detailed phishing and
customer email data and Appendix C: Templates for a detailed explanation on complexity levels.
Over six weeks, EXAMPLE’s targeted us