November 15, 2022 

1:00pm – 4:00pm ET

#FISSEA2022 | nist.gov/fissea

Federal Information Security Educators (FISSEA) 

Fall Forum

Role-Based Training
#FISSEA2022

Welcome and Opening Remarks

Maureen Premo

FISSEA Co-Chair

Cybersecurity & Infrastructure Security Agency

Department of Homeland Security
#FISSEA2022

Get Involved

Subscribe to the FISSEA Mailing List 

FISSEAUpdates@list.nist.gov

Volunteer for the Planning Committee

Serve on the Contest or Award Committees for 2023

Email fissea@list.nist.gov

Submit a presentation proposal for a future FISSEA Forum 

https://www.surveymonkey.com/r/fisseacallforpresentations
#FISSEA2022

Paula Januszkiewicz

Founder and CEO

Microsoft Regional Director, CQURE

Need for Skills - The Importance of Role-based Training in Tackling 

the Ever-changing Landscape of Threats

Keynote
Need for Skills: The Importance

of Role-based Training in 

Tackling the Ever-changing 

Landscape of Threats

FISSEA Fall Forum, November 15th 2022

Paula Januszkiewicz
Paula Januszkiewicz

CQURE: CEO, Cybersecurity Expert, Penetration Tester

CQURE Academy: Trainer

Microsoft MVP on Cloud and Datacenter Management

Microsoft Regional Director

paula@cqure.us

@PaulaCqure @CQUREAcademy

www.cqureacademy.com

Need for Skills

The Importance of Role-based Training in

Tackling the Ever-changing Landscape of 

Threats

What does CQURE do?

1. Consulting Services:

Extensive IT Security Audits and Penetration Tests

of all kinds,

Configuration Audit and Architecture,

Design Social Engineering Tests,

Advanced Troubleshooting and Debugging,

Emergency Response Services

2. R&D & CQLabs Tools & Hacks Publications

3. Trainings & Seminars:

Offline (mainly in New York or via our partners 

worldwide),

Online
The Impact of Cybercrime
50%

Over 50 percent of all cyber 

attacks are done on SMB’s.

32/65%

Out of all ransomware 

victims, 32 percent pay the 

ransom, but they only get 

65 percent of their data 

back.

62% 

of incidents in the System 

Intrusion pattern involved 

threat actors compromising 

partners.

77% 

of organizations do not 

have a cyber security 

incident response plan.

Impactful Hacking Stats (2022)

Source: Source: Purplesec (2022), Cloudwards (2022), Data Breach Investigation Report (2022); Trend Micro, IBM, CSO (2021); 
“ THERE ARE TWO KINDS OF BIG COMPANIES, THOSE 

WHO’VE BEEN HACKED, AND THOSE WHO DON’ T KNOW 

THEY’VE BEEN HACKED.” -JAMES COMEY, FORMER FBI DIRECTOR

Median number of 

days attackers are 

present on a victims 

network before 

detection

200+

Days after detection 

to full recovery

80

Impact of lost 

productivity and 

growth

$3Trillion

Average cost of a data 

breach (15% YoY 

increase)

$3.9Million
2023 will be a year of a well-written and welltested incident response plan (cyber crisis 

management plan).
Companies should revise goals of penetration 

tests.
Cybersecurity awareness trainings 

should be multi-channeled 

& should reference personal life. 
Effective cyber defense is a long game that 

requires continuous strategic investment.
Workstation security and identity security are 

one of the top priorities now.
Six Techniques to Understand the 

Landscape of Threats
#1

Well Configured Firewall
#1 Well Configured Firewall
Demo:

Phishing Little
Role: Network Engineer

Primary Skill: Network Security

Additional Skills: Computer Network 

Defense, Cryptography & Encryption, 

Architecture

#1 Well Configured Firewall
#2

Usage of MFA and 

Conditional Access
#2 Usage of MFA and Conditional Access
Password Spraying

Attacker

Joe

Louise

Glenn

Ragnar

Dave

Jane

Company16

July2016

Pa$$w0rd!

Company16

Qwerty123

Summer2016

!QAZ2wsX

Authentication System 

e.g. Active Directory
Demo:

Bypassing MFA
Role: Security Administrator

Primary Skill: Security Configuration 

Management

Additional Skills: Identity Management, 

Systems and Application Security, 

Architecture

#2 Usage of MFA and Conditional Access
#3

Network Segmentation 

and SMB Signing
#3A Network segmentation 
Demo:

SMB Relay
#3B Server Message Block Signing (or alternative) 
Role: Enterprise Security Architect

Primary Skill: Defense in Depth

Additional Skills: Computer Network 

Defense, Architecture, Data Security

#3 Network Segmentation and SMB Signing
#4

Whitelisting
#4A Disallowing unusual code execution
#4B Whitelisting on board 
Demo:

Forensics Operations
Role: Security Administrator

Primary Skill: Configuration 

Management

Additional Skills: Information Systems, 

Systems and Applications Security

#4 Whitelisting
#5

Configuration Audit
#5A Decommission of old protocols or their default 

settings
Demo:

Clear Text Queries
#5B Review of the solutions that we trust by default
Demo:

Extracting User’s Secrets
# 5C Do not fall for hipster tools 
Role: System/Network Administrator

Primary Skill: Systems and Application 

Security

Additional Skills: Computer Network 

Defense, Database security

#5 Configuration Audit
#6

Connecting the dots

(monitoring identity etc.)
#6 Monitoring privileged accounts and identity misuse
Demo:

Accounts with Issues
Demo:

DPAPI Beauty
Role: Cybersecurity Analyst

Primary Skill: Identity Management & 

Monitoring

Additional Skills: Incident Management, 

Digital Forensics, Cyber Threat 

Intelligence

#6 Connecting the dots (monitoring identity 

etc.)
Why human 

factor is so 

important?
Summary

#1

Well Configured Firewall

#2 

Usage of MFA and Conditional 

Access

#6

Connecting the Dots

(monitoring identity etc.)

#3

Network Segmentation and 

SMB Signing

#5

Configuration Audit

#4

Whitelisting
Summary

Every employee in the 

organization is the first line of 

defense when it comes to 

protecting your organization 

from cyber threats.

Remote workers will continue 

to be a target for 

cybercriminals. As a side effect 

of remote workforces, cloud 

breaches will increase.

Prevention is the key to success:

How can we know what to 

prevent if we do not know what 

is the threat? 

The cybersecurity skills gap will 

remain an issue. It’s time to act 

– now!

Never leave your devices 

unlocked and unattended.

Be cautious of links and 

attachments in emails – double 

check with the sender if you 

feel uneasy.
Thank you! 
DOWNLOAD THE TOOLS

https://resources.cqureacademy.com/tools/

Username: student

Password: CQUREAcademy#123!
Visit our BLOG and discover more about 

cybersecurity solutions & tools:

https://cqureacademy.com/blog
If you want level up your 

Windows Cybersecurity Skills 

JOIN OUR ONLINE TRAININGS
Paula Januszkiewicz

CQURE: CEO, Cybersecurity Expert, Penetration Tester

CQURE Academy: Trainer

Microsoft MVP on Cloud and Datacenter Management

Microsoft Regional Director

paula@cqure.us

@PaulaCqure @CQUREAcademy

www.cqureacademy.com

Need for Skills

The Importance of Role-based Training in Tackling the Ever-changing Landscape of Threats
#FISSEA2022

Are There Any Questions?

Q&A
#FISSEA2022

Cyber Work Roles Assigned...Now What? Let’s Talk Resources

Featured Presentation

Susan Hansche

Department of Homeland Security

Stacey Wise

Pension Benefit Guaranty Corporation

Cybersecurity & Infrastructure Security Agency IT Specialist
Cyber Work 

Roles Assigned...

Now What? 

Let’s Talk Resources

Stacey Wise, PBGC

Susan Hansche, DHS CISA
HELLO!

I am Susan Hansche

CISA Cybersecurity Division 

60
HELLO!

I am Stacey Wise 

Pension Benefit Guaranty Corporation

(ECD – Enterprise Cybersecurity Department)

Role Based Training Program Lead 

61
Process for 

Identifying and 

Maintaining 

Roles 1
“

▸ Implement NIST Cybersecurity Workforce 

Framework based on PGBC policies

▸ Initial Goal: Identify who has “significant security 

responsibilities”

▸ Process: Survey hiring managers to: identify if a 

work role was identified in their department, how 

many, and a point of contact

▸ Feedback loop: Review, edit, review, edit and 

review, edit

63
Process for Identifying and Maintaining Roles

(continued)

▸ Introduce 1-2 new roles every year as a deliberate 

thoughtful part of the process

▸ Internal “master” list and from that we developed Role 

Based Training (RBT) policy (Bulletin 13 which outlines 

all role-based training requirements) in addition to 

training policies and procedures

▸ Work role “title” is not the same as the hiring/job “title” 

(IT Specialist, etc.)

64
The Work 

Roles...so far

Defined in Bulletin 13 

Reference NIST SP 800-181 NICE Framework

65
▸ Risk Executives

▹ Chief Information Officer (CIO)

▹ Chief Information Security Officer (CISO)

▹ Risk Management Officer (RMO)

▸ Authorizing Officials (AOs)

▸ Common Control Providers (CCPs)

▸ Security Control Assessors (SCAs)*

▸ Information Owners (IOs) and Information System Owners

(ISOs)

▸ Information System Security and Privacy Officers and 

Managers (ISSPOs and ISSMs)

▸ Incident Response Handlers (IRHs) **

The work roles that are trained on an 

annual basis:

66

▸ Privacy Executives 

▹ Senior Agency Official for Privacy (SAOP)

▹ Chief Privacy Officer (CPO)

▸ Cyber Policy & Planning (CPP)

▸ Cyber Training Educator (CTE)

▸ Threat & Warning Analysts (TWA)
What Training is 

needed and where 

do you find it........

▸ Everyone identified in a significant cyber work 

role is required to take one hour of training 

relevant to their role each year

▸ Work with the POC to identify the training that 

is appropriate for the role

67
68

Challenge: Find training that is significant to each role 

▸ How do we find free resources?

▸ What platforms are available?

▸ How do we find courses with valuable 

content?

▸ Do we look at on demand course or 

instructor lead?
Interactive Session

69
Interactive Session

▸ What to look for when choosing RBT?

▸ Where do you get your training from?

▹ External Speakers (if so, where from?)

▹ COTS Training on Internal LMS (i.e., Skillsoft, Precipio)

▹ Design and Develop Internally/Post on Internal LMS

▹ Industry Training (i.e., AWS, Palo Alto, SANS)

▹ Certificate Training (i.e., ISC2, ISACA, PMP)

▹ Federal / MS-ISAC resources?

▹ Academic?

▸ How do provide variety each year?

70

All options are important – this is anonymous posting, so please share!
Interactive Session

▸ What challenges are you having for Role-Based Training? 

▸ What has worked for you?

▸ What hasn’t worked for you?

▸ What would you like to know more about?

71

All options are important – this is anonymous posting, so please share!
72

THANKS!

Any questions?

You can contact us: 

▸ wise.Stacey@pbgc.gov

▸ susan.hansche@cisa.dhs.gov
#FISSEA2022

A Tailored Approach to Workforce Training

Featured Presentation

Scott Anderson Sharon McPherson

Senior Cybersecurity Specialist for Policy, Training & Awareness 

Cyber Workforce Management Department of Veteran Affairs

Department of Veteran Affairs
A TAILORED APPROACH TO 

WORKFORCE TRAINING

Sharon McPherson and Scott Anderson

Office of Information and Technology

Department of Veteran Affairs

FISSEA Presentation

November 15, 2022

FOR INTERNAL USE ONLY
Current Landscape

FOR INTERNAL USE ONLY Office of Information and Technology 75

71% of organizations 

report a shortage of 

cyber skills that 

significantly damaged 

their ability to function 

(Lewis, 2019)

North American 

cybersecurity 

professionals are 

significantly more 

concerned about skill 

shortages than others, 

with 69% of respondents 

saying that this will be a 

future challenge (ISC2 

Study, 2022)

The primary cause for 

workforce shortages is 

the absence of programs

that produce the 

necessary volume of 

trained cybersecurity 

workers. (Lewis, 2022).
Challenges

FOR INTERNAL USE ONLY Office of Information and Technology 76

• Lack of targeted training for specific work roles

• Existing training is too broad

• Limited hands-on training for skill development

• No portability of skills across the Federal Government

!
What We Need

FOR INTERNAL USE ONLY Office of Information and Technology 77

Tailored approaches that provide more relevant and practical training to 

support the execution of tech work.

Targeted 

training for 

specific work 

roles

Hands-on 

training for 

skill 

development

Skills-based 

models

Portability of 

skills within 

Federal 

Government
VA’s Approach to Training

What We’re Doing

• Work Role Skills-Based Profiles 

(SBP)

⎯ Targeted training for OIT 

Competencies and technical 

work

• Self-assessments / Skills 

Maturity Assessments (SMAs)

• Role-Based Training (CTA)

• Experience-Based Learning

⎯ On the Job Training (OJT)

⎯ Hands-on Learning

⎯ Skills Execution

What We’re Using

• Career Pathway tool available 

on NICCS site

⎯ Career Pathways roadmap 

offers a digitalized 

experience on how to 

explore and build 

personalized career 

roadmaps across the 52 

work roles

FOR INTERNAL USE ONLY Office of Information and Technology 78
Cyber Workforce Training (CWT) for Career Development

FOR INTERNAL USE ONLY Office of Information and Technology 79
Skill-Based Profiles

FOR INTERNAL USE ONLY Office of Information and Technology 80

Veteran Affairs X + _ x

https://www.oit.va.gov/...

Gates, William – Software Developer

Sign out

Enter my Profile
Skills Maturity Assessment

To measure an individual’s demonstration of the skills needed to 

perform tasks in a given work role (as defined by the NICE Framework 

for Cybersecurity)

Identify skill gaps to support development

Determine skill maturity over a period of time

Provide an opportunity to highlight developmental opportunities

FOR INTERNAL USE ONLY Office of Information and Technology 81

Skills Maturity Assessment is a tool designed to support programs focused on 

learning and development of skill proficiency.

The goals of the assessment are:
Methodology for Developing a Skills Maturity Assessment (SMA) Tool

• The Skills Maturity Assessments tool measures an individual's 

development, identifies areas of strength, and determines possible skill 

gaps to building maturity in a cyber role.

FOR INTERNAL USE ONLY Office of Information and Technology 82

Work Role Task 

Identification

• Identify tasks, 

abilities, and 

knowledge aligned to 

the work role using 

the NICE 

Cybersecurity 

Framework

Task Permutations 

and Proficiency 

Levels Design

• Design proficiency 

levels for each work 

role task using SME 

input

Task Behavior 

Development

• Develop supporting 

behavioral indicators 

using SME input for each 

task’s proficiency levels

• Indicators help the 

assessor determine what 

successful performance is 

for each proficiency level

SMA Tool Creation

• Create a user-friendly 

tool that quickly 

assesses an individual’s 

task proficiency for 

each of the work role 

tasks

• Provide assessment 

results in a scorecard 

format.
What is the Cyber Training Academy? 

• The Cyber Training Academy (CTA) provides learning activities for all VA staff to 

support them in fulfilling the tasks associated with their cyber work roles.

FOR INTERNAL USE ONLY Office of Information and Technology 83

Ensure all staff has and maintains 

the requisite knowledge, skills, 

abilities, and technical execution 

of tasks that align to the cyber 

work roles within the NICE 

workforce framework. 

Address the development of 

practicable and portable skills by 

create consistent learning 

opportunities for the cyber 

workforce and adopt an integrated 

learning approach that 

incorporates a variety of training 

types, such as skill-based training, 

individual and collective training, 

virtual training, hands-on 

laboratories, mission-based 

activities, and exercises based on 

real-world scenarios. 

Promote employee development 

and growth will not only create a 

skilled workforce that is equipped 

to handle future cyber needs and 

vulnerabilities but will also 

position VA as a cyber employer 

of choice in today’s highly 

competitive recruiting 

environment.
Experiential Learning

• What is Needed for a Successful Experiential Learning Program? 

FOR INTERNAL USE ONLY Office of Information and Technology 84

Built into 

Existing 

Processes & 

Programs

Alignment to 

NICE Work 

Roles and 

associated 

KSAs

Alignment to 

NICE Work 

Roles and 

associated 

KSAs

Specific 

Learning 

Outcomes and 

Metrics

Program 

Focus on 

Skills 

Development
OJT Process Overview

FOR INTERNAL USE ONLY Office of Information and Technology 85

Provides real-time 

training through 

shadowing, 

observation, and skill 

execution

Focuses on tactical 

and applied learning 

to teach new skills or 

improve skills

Building into existing 

process/programs –

onboarding experience 

for new employee, or 

cross-training (with other 

work roles) for existing 

employees Address at least 70% 

of core tasks 

associated with a 

work role at the 

appropriate 

proficiency level

Mechanism to 

address skills gaps 

identified through 

SMA

Incorporates SMA’s 

to measure a 

learner’s 

independence 

execution of work 

role tasks

OJT
Summary

A constantly evolving cyber landscape requires organization to 

take a more proactive role in the learning and development of 

their workforce by:

• Targeted training for specific work roles

• Hands-on training for skill development

• Skills-based models

• Support portability of skills within Federal Government

FOR INTERNAL USE ONLY Office of Information and Technology 86
QUESTIONS?

87
c

BREAK

The Forum will resume at 2:45pm EDT

#FISSEA2022 | nist.gov/fissea

Federal Information Security Educators (FISSEA) 

Fall Forum
#FISSEA2022

Menachem Goldstein

FISSEA Co-Chair

Cybersecurity Specialist, Enterprise Cybersecurity Department

Pension Benefit Guaranty Corporation

Welcome Back!
#FISSEA2022

Donald Costello

Associate Professor

University of Nebraska-Lincoln

Lightning Fast View of Supply Chain Defense and Offense

FISSEA Ignite!
Lightning Fast View of 

Supply Chain Defense and 

Offense

FISSEA TALK NOVEMBER 15 2022
Integrate Supply Chain Management with 

Cybersecurity 

• Know SC Management well

• Recognize without denial or minimization that 

SC’s are under attack.

• Despite the fact the Cybersecurity itself involves 

a lot of Math it still has to be managed

• One aspect of SCM is the organization of the 

enterprise 
What is Change Management ?

• Change management :the process of taking an 

individual or a group of people from a current state to a 

more desired state. It Is LARGELY due to new 

technology and globalization.

Through 2024, 50% of supply chain organizations will 

invest in

applications that support artificial intelligence and 

advanced analytics capabilities organizations WILL 

seek tools that help them make better and more 

informed decisions faster.
Rapid Technology Changes greatly Impact SCD

Supply Chain Digitization. ... 

 Supply Chain Solutions Will Continue to Move to the 

Cloud. ... 

 Omnichannel Supply Chains Become the Norm. ... 

 Sustainability Is Becoming Essential. ... 

 Growth in Circular Supply Chains. ... 

 Agile Supply Chains. ... 

 Internet of Things.

 New technology => New Ideas => New Terminology
Supply Chain attacks & Ransomware

• A supply chain attack breaks into the victim’s network 

by taking advantage of vulnerabilities in the computer 

systems of the company’s supply chain partners. The 

attacks are an attempt to infiltrate an organization’s 

computers to gain access to the networks of its many 

suppliers and partners. Supply chains are perceived as 

especially vulnerable to ransomware attacks due to the 

broad targets they represent and the opportunity to 

cause large-scale damage via a single exploited 

vulnerability.
Look and see that 

• Supply Chain Management is a complex and fascinating field

• The rate of technological change is so large

• You, we have to study and learn about Change Management

• Recognize that we are in a Technology War

• Know that NIST exists and it is an essential tool in understanding 

SC and their defense,

• Know that there is something called a Cybersecure attack to match 

Supply Chain attacks. 
Lightning Fast Key Points made 

Environment

• Systems view essential

• Supply Chain Management 

Basics Change

• Understanding Change 

Management Key

• We are in a Technology War

• The Art of War

You

• Do a YouTube on a Systems 

View and Change Management

• Keep Instagram of how your 

organization is using Supply 

Chain Management

• Allocate time each week to 

study Cybersecurity
The Art of War 

• Sun Tzu is traditionally credited as the author of The Art of 

War, an influential work of military strategy that has affected 

both Western and East Asian philosophy and military 

thinking. It focuses much more on alternatives to battle and 

even to war than on war itself, such as stratagem, delay, the 

use of spies, the making and keeping of alliances, the uses 

of deceit, and a willingness to submit, at least temporarily, to 

more powerful foes.[

• Google and You Tube art of WAR
Contact

• dcostello2@unl.edu
#FISSEA2022

Tamara Kravitz

FISSEA Contest Lead

Security Awareness and Training Contest 

Recognition
FISSEA

Security Awareness and Training Contest

Contest Winners

Tamara Kravitz, PMP

FISSEA Working Group Member

November 2022
Contest

Categories

1. Awareness Poster

2. Awareness Video

3. Awareness Website

4. Innovative Solutions

5. Awareness Newsletter

6. Training Awareness

7. Cybersecurity Podcast

8. Cybersecurity Blog

Judges

Not affiliated with any 

of the groups that 

submitted entries

From various 

positions, industries, 

and academics.
Poster Entries (6)

1. Cofense

2. Labcorp

3. KnowBe4

4. Department of State

5. Federal Retirement Thrift 

Investment Board

6. United States Postal Service
Best Poster

Organization: Labcorp
Video Entries (6)

1. Cofense

2. Federal Retirement Thrift 

Investment Board (FRTIB)

3. Dept. of State (DOS)

4. Labcorp

5. Indian Health Service (IHS)

6. U.S. Office of Personnel 

Management (OPM)
Video Winner!

Office of Personnel Management, Office 

of the Chief Information Security 

Officer, Cybersecurity Program
Website Entries (4)

1. Indian Health Service (IHS)

2. Cofense

3. U.S. General Services 

Administration (GSA)

4. KnowBe4
Website Winner!

Cofense
Innovative Solutions

Entries (8)

1. ECS

2. United States Postal Service

(USPS)

3. Centers for Medicare 

& Medicaid Services (CMS)

4. Labcorp

5. U.S. Department of Education

6. Idaho National Lab

7. Cofense

8. Indian Health Service (IHS)
Innovative Solutions Winner!

Labcorp
Newsletter Entries (6)

1. Centers for Medicare & Medicaid Services 

(CMS)

2. Indian Health Service (HIS)

3. Labcorp

4. Cofense

5. U.S. Department of Education 

6. Department of State (DOS)
Innovative Solutions Winner!

Indian Health Service
Training Entries (8)

1. Cofense

2. Centers for Medicare & 

Medicaid Services

3. KnowBe4

4. Labcorp

5. Social Security Administration

6. Indian Health Service

7. U.S. Department of Education

8. DHS CISA
Training Winner(s)!

Cofense
Training Winner(s)!

U.S. Department of Education
Podcast Entries (2)

1. Centers for Medicare 

& Medicaid Services

2. KnowBe4
Podcast Winner

KnowBe4

The Security Masterminds Podcast takes a deep dive into 

cybersecurity topics with engaging and thought-provoking 

discussions. Hosts Erich Kron and Jelle Wieringa take a deep 

dive into cybersecurity topics, trends, and challenges with 

industry leaders and experts from around the world.

Guests have included chief information security officers 

(CISOs), social engineers, industry leaders, experts on 

emerging technologies and movie directors. The monthly 

podcast has rapidly grown a loyal fan base who appreciate the 

depth of discussions and the 'no filler' approach the podcast 

takes. Security Masterminds brings the best in all things 

cybersecurity, taking an in-depth look at the industry's most 

pressing issues and trends.
Blog Entries (2)

1. Indian Health Service

2. KnowBe4
Blog Winner
Thanks to all 

who submitted entries!

Thanks to our judges!
#FISSEA2022

Security Awareness and Training People’s Choice 

Results

Tamara Kravitz

FISSEA Contest Lead
PEOPLE’S CHOICE AWARDS

CATEGORY (Votes) WINNER (Top Percentage of Votes)

Awareness Poster (57) COFENSE (47.11%)

Awareness Video (50) COFENSE (44.64%)

Awareness Website (72) COFENSE (60.50%)

Innovative Solutions (64) US Department of Education (44.03%)

Awareness Newsletter 

(74)

US Department of Education (51.03%)

Training Awareness (64) US Department of Education (36.57%)

Cybersecurity Podcast 

(50)

Centers for Medicare and Medicaid Services 

(56.82%)

Cybersecurity Blog (51) Indian Health Service (54.04%)
#FISSEA2022

“Choose Your Own Misadventure”

Mark Henderson

Internal Revenue Service
FISSEA Fall Forum

“Choose your own 

misadventure”

Mark Henderson

Internal Revenue Service

Online Fraud Detection and Prevention

OS:CTO:C:O:OFDP

124
Disclaimer

Any views or opinions are my own and do not necessarily reflect the 

official views of the U.S. Treasury.

125
Overview

• present select findings from NIST 8420A

• review potential actions of phishing exercise recipients

• provide reasons why exercises typically cause issues

• describe operational impacts of phishing exercises

• compare data from observed phishing exercises 

• review prohibitions against using agencies or brands

• demonstrate ways to reduce the likelihood of issues with phishing 

exercises

126
Perform phishing exercises (85%)

127
Phishing click rates (#2) and employee 

reporting of simulated phishing (#4)

128
If 40% consult HR/Legal ... then 60% don’t

129
Additional findings from NIST 8420A

• “Ninety percent of survey participants are part-time” [p. 9]

• “fairly equally distributed between Departments (32.3%), sub- components (31.3%), and independent agencies (35.4%)” [p. 10]

• “Organizations with between 1,000 and 4,999 federal employees

made up the largest subset (29.2% of organizations), followed by 

organizations with 50,000 or more employees (21.9%)” [p. 11]

• “32% of the 28 organizations had programs of less than 10,000 

federal employees and contractors (none less than 1,000)” [p. 12]

• “... (n=24, 86%) have a mix of federal employees and contractors

working together on the program.” [p. 15]

130
What is a “controversial” lure?

If you don’t know, please ask your Human Resources and/or Legal 

department

Recent examples [not meant to be exhaustive]:

• Fake bonuses

• GoDaddy, National Institutes of Health study (large Italian hospital), OSHU 

(oshu.edu), Tribune, West Midlands Trains

• Fake employee appreciation awards

• Tacoma Public Schools in Washington

• Use of COVID-19
Simplified actions of exercise recipients

“Click” = Fail

• Receive the exercise landing page

“Don’t click” = Pass

• Do not receive the exercise landing page

132
“Clickers”

open email and click attachment (or URL)

• deliberately [“clickers”]

• by mistake (e.g., trying to report the email) [the “accidental” clicker]

• by analysis (e.g., use a URL analysis engine) [the “curious” clicker]

133
“Non-clickers”

• open email, don’t click and report internally

• open email, don’t click, don’t report (e.g., delete)

• don’t open email but delete by accident (e.g., cleaning Inbox)

• don’t open email (e.g., sick, vacation, etc.) ... but might later

• open email, don’t click and report externally

134
Why some phishing exercises cause issues

• Some cybersecurity professionals are not familiar with relevant laws

• Some organizations:

• do not consult counsel or HR before conducting a phishing exercise

• do not notify and do not obtain permission to use an agency or brand

• “Non-clickers” are unpredictable

• When contact information is not available, external organizations will 

have difficulty contacting the responsible parties

135
Prohibition against USG agencies

“Copyright Exceptions for U.S. Government Works,” 

https://www.usa.gov/government-works#item-206099 (“You cannot 

use government trademarks or government agencies' logos without 

permission.”)
IRS-specific prohibitions

• 15 U.S.C. § 1051 (“irs” is a registered servicemark) 

• 31 U.S.C. § 333 (“any colorable imitation”)

• IRS.gov “Report Phishing” https://www.irs.gov/privacydisclosure/report-phishing (“What if I want to train my employees on 

IRS or tax-related phishing emails by conducting a tax-related phishing 

exercise?”)
Department of Education

• https://fsapartners.ed.gov/knowledge-center/library/electronicannouncements/2021-12-15/guidance-about-use-protected-brandsand-official-government-names-logos-or-insignia

• https://fsapartners.ed.gov/knowledge-center/topics/fsacybersecurity-announcements-and-guidance (FSA Cybersecurity 

Topics)

138
Prohibition against use of brands

• Trademark law(s)

• “Throughout the life of the registration, you must police and enforce your 

rights.” [USPTO]

• Domain Name Disputes (dndisputes.com) [Uniform Domain-Name 

Dispute-Resolution Policy]

• Facebook UDRP against Proofpoint

• Belfius Bank UDRP against Phished

• “Stop Phishing with Bad Fake Bait” (EDUCAUSE) [see comments from 

Shane McGee CPO/GC at Cofense]
What if “non-clickers” reported 

externally to a USG agency?

140
“Anti-Phishing Testers Put Themselves on the 

Hook” [August 2020]

141
What if “non-clickers” reported 

externally to local law 

enforcement?

142
Tredyffrin Police Department [03/22/2016?]

“This scam was brought to their 

attention last week, when a local 

business' employee received one

such alert via email.”

143
Community coverage [03/23/2016]

144
Local media coverage [3/25/2016]

• “Philadelphia-area residents 

have been targeted, and the 

level of information the 

perpetrator has is downright 

scary”

• “The Tredyffrin Police 

Department in Chester County 

[explained] that three local 

residents reported receiving 

emails notifying them of 

speeding infractions”

145
KnowBe4 blog [03/27/2016]

146
CSO (“Salted Hash”) coverage [03/28/2016]

147
Tripwire article [03/29/2016]

148
TV coverage [3/29/2016]

149
What if “non-clickers” at a USG 

agency reported externally to 

other USG agencies?

150
DOJ TSP-themed phishing exercise [2009]

The “email had circulated quickly to other agencies ...” including: 

• the National Oceanic and Atmospheric Administration (NOAA)

• the Economic Development Administration at the Commerce Department

• the Federal Aviation Administration (FAA)

• National Highway Traffic Safety Administration at the Transportation Department 

(NHTSA)

• the Government Accountability Office (GAO)

• the General Services Administration (GSA) alerted the TSP board to the e-mail”

DOJ “... modified our protocols to make sure that doesn't happen in the future.”

151
Army TSP-themed phishing exercise [2014]

Exercise recipients “... forwarded the e-mail to thousands of friends and 

colleagues at the:

- Defense Department (DoD)

- the FBI

- Customs and Border Protection (CBP)

- the Labor Department (DoL) and other agencies.”

A Washington Post article highlighted:

“Defense officials say they will require more oversight of security tests that 

try to trip up employees”

152
What if “non-clickers” reported 

externally to federal law 

enforcement?

153
Phishing Exercise at the DNC [08/2018]

“discovered after national party 

officials already had contacted 

federal law enforcement fearing a 

malicious hacking attempt”

“But cybersecurity protocols, Lord 

said, require that an entity 

conducting phishing tests notify 

other relevant parties so they 

don’t see red flags.”

154
Likelihood of external notifications

Number of 

employees

Click rate 

(10%)

“Non-clickers” 

(90%)

External notifications 

(1% of non-clickers)

10 1 9 N/A

100 10 90 1

1,000 100 900 9

10,000 1,000 9,000 90

100,000 10,000 90,000 900

155
Avoiding issues with phishing exercises

1. consider an alternative (e.g., tabletop exercises)

2. conduct agency-neutral/brand-neutral exercises unless you have 

the explicit permission of the agency/brand

3. have counsel and/or HR review potential exercise lures for potential 

issues before launching your campaign

4. provide “honest signals” or identifiers within the phishing exercise 

campaign (e.g., email headers, landing page, RFC 9116, etc.) [see 

Notes]

156
Green = “Go” (Likelihood of issues very low)

• The phishing exercise does not reference an agency and/or brand

• Controversial phishing exercise lures have been reviewed and 

approved by Legal and Human Resources (HR)

• “Honest Signals” are present throughout the phishing exercise 

construction and/or delivery

157
Yellow = “Proceed with Caution” 

• Agency indirectly referenced (e.g., W2, 1099, etc.)

• Brand indirectly referenced

• Partial review of controversial phishing exercise lures by Legal and 

Human Resources (HR)

• Partial or no “Honest Signals” present throughout phishing exercise 

construction and/or delivery

158
Red = STOP!

• The phishing exercise references an agency or brand directly

• Controversial phishing exercise lures were not checked by Legal or 

Human Resources (HR)

• Few if any “Honest Signals” present throughout phishing exercise 

construction and/or delivery

159
Questions?

160
Thank you!

161
#FISSEA2022

Menachem Goldstein

FISSEA Co-Chair

Cybersecurity Specialist, Enterprise Cybersecurity Department

Pension Benefit Guaranty Corporation

Closing Remarks
#FISSEA2022

Get Involved

Subscribe to the FISSEA Mailing List 

FISSEAUpdates@list.nist.gov

Volunteer for the Planning Committee

Serve on the Contest or Award Committees for 2023 

Email fissea@list.nist.gov

Submit a presentation proposal for a future FISSEA Forum 

https://www.surveymonkey.com/r/fisseacallforpresentations
THANK YOU

https://www.surveymonkey.com/r/2022fisseafallforum

We look forward to receiving your feedback via the post-event survey!

#FISSEA2022 | nist.gov/fissea
#FISSEA2022

FISSEA Winter Forum

February 14, 2023

1:00pm – 4:00pm EDT

REGISTER TODAY: nist.gov/fissea
Federal Information Security Educators 

(FISSEA) Conference

April 19, 2023

#FISSEA2022 | nist.gov/fissea

SAVE THE DATE