Risk-Informed and 

Performance-Based 

Oversight of Radiological 

Emergency Response 

Programs

Office of Nuclear Security and Incident Response

NUREG/CR-7195
NRC Reference Material 

As of November 1999, you may electronically access 

NUREG-series publications and other NRC records at 

NRC’s Public Electronic Reading Room at 

http://www.nrc.gov/reading-rm.html. Publicly released 

records include, to name a few, NUREG-series 

publications; Federal Register notices; applicant, 

licensee, and vendor documents and correspondence; 

NRC correspondence and internal memoranda; bulletins 

and information notices; inspection and investigative 

reports; licensee event reports; and Commission papers 

and their attachments. 

NRC publications in the NUREG series, NRC 

regulations, and Title 10, “Energy,” in the Code of

Federal Regulations may also be purchased from one 

of these two sources. 

1. The Superintendent of Documents

 U.S. Government Printing Office

Mail Stop SSOP

Washington, DC 20402–0001

Internet: bookstore.gpo.gov

Telephone: 202-512-1800

Fax: 202-512-2250 

2. The National Technical Information Service 

Springfield, VA 22161–0002

www.ntis.gov

1–800–553–6847 or, locally, 703–605–6000 

A single copy of each NRC draft report for comment is 

available free, to the extent of supply, upon written 

request as follows:

Address: U.S. Nuclear Regulatory Commission

 Office of Administration 

Publications Branch

 Washington, DC 20555-0001

E-mail: DISTRIBUTION.RESOURCE@NRC.GOV

Facsimile: 301–415–2289 

Some publications in the NUREG series that are 

posted at NRC’s Web site address 

http://www.nrc.gov/reading-rm/doc-collections/nuregs 

are updated periodically and may differ from the last 

printed version. Although references to material found on 

a Web site bear the date the material was accessed, the 

material available on the date cited may subsequently be 

removed from the site.

Non-NRC Reference Material 

Documents available from public and special technical 

libraries include all open literature items, such as books, 

journal articles, transactions, Federal Register notices, 

Federal and State legislation, and congressional reports. 

Such documents as theses, dissertations, foreign reports 

and translations, and non-NRC conference proceedings 

may be purchased from their sponsoring organization. 

Copies of industry codes and standards used in a 

substantive manner in the NRC regulatory process are 

maintained at—

The NRC Technical Library

Two White Flint North

11545 Rockville Pike

Rockville, MD 20852–2738 

These standards are available in the library for reference 

use by the public. Codes and standards are usually 

copyrighted and may be purchased from the originating 

organization or, if they are American National Standards, 

from—

American National Standards Institute

11 West 42nd Street

New York, NY 10036–8002

www.ansi.org

212–642–4900 

AVAILABILITY OF REFERENCE MATERIALS

IN NRC PUBLICATIONS

Legally binding regulatory requirements are stated only 

in laws; NRC regulations; licenses, including technical 

specifications; or orders, not in NUREG-series 

publications. The views expressed in contractorprepared publications in this series are not necessarily 

those of the NRC.

The NUREG series comprises (1) technical and 

administrative reports and books prepared by the staff 

(NUREG–XXXX) or agency contractors (NUREG/CR–

XXXX), (2) proceedings of conferences (NUREG/CP–

XXXX), (3) reports resulting from international 

agreements (NUREG/IA–XXXX), (4) brochures 

(NUREG/BR–XXXX), and (5) compilations of legal 

decisions and orders of the Commission and Atomic and 

Safety Licensing Boards and of Directors’ decisions

under Section 2.206 of NRC’s regulations (NUREG–

0750).

DISCLAIMER: This report was prepared as an account 

of work sponsored by an agency of the U.S. 

Government. Neither the U.S. Government nor any 

agency thereof, nor any employee, makes any warranty, 

expressed or implied, or assumes any legal liability or 

responsibility for any third party’s use, or the results of 

such use, of any information, apparatus, product, or 

process disclosed in this publication, or represents that 

its use by such third party would not infringe privately 

owned rights.
Risk-Informed and 

Performance-Based 

Oversight of Radiological 

Emergency Response 

Programs

Manuscript Completed: August 2014 

Date Published: May 2015

Prepared by: 

Randolph Sullivan, Thomas Park*, Abbey Jorstad*, Douglas Hoell* 

*Obsidian Analysis, Inc.

 1776 Eye Street NW, 4th Floor

 Washington, DC 20006

R. Sullivan, NRC Technical Monitor

NRC Job Code R3151

Office of Nuclear Security and Incident Response

NUREG/CR-7195

 

iii

ABSTRACT

This report provides a concept for performance-based oversight of offsite radiological 

emergency response preparedness in jurisdictions surrounding commercial nuclear power 

plants. That is, it provides the framework for development of an alternative oversight regimen in 

which regulators consider inputs and enablers of performance (such as plans and training) only 

when jurisdictions cannot demonstrate adequate performance. The report proposes an initial set 

of objective performance indicators for demonstration in drills and exercises. It further proposes 

differential levels of oversight intervention based on the degree to which the jurisdictions meet 

or fail to meet performance targets. The report considers whether this performance-based 

oversight regimen would enable better integration of offsite radiological emergency response 

preparedness with all-hazards preparedness. Finally, the report briefly considers aspects of 

implementing the concept, such as potentially required regulatory changes. While the ingestion 

pathway is important to public health and safety, it is not considered for regulation in the scope 

of this report. The report concludes that a performance-based oversight system is feasible and 

could enhance all-hazards integration along with reasonable assurance. However, 

implementation of a performance-based oversight regimen likely would require more resources 

than are currently applied in order to ensure a high level of emergency preparedness.

v

CONTENTS

ABSTRACT................................................................................................................................iii

CONTENT ..................................................................................................................................v

LIST OF FIGURES ....................................................................................................................ix

LIST OF TABLES.......................................................................................................................xi

ABBREVIATIONS.................................................................................................................... xiii

1. INTRODUCTION............................................................................................................ 1

2. RATIONALE FOR CONSIDERING CHANGES TO THE CURRENT REGULATORY

REGIMEN ...................................................................................................................... 3

2.1 More Efficient Use of Personnel and Equipment For Response, Demonstration, 

and Oversight.......................................................................................................... 3

2.2 Enhanced Oversight Focus on the Most Risk-Significant Aspects of Radiological

Emergency Response Programs............................................................................. 3

2.3 Potential Alignment with Other Elements of Nuclear Reactor Safety Oversight and 

All-Hazards Preparedness Initiatives....................................................................... 4

2.4 Improved Public Understanding .............................................................................. 5

2.5 Enhanced Flexibility in Response Options............................................................... 7

3. PERFORMANCE GOALS FOR OFFSITE RESPONSE ................................................. 9

4. DESIGN CONSIDERATIONS FOR A REVISED REGULATORY REGIMEN.................11

4.1 Stakeholder Analysis..............................................................................................11

4.1.1 Nuclear Regulatory Commission Licensees .................................................11

4.1.2 State and Local Governments......................................................................12

4.1.3 General Public .............................................................................................12

4.1.4 Federal Emergency Management Agency....................................................13

4.2 Proposed Characteristics of a Revised Regulatory Regimen..................................14

4.2.1 Transparency in Methodology for Determining Reasonable Assurance........14

4.2.2 Flexibility ......................................................................................................14

4.2.3 Streamlining and Rationalization ..................................................................14

4.2.4 Resource Neutrality......................................................................................14

4.2.5 Adequacy of Available Information ...............................................................15

4.2.6 Other Considerations ...................................................................................15

5. ELEMENTS OF RERP AND THEIR RISK SIGNIFICANCE...........................................17

5.1 Radiological Emergency Response Logic ..............................................................17

5.2 Radiological Emergency Response Tasks and Risk Significance...........................19

5.3 High Risk Significance Tasks .................................................................................20

5.4 Medium Risk Significance Tasks............................................................................22

5.5 Low Risk Significance Tasks ..................................................................................22
vi

6. PERFORMANCE-BASED OVERSIGHT SYSTEM FOR RADIOLOGICAL 

EMERGENCY RESPONSE PROGRAM .......................................................................23

6.1 Decomposition of Tasks into Subtasks...................................................................23

6.2 Methods and Frequency of Performance Demonstration........................................24

6.3 Performance Indicators and Acceptable Performance............................................26

6.3.1 Concepts for Performance Indicators ...........................................................26

6.3.2 Modification of Existing Radiological Emergency Preparedness Program 

Criteria .........................................................................................................27

6.4 Aggregate Timeliness of Performance ...................................................................28

6.4.1 Construct of Problem ...................................................................................28

6.4.2 Assessing Timeliness of Protective Action Decision-Making and Public 

Notification ...................................................................................................29

6.4.3 Options for Assessing Results......................................................................30

6.4.4 Limitations....................................................................................................31

6.5 Alternative, Proxy Indicators for Performance.........................................................31

6.5.1 Evacuation ...................................................................................................31

6.5.2 Shelter-in-Place............................................................................................32

6.5.3 Potassium Iodide Distribution.......................................................................32

6.6 Issue Handling and Differential Levels of Oversight................................................32

6.7 Enhancement of Risk Focus...................................................................................38

7. CONSISTENCY WITH PROPOSED ONSITE PERFORMANCE-BASED

OVERSIGHT .................................................................................................................41

8. INTEGRATION WITH ALL-HAZARDS INITIATIVES .....................................................43

8.1 Objectives for Integration .......................................................................................44

8.1.1 Meaning of “Integration” ...............................................................................44

8.1.2 Rationale for Integration...............................................................................44

8.1.3 Dimensions of Integration.............................................................................44

8.1.4 Objectives for Integration .............................................................................45

8.2 Integration Context: All-Hazards Initiatives, Processes, and Systems ....................45

8.2.1 Scope of All-Hazards Initiatives....................................................................45

8.2.2 Means to Influence Compliance ...................................................................46

8.2.3 Key All-Hazards Initiatives for Integration.....................................................49

8.3 All-Hazards Integration Potential: Offsite................................................................52

8.3.1 Risk Prioritization: Threat and Hazard Identification and Risk Assessment ..52

8.3.2 Performance Targets (National Preparedness Goal, Threat and Hazard 

Identification and Risk Assessment).............................................................52

8.3.3 Funding (Grants)..........................................................................................55

8.3.4 Performance Input: Plans (CPG 101)...........................................................56
vii

8.3.5 Performance Input: Organization (National Incident Management System 

Incident Command System).........................................................................57

8.3.6 Performance Input: Equipment (National Incident Management System 

Resource Typing).........................................................................................57

8.3.7 Performance Input: Training (National Incident Management System 

Qualification Systems)..................................................................................59

8.3.8 Performance Input: Exercise Doctrine (Homeland Security Exercise and 

Evaluation Program) ....................................................................................59

8.3.9 Categorization and Reporting: Core Capabilities..........................................60

8.3.10Categorization and Reporting: Exercise Evaluation (Homeland Security 

Exercise and Evaluation Program)...............................................................63

8.3.11Categorization and Reporting: Exercise Evaluation (National Exercise 

Program)......................................................................................................65

8.3.12 Categorization and Reporting: Assessments (National Preparedness 

Report, State Preparedness Report) ............................................................66

8.4 All-Hazards Integration Potential: Onsite................................................................66

8.5 All-Hazards Integration Prospects: Summary .........................................................67

9. IMPLEMENTATION CONSIDERATIONS......................................................................71

9.1 Potentially Increased Resource Burdens for Federal Oversight..............................71

9.2 Impact of Implementation on OROs .......................................................................72

9.3 Required Changes in Existing Regulations and Guidance......................................72

9.3.1 10 CFR 50.47, Emergency Plans, and 10 CFR 50 Appendix E ....................72

9.3.2 44 CFR 350, Review and Approval of State and Local Emergency 

Response Plans and Preparedness .............................................................73

9.3.3 44 CFR 351, Radiological Emergency Planning and Preparedness .............73

9.3.4 44 CFR 352, Commercial Nuclear Power Plants: Emergency 

Preparedness Planning................................................................................73

9.3.5 44 CFR 354, Fee for Services ......................................................................74

9.3.6 NUREG-0654/FEMA-REP-1, Rev. 1 and Guidance Generally .....................74

9.4 Notional Implementation Process and Milestones ..................................................74

10. CONCLUSIONS............................................................................................................77

11. REFERENCES..............................................................................................................79

11.1Law and Regulation................................................................................................79

11.2Compilations of Presidential Documents ................................................................79

11.3Nuclear Regulatory Commission References .........................................................80

11.4Other References...................................................................................................81

APPENDIX A: SUBTASK CHECKLISTS................................................................................A-1

APPENDIX B: INITIAL REVIEW OF EXISTING CRITERIA....................................................B-1

APPENDIX C: ONE SOFTWARE TOOL FOR EVALUATION ................................................C-1

ix

LIST OF FIGURES 

Figure 1. Organization of This Report ............................................................................. 2

Figure 2. The NRC’s Overall “Reasonable Assurance” of Public Health and Safety............. 5

Figure 3. Layers of Interpretation for “Reasonable Assurance” of Offsite Preparedness....... 7

Figure 4. Radiological Emergency Response Tasks and Risk Significance....................... 20

Figure 5. Time for Protective Action Decision-Making and Notification of the Public .......... 28

Figure 6. Emergency Response Planning Areas in EPZ ................................................. 29

Figure 7. Example of State Preparedness Data in National Preparedness Report............. 48

Figure 8. NIMS ICS Generic Incident Organizational Structure........................................ 51

Figure 9. Example of an HSEEP EEG........................................................................... 64

Figure C-1. ARPAT Summary Screen ............................................................................ C-1

Figure C-2. ARPAT Data Entry ...................................................................................... C-2

xi

LIST OF TABLES

Table 1. Phases of a Radiological Release .................................................................. 18

Table 2. Protective Actions and Exposure Pathways of Concern, by Phase..................... 18

Table 3. Proposed Frequency of Subtask Demonstrations Based on Resource 

Requirements and Risk Significance............................................................... 25

Table 4. Methods for and Frequency of Subtask Demonstrations ................................... 25

Table 5. Mean Number of ARCAs and Deficiencies by FEMA Region, 1999-2012 ........... 34

Table 6. Proposed Evaluative Schema for Performance-Based RERP Oversight, Single 

Demonstration of Subtask.............................................................................. 36

Table 7. Proposed Evaluative Schema for Performance-Based Oversight, Cumulative 

Biennial Subtask and Task Performance......................................................... 37

Table 8. Proposed Evaluative Schema for Performance-Based Oversight, Cumulative 

Biennial ORO RERP “Cornerstone” Performance............................................. 38

Table 9. Number of Deficiencies by Type, 1999-2012 ................................................... 40

Table 10. Relative Contribution of Grants and Nuclear Industry Fees to Selected State 

Emergency Management Agency Budgets ...................................................... 49

Table 11. Selected Standardized Forms under NIMS ICS ............................................... 51

Table 12. Example of Capability Targets in the National Preparedness Goal..................... 53

Table 13. Selected Capability Targets from Three States’ 2012 THIRAs........................... 53

Table 14. State Approaches to RERP Requirements in All-Hazards Planning ................... 56

Table 15. NIMS Typed Resource Definition with Radiological Detection Capability ............ 58

Table 16. Response Core Capabilities from the National Preparedness Goal.................... 61

Table 17. Crosswalk of Response Core Capabilities, Current REP Program Manual 

Assessment Areas, and Tasks in the Proposed RIPB RERP Oversight 

Construct ..................................................................................................... 62

Table 18. Rating Scales for HSEEP and RERP Exercise Evaluations............................... 65

Table 19. Factors for Assessing Integration Prospects of All-Hazards Initiatives with 

RIPB RERP.................................................................................................. 68

Table 20. Notional Implementation Process and Milestones for the RIPB RERP Oversight 

Construct ..................................................................................................... 75

Table B-1. Review of Existing Criteria for Actual or Implied Quantitative Measures 

and Indicators............................................................................................B-1

xiii

ABBREVIATIONS

μCi/cc Microcuries per Cubic Centimeter

AAR After-Action Report

ALC Annual Letter of Certification

ARCA Area Requiring Corrective Action

ARPAT All-hazards Response and Preparedness Assessment Tool

CI Capability Indicator

CPG Comprehensive Preparedness Guide

DHS Department of Homeland Security

DQI Deductive Quantification Index

EAS Emergency Alert System

EMPG Emergency Management Performance Grant

EMT Emergency Medical Technician

EOC Emergency Operations Center

EOP Emergency Operations Plan

EP Emergency Preparedness

EPA Environmental Protection Agency

EPZ Emergency Planning Zone

ETE Evacuation Time Estimate

FEMA Federal Emergency Management Agency

HAZUS Hazards United States

HSEEP Homeland Security Exercise and Evaluation Program

HSPD Homeland Security Presidential Directive

ICS Incident Command System

IP Improvement Plan

JIC Joint Information Center

KI Potassium Iodide

NEI Nuclear Energy Institute

NEP National Exercise Program

NIMS National Incident Management System

NPP Nuclear Power Plant

NPR National Preparedness Report

NPRM Notice of Proposed Rulemaking

NRC Nuclear Regulatory Commission

ORO Offsite Response Organization

PAD Protective Action Decision

PAG Protective Action Guidelines

PKEMRA Post-Katrina Emergency Management Reform Act

PM Performance Measure
xiv

PPD Presidential Preparedness Directive

REP Radiological Emergency Preparedness

RERP Radiological Emergency Response Program

RIPB Risk-informed, performance-based

ROP Reactor Oversight Process

SAV Staff Assistance Visit

SNPRM Supplemental Notice of Proposed Rulemaking

SNRA Strategic National Risk Assessment

SPR State Preparedness Report

TEP Training and Exercise Plan

THIRA Threat and Hazard Identification and Risk Assessment
1

1. INTRODUCTION

The Nuclear Regulatory Commission (NRC) relies upon a defense-in-depth approach to ensure 

adequate protection of public health and safety against risks from licensed nuclear power plant

(NPP) operations. The 2011 incident in Fukushima, Japan, is a reminder that while the 

likelihood that high-quality designed components, safety systems, highly trained operators, and 

containment systems all will fail is low, it is not zero. The ability of both licensees and State and 

local offsite response organizations (OROs) to respond to an emergency is the critical final link 

in the defense-in-depth concept.

The NRC must find there is “reasonable assurance that adequate protective measures can and 

will be taken in the event of a radiological emergency,” or it can take enforcement action to 

include shutting down a reactor (see 10 CFR 50.54(s)(2)(ii)). Inputs to that determination come 

from the NRC’s own assessment of licensee emergency preparedness and the Federal 

Emergency Management Agency’s (FEMA) assessments regarding whether State and local 

emergency plans are adequate and capable of being implemented.

The NRC’s reasonable assurance determinations and FEMA’s Radiological Emergency 

Preparedness (REP) Program rely on 16 planning standards in regulation (at 10 CFR 50.47(b) 

and 44 CFR 350.5), as well as criteria in NUREG-0654/FEMA-REP-1, Revision 1, Criteria for 

Preparation and Evaluation of Radiological Emergency Response Plans and Preparedness in 

Support of Nuclear Power Plants. While there has been additional guidance, including a 2013

update to FEMA’s REP Program Manual, these foundational standards and criteria date from 

1980. They focus on review of emergency plans for adequacy and on demonstrating the ability 

to implement plans, mainly through exercises.

However, in the voting record for SECY-06-0200, the NRC’s Commissioners endorsed 

exploration of a more “risk-informed, performance-based” (RIPB) regulatory approach for 

reasonable assurance determinations regarding emergency preparedness, including offsite 

preparedness. As used in this report, “risk-informed” oversight involves focusing on the 

elements most significant to protecting public health and safety. “Performance-based” oversight 

involves demonstrating achievement of successful outcomes rather than compliance with 

procedural requirements.1

This report offers an initial proposal for the development of an RIPB oversight regimen for offsite 

Radiological Emergency Response Programs (RERP). While the ingestion pathway is important 

to ensuring public health and safety, it is not considered for regulation in the scope of this report.

The remaining report chapters address the topics shown in Figure 1 on the next page.

 1 See NRC 2012. The term “risk-informed” as used for onsite emergency preparedness (EP) has 

traditionally meant to incorporate risk insights into the program and supporting guidance where they can 

be identified. Risk-informing EP does not rely on quantitative analysis of core damage frequency but 

assesses qualitatively the risk to the public should an NPP accident occur. This practice was formalized 

for regulatory purposes in the EP Cornerstone of the reactor oversight process. (For example, see IMC 

0308, Attachment 3, Appendix B: “the EP [significance determination process] is risk-informed, rather 

than risk-based, and does not involve numerical estimates of risk metrics such as core damage frequency 

(CDF) or large early release fraction (LERF).”) The EP significance determination process assumes that 

the EP program is needed for protection of public health and safety in order to assess the significance of 

degraded program elements. This regulatory structure recognizes EP as a defense-in-depth measure. 

Another way of viewing this use of risk information is that for oversight purposes core damage is assumed 

to be progressing and program elements that reduce consequences are risk-significant.
2

Figure 1. Organization of This Report
3

2. RATIONALE FOR CONSIDERING CHANGES TO THE CURRENT 

REGULATORY REGIMEN

The current system for providing “reasonable assurance” of offsite radiological preparedness 

has served for over 30 years. However, a shift to outcome-oriented performance measures, 

linked to an overarching health and safety goal for assessing offsite radiological emergency 

response preparedness, could include the following benefits:

(1) More efficient use of personnel and equipment for response, demonstration, and 

oversight,

(2) Enhanced oversight focus on the most risk-significant aspects of RERP,

(3) Potential alignment with other elements of nuclear reactor safety oversight as well as allhazards preparedness initiatives,

(4) Improved public understanding of how “reasonable assurance” relates to safety, and

(5) Enhanced flexibility in response options.

2.1 MORE EFFICIENT USE OF PERSONNEL AND EQUIPMENT FOR RESPONSE,

DEMONSTRATION, AND OVERSIGHT

One hope for a performance-based system is that it could require fewer resources to achieve 

substantially the same outcome: it could be more efficient.

An oversight system focused on compliance requires an assignment of response resources to

comply with the requirement, an assignment of resources to demonstrate compliance with the 

requirement, and an assignment of oversight resources to ensure compliance with the 

requirement—whether or not the requirement has been proven to contribute even marginally to 

better health and safety outcomes. If the requirement does not contribute to ensuring an 

adequate health and safety outcome, then the investment of resources in meeting that 

requirement is wasted. Even if the requirement contributes only marginally to health and safety, 

an alternative use of resources may contribute more. Tying up resources in a compliance 

requirement when better uses are available also constitutes waste. A shift to performancebased oversight has the potential to reduce or eliminate such waste. 

2.2 ENHANCED OVERSIGHT FOCUS ON THE MOST RISK-SIGNIFICANT ASPECTS OF 

RADIOLOGICAL EMERGENCY RESPONSE PROGRAMS

To achieve efficiencies, a performance-based oversight regimen must be risk-informed (i.e., 

focused on those aspects of radiological emergency response that contribute most to reducing 

the risk that the public would receive an unsafe dose of radiation in the event of a radiological 

emergency). Consideration of a shift to performance-based oversight offers a chance to 

reconsider which elements of RERP are most risk-significant and should therefore receive 

greater relative attention and resources.
4

2.3 POTENTIAL ALIGNMENT WITH OTHER ELEMENTS OF NUCLEAR REACTOR 

SAFETY OVERSIGHT AND ALL-HAZARDS PREPAREDNESS INITIATIVES

FEMA’s adoption of more outcome-based metrics supporting a defined health and safety 

outcome could improve the linkage between FEMA’s offsite preparedness determinations and 

the NRC’s overall “reasonable assurance” determinations.

First, the NRC could judge the contributions of onsite emergency preparedness to meeting the 

same health and safety outcome. For example, if the overarching metric involved timeliness for 

completing protective actions from awareness of an event to evacuation (if appropriate) of the 

emergency planning zone (EPZ), the time taken by licensees to complete their sequence of 

actions could reduce time available for OROs to complete their actions. The NRC and FEMA 

would be able to coordinate holistically any needed emergency preparedness improvements for 

both onsite and offsite response in order to achieve the target outcome.

Second, more frequent collection of performance demonstration results and the development of 

rules for aggregating results from these individual performance demonstrations over time could 

allow for greater comparability of presentation between FEMA’s offsite RERP determinations 

and the NRC’s current Reactor Oversight Process (ROP). If all parties were willing, an offsite 

RERP “cornerstone” rating could be given for each site. 

Third, the NRC could more easily incorporate outcome-oriented, quantitative FEMA findings into 

an RIPB scheme for oversight of all elements of its defense-in-depth approach (see Figure 2 on 

the next page) if the Commission chose to move further in this direction. While the NRC has had 

quantitative objectives to inform regulation since its Safety Policy Goal of 1986, the NRC’s 

Fukushima Near-Term Task Force has recommended establishing a “logical, systematic, and 

coherent regulatory framework for adequate protection that appropriately balances defense in 

depth and risk considerations.”1 To the extent the NRC can estimate probability of success or 

failure for each defense-in-depth element (including emergency preparedness), along with the 

range of potential consequences from accident sequences, the NRC may be able to offer a 

more transparent basis for its overall “reasonable assurance” determinations in the future. 

More importantly for FEMA, a shift to a more performance-based oversight regimen could offer 

opportunities to enhance integration of RERP and all-hazards initiatives. If oversight focuses on 

achievement of outcomes rather than compliance of inputs, OROs may be able to increase 

linkages between all-hazards and RERP-specific efforts. For example, jurisdictions that maintain 

a stand-alone RERP plan primarily to satisfy separate RERP oversight requirements, regardless 

of the likely use of the plan in an actual incident, could base their decision strictly on whether a 

stand-alone RERP plan enhances RERP response. Over time such jurisdictions likely would 

integrate all but the truly RERP-specific content with the all-hazards plan. 

 1 The 1986 Safety Policy Goals statement included the quantitative health objective that “the risk to the 

population in the area near an NPP of cancer fatalities that might result from nuclear power plant 

operation should not exceed one-tenth of one percent (0.1 percent) of the sum of cancer fatality risks 

resulting from all other causes.” Targets also exist for core damage frequency (reflecting effectiveness in 

controlling and mitigating initiating events) and “large early release frequency” (reflecting effectiveness of 

containment barriers). See SECY-00-007. Apostolakis (2012) reviews the NRC’s history of increasingly 

incorporating risk considerations into its work. 
5

Figure 2. The NRC’s Overall “Reasonable Assurance” of Public Health and Safety

2.4 IMPROVED PUBLIC UNDERSTANDING

It is possible that a more objective and performance-based oversight system for offsite RERP 

could make reasonable assurance determinations easier for the public to grasp. The current 

system of oversight involves a combination of plan reviews, exercises, and certifications to 

determine “adequacy of the plans and capabilities of State and local governments to effectively 

implement the plans” (44 CFR 350.1). The system of oversight typically involves the following 

process:

• States submit plans for review. For each site within the State, a State submits its 

radiological emergency plan and the plans of local jurisdictions in the 10-mile plume 

exposure pathway EPZ and 50-mile ingestion pathway EPZ. The State must give its 

opinion that the plans are adequate to protect the public health and safety of its citizens 

in the EPZs and that the State and local governments can, and intend to take,

appropriate protective actions in the event of a radiological emergency.

• FEMA regions review each plan in detail. FEMA publishes a Federal Register notice that 

the plans are available for review, provides copies to an interagency Regional 

Assistance Council for review and input, and completes an assessment of the adequacy 

of the State and local governments’ capability to implement the plans. Reviews are 

keyed to the planning standards in regulation, NUREG-0654 and its supplements, and 

the REP Program Manual.

• FEMA evaluates an exercise of each plan. The licensee conducts an exercise with the 

State and local governments, allowing FEMA to evaluate the ORO plans and their 
6

implementation. After the initial qualifying exercise for plan approval, the State and local 

governments within the 10-mile EPZ participate in such a joint exercise at least once 

every two years. (States with multiple sites may rotate their full participation among 

sites.)

• A public meeting is held regarding the plan and the exercise. At or near the licensee 

facility, a public meeting must be held to describe the plan and the exercise, answer 

questions about the FEMA review, and receive suggestions for improvements. (After the 

initial qualifying exercise, subsequent public meetings regarding exercises focus on 

explaining the exercise process.)

• The FEMA Regional Administrator forwards an evaluation of the plan to the FEMA 

Deputy Administrator. Once an exercise and public meeting have been held, the FEMA 

Regional Administrator will forward to the Deputy Administrator an evaluation of the plan, 

results of the exercise (including deficiencies noted and corrections made), a summary 

of deficiencies identified during the public meeting, recommendations made to the State, 

and the State’s commitments and actions to address the deficiencies and 

recommendations.

• The FEMA Deputy Administrator approves the plan or indicates that it is inadequate. The 

FEMA Deputy Administrator determines whether the plans are adequate and capable of 

being implemented. Approval is subject to appeal within 30 days by any interested 

person, but only on the grounds that the decision was unsupported by substantial 

evidence. After initial approval, FEMA continues to review the adequacy of plans and 

preparedness over time, and may withdraw approval at any time (e.g., on the basis of 

biennial exercise findings, a disaster-initiated review, or other evidence).

• Determination that the plan is inadequate initiates a 120-day window to take action on 

deficiencies to FEMA’s satisfaction or face withdrawal of approval. If notified that the 

plan is not adequate, the State has 120 days to correct deficiencies or provide a plan 

and timeline for correcting the deficiencies. If after 120 days (or the agreed timeline) the 

deficiencies are not corrected, FEMA may notify the Governor, the NRC, other agencies, 

and the public that approval (i.e., that FEMA has “reasonable assurance” that plans are 

adequate and capable of being implemented) is withdrawn.

• FEMA’s withdrawal of approval escalates to the NRC, and the NRC decides on an 

appropriate response. If the NRC determines that reasonable assurance is 

compromised, the NRC may give the licensee 120 days to correct the problem or face 

possible enforcement action (although the Commission is not constrained to a 120-day 

timeline). The Commission will determine whether the reactor should be shut down until 

the deficiencies are corrected or whether other enforcement action is appropriate. The 

Commission takes into account whether the licensee can demonstrate that the 

deficiencies are not significant, or whether there are compensatory actions the licensee 

can take. 

FEMA supplements the plan reviews and exercises with Staff Assistance Visits (SAVs) and an 

Annual Letter of Certification (ALC) from each REP Program State.2

This approach is well institutionalized among licensees, State and local governments, FEMA, 

and the NRC. The 16 planning standards defined in regulation (10 CFR 50.47(b) and 44 CFR 

350.5) have served as a stable foundation for plan reviews and exercise evaluation for more 

than 30 years.

 2 The process description is drawn from the summary of 44 CFR 350 given in FEMA 2013f, pp. IV-2 to IV9. See also pp. IV-36 to IV-37 and IV-49 to IV-57 regarding SAVs and ALCs, respectively. 
7

The planning standards also rely on two additional layers of interpretation. (See Figure 3, 

below.) NUREG-0654/FEMA-REP-1, Revision 1, offers more than 190 criteria for review of 

plans. FEMA’s April 2013 REP Program Manual additionally offers six “assessment areas” with 

25 sub-elements and 32 demonstration criteria, mapped to the NUREG-0654 plan review 

criteria.

Figure 3. Layers of Interpretation for “Reasonable Assurance” of Offsite Preparedness

The benefit of this structure is that it allows the flexibility to make incremental changes without 

the time-consuming effort of revising the underlying regulations. For example, FEMA and the 

NRC have jointly issued four supplements to NUREG-0654. In 2012 FEMA and the NRC 

announced plans to begin revising NUREG-0654, with the new document to be issued in 2017 

(see 77 FR 65700). Also, in response to a multi-year REP Program strategic review, FEMA 

streamlined and reorganized exercise evaluation criteria for interim use in 2001 (see 66 FR 

31342). Two of those criteria were dropped by the time a consolidated REP Program Manual 

was published in October 2011 and revised in April 2012 and June 2013. 

The drawback of the existing layered approach is that it is not transparent. While all of the 

documents are made publicly available, identifying how compliance or non-compliance with 

hundreds of separate criteria translates to a single FEMA “reasonable assurance” determination 

is difficult to communicate. Further, it is unclear what degree of safety the public can expect 

based on a FEMA determination that a plan and its supporting capability are “adequate.”

A potential improvement for oversight of offsite radiological emergency preparedness would be 

to define objective performance measures and describe their contribution to an overall safety 

outcome or consequence metric. That metric could involve target values for an averted dose of 

radiation, a maximum dose and the probability of exceeding it, or (as developed further in this 

report) a maximum amount of time available to implement a protective action, among other 

possibilities. While there could be debate on whether or not the target values provide the degree 

of safety desired, refocusing that debate on health and safety outcomes rather than inputs could 

enhance public understanding of what is meant by “reasonable assurance.”

2.5 ENHANCED FLEXIBILITY IN RESPONSE OPTIONS

Finally, a performance-based oversight system could offer the benefit of greater flexibility. As 

already noted, not being constrained to comply with requirements that do not directly support 

public health and safety outcomes may help both OROs and oversight agencies reduce waste 
8

by reallocating resources to more risk-significant activities. For oversight, this may require 

developing a range of response options dependent on the risk significance of any less-thantarget performance, rather than a single menu of options applicable to each and every instance 

of non-compliance or failure to perform as prescribed. 

Also as noted, the flexibility could allow for greater integration and alignment with all-hazards 

activities.

Yet more important for public health and safety may be that flexibility for OROs in how to 

respond (i.e., how to achieve a public health and safety performance outcome) could enable 

greater innovation and learning. Improvements and best practices are more likely to emerge for 

the RERP community where there is room for OROs to experiment and apply judgment to 

unique circumstances. A performance-based oversight system focused on outcomes could 

create even more freedom to foster innovation and improvement in ensuring public health and 

safety.
9

3. PERFORMANCE GOALS FOR OFFSITE RESPONSE

The goals for offsite radiological emergency response remain the same regardless of the 

oversight mechanisms involved. An RIPB oversight regimen should seek to ensure that: 

• State and local officials can immediately receive licensee notification of emergency 

conditions, understand the potential hazard, and alert or activate necessary response 

capabilities;

• State and local officials can independently assess radiological conditions in support of 

protective action decision-making (e.g., dose projections, field monitoring);

• State and local officials can make timely protective action decisions (PADs) appropriate 

to the emergency conditions;

• State and local officials can rapidly alert and inform the public regarding emergency 

conditions and appropriate protective measures in a way that supports State and local 

implementation of those measures;

• State and local officials can implement PADs in a timely manner to eliminate or 

substantially reduce public radiation exposure; and

• All persons in the EPZ are accounted for in PADs, including emergency workers, the 

non-English speaking populations, persons with access and functional needs, and 

persons under the care of others in facilities (e.g., schools, prisons, health care facilities, 

etc.). 

11

4. DESIGN CONSIDERATIONS FOR A REVISED REGULATORY 

REGIMEN 

4.1 STAKEHOLDER ANALYSIS

Successfully implementing any change will require satisfying the needs of the community of 

stakeholders and giving the stakeholders the opportunity to participate in the development of the 

change. The NRC’s interests in exploring change are reflected in Chapter 2, above. The 

discussion below reviews some potential interests of the community of stakeholders beyond the 

NRC (e.g., NRC licensees, State and local governments, the general public, and FEMA) in the 

current system, along with how those interests may translate to design considerations. All 

stakeholders are assumed to share an interest in safety.

4.1.1 Nuclear Regulatory Commission Licensees

NRC licensees’ stake in ORO oversight is: (a) to maintain their operating license, and (b) to 

maintain the confidence of surrounding communities in the ability of the licensee and the ORO 

together to ensure safety in the event of an accident. Licensees also pay fees to fund Federal 

and, typically, State and local efforts to maintain offsite radiological emergency response 

preparedness. For NRC licensees, design considerations may be:

• Stability of Requirements. The nuclear industry prefers stable requirements. The Nuclear 

Energy Institute’s (NEI) initial 2008 reaction to developing a performance-based 

oversight regimen was that the current approach had been “successfully implemented 

for many years.” NEI noted that other regulatory changes were underway, and asked 

that a new approach be considered only after those changes had been implemented

(see Nelson 2008).

• Stability of Results. NEI also noted that for onsite emergency preparedness programs 

there were “relatively few issues noted” and these could be addressed in a timely 

manner (Nelson 2008). Similarly, few offsite deficiencies identified by FEMA lead to 

enforcement actions by the NRC against licensees. While the nuclear industry has a 

strong interest in demonstrating the safety of its operations, industry has no direct 

control over offsite response. Industry likely will not want outcome-oriented performance 

standards that prove overly stringent, leading to an increase in deficiencies. While 

transparency is a benefit of objective performance metrics, it also poses a risk.

• Avoidance of Cost Increases. The nuclear industry faces strong competition from other 

power generation sources and faces pressure to contain costs. Industry is unlikely to 

embrace a new oversight regimen for offsite radiological emergency preparedness if 

additional costs are required to implement and support it. Providing reasonable 

assurance based solely on performance demonstrations could require an increase in the 

number of such demonstrations, and it is as yet undetermined whether the reduction in 

costs for compliance monitoring (plan reviews, etc.) would offset any increase in 

demonstration costs. 

• Flexibility and Performance Focus. Related to stability of requirements, industry has 

suggested that the requirements be made less prescriptive and more results-focused. 

“The NUREG [NUREG-0654] is prescriptive in that it defines elements—and in some 

cases exact methods—for implementation of the Planning Standards. ... In order to 

safeguard document longevity, the [revised] NUREG guidance should be flexible and not 

overly prescriptive. This flexibility should allow for alternative approaches to achieve the 

same result and enable the use of evolving technology” (Perkins-Grew 2013).
12

4.1.2 State and Local Governments

State and local governments influence safety oversight by: (a) providing input into planning and 

design and (b) complying with the requirements using available resources.1 Given budget 

pressures on State and local governments, States have an interest in ensuring the level of 

funding provided by licensees for radiological emergency preparedness is not reduced due to 

any savings from efficiency, where needs still remain. State and local governments also must 

balance requirements for radiological emergency preparedness with preparedness 

requirements for other hazards. For State and local governments, design considerations may 

include:

• Continuity. Most State and local commenters on possible revision of NUREG-0654 did

not call for wholesale change. One commenter, echoed by two others, noted that the “16 

planning standards as they currently exist in NUREG-0654 are still applicable” (Mulligan 

2013).

• Streamlined, Rationalized Requirements. Some comments noted duplicative

requirements (e.g., providing information or demonstrations for plan reviews, exercises, 

and ALCs) or lack of clarity on requirements (e.g., having to provide lists of trainings 

conducted without an approved training curriculum to evaluate against). See for example 

Engelhart (2013). 

• Cost Concern. A new oversight regimen should attempt not to place significantly more 

burdens (financial, time, or opportunity cost) on State and local government responders, 

nor reduce the resources available to maintain radiological safety. “The States must 

prioritize their limited resources to ensure adequate focus on the required items” (Klinger 

2013). It is unclear whether cost increases from conducting exercises and drills more 

frequently would be fully offset by a reduction in compliance costs and in the scope of 

the exercises (e.g., from large-scale exercises to task-focused drills and demonstrations, 

emphasizing the most risk-significant tasks).

• Flexibility. In commenting on possible revision of NUREG-0654, one State requested “an 

evaluation and approval process that is flexible enough to apply to future technologies, 

so that regulatory documents do not have to be revised to allow [their] usage.” This State 

also noted that each site has unique circumstances (Engelhart 2013). Another State 

pointed out that FEMA’s REP Program Manual mentions “alternate forms of 

demonstration” to meet requirements and requested that a revised NUREG-0654 

provide for and discuss these acceptable alternatives (Bear 2013).

• Input. Many State commenters on possible revisions to NUREG-0654 stressed the 

importance of State involvement in defining the oversight requirements. An extension of 

this may be to expand State input into the determination of oversight assessments, such 

as allowing States themselves to highlight where potential deficiencies exist and propose 

corrective actions. 

4.1.3 General Public

The general public has mixed views on the safety of nuclear power.2 To the extent the public 

equates radiological emergency preparedness with being able to evacuate quickly, growing 

 1 States are preempted from regulating radiological safety aspects of NPPs (as opposed to need, 

reliability, or cost of nuclear power). See Congressional Research Service (2011). Also, State and local 

governments cannot preclude “reasonable assurance” determinations by opting not to participate in the 

oversight process per 10 CFR 50.47(c)(1). 

2 See NEI (2012) and Yale (2012) for contrasting poll results on the U.S. public’s safety perceptions. NEI 

reports a September 2012 poll showing 17 percent find plants “unsafe” to some degree. Yale reports a 
13

populations in EPZs and increased traffic congestion on road networks sow doubt. Advocacy 

groups and public commenters in the environmental impact study process for NPP license 

renewals often express these doubts in performance terms. They question whether the plan 

reviews and exercise demonstrations used for “reasonable assurance” determinations actually 

mean the licensee, ORO, and community could implement an evacuation in time for the EPZ 

population to be “safe.” What the NRC and FEMA mean by “reasonable assurance” may not 

equate to “safe” in the public’s mind and may need to be defined more explicitly. How 

determinations are made should also be understood, to aid in gaining trust and confidence in 

the process. Resulting considerations are: 

• No Loss of Useful Information. Changes to the offsite regulatory regimen should not 

eliminate the ability to gain information on capability enablers (i.e., plans, organization, 

equipment, and training) in addition to performance, where performance does not create 

confidence. Such information may be useful for interpreting performance and developing 

corrective actions.

• Transparency. Any changes made should enhance transparency and the ability to 

understand the “reasonable assurance” determination.

4.1.4 Federal Emergency Management Agency

FEMA retains responsibility for oversight of State and local OROs’ radiological emergency 

preparedness, in coordination with the NRC. FEMA has explored changes (e.g., the Strategic 

Review in the late 1990s) and adapted elements of the regimen to address all-hazards 

imperatives (e.g., adoption of elements of the Homeland Security Exercise and Evaluation 

Program, or HSEEP). FEMA has an interest in ensuring it can maintain effective RERP-specific 

oversight within other all-hazards requirements and in seeing the results of its oversight efforts 

aligned with the NRC’s overall oversight process. Considerations include:

• Balance and Linkage between Onsite and Offsite Emergency Preparedness Oversight. 

FEMA has an interest in ensuring the oversight approaches and requirements for onsite 

and offsite emergency preparedness support are consistent with, and do not undermine,

one another.

• Integration with All-Hazards Policy. FEMA’s RERP oversight must focus on the elements 

necessary to ensure preparedness specific to the radiological hazard to public health 

and safety from commercial NPPs. However, FEMA as a whole needs to understand 

and report to Congress whether the Nation is prepared for all hazards under terms of 

Presidential Policy Directive 8 and its National Preparedness Goal (DHS 2011a). Where 

consistency is impractical, quantitative and objective results may be easier to translate 

from one construct to the other. 

• Stability. Implementation of changes should not undermine confidence in current 

“reasonable assurance” determinations.

• Cost Concern. In a tight budget environment, FEMA shares an interest with State and 

local governments in not facing a significantly greater burden (financial, time, or 

opportunity cost) under a new oversight regimen or a reduction in resources available to 

maintain radiological safety. 

 

May 2011 poll (shortly following the Fukushima incident) in which 53 percent think of nuclear power as a 

“disaster” or “bad,” compared to 34 percent in 2005.
14

4.2 PROPOSED CHARACTERISTICS OF A REVISED REGULATORY REGIMEN

Design considerations following from an examination of potential stakeholder interests are 

provided below. 

4.2.1 Transparency in Methodology for Determining Reasonable Assurance

• Objective Measures of Performance. To the extent feasible, assessment of offsite 

radiological emergency preparedness will involve objective measures of performance 

(and capability as necessary), to include both binary and quantitative measures of 

reliability and timeliness.

• Overarching Safety Goal. Measures ideally would help gauge contribution to an 

overarching safety goal or goals. The primary goal presented in this document is to 

ensure that an offsite radiological emergency response is able to ensure public health 

and safety during a radiological emergency. Although that goal could be expressed in 

terms of dose averted or received, this paper focuses on accomplishing a sequence of 

actions in time to avoid or minimize exposure. 

4.2.2 Flexibility 

• Focus on Performance. Oversight will focus on performance demonstrated during drills 

and exercises, with specificity in the competencies to be demonstrated and standards to 

judge success. 

• Flexibility of Data Inputs: Use of Non-REP Responses or Exercises as Proxies. Credit for 

demonstrated performance in a non-REP context may be given if it can be documented 

or otherwise validated that performance met or exceeded the relevant performance 

standard. 

4.2.3 Streamlining and Rationalization

• Focus on Risk-Significant Elements. Oversight will focus on elements most critical to 

ensuring adequate protection of public health and safety during the plume release phase 

of a radiological event. 

• Differential Levels of Scrutiny Based on Performance. Performance demonstrations may 

be supplemented with self-certified information including capability indicators (see 6.3.2, 

6.5, and Appendix B). However, reviews of the adequacy of capability elements 

(plans/procedures, organization/staffing, training, facilities/equipment) will occur only if 

adequate performance cannot be demonstrated. 

4.2.4 Resource Neutrality

• No Significant Net Change. Where resource information is available, the oversight 

program will be designed to involve approximately the same level of effort as is currently 

required. The focus and allocation of that effort will shift from compliance reviews of 

plans and training to demonstrations of task performance, and from all elements of 

overall RERP capability to those most critical to public health and safety. This will require 

more frequent demonstration of capabilities within a performance cycle through tightly 

scoped functional exercises and drills, rather than reliance on a large-scale biennial 

exercise.
15

4.2.5 Adequacy of Available Information

• Alternative, Proxy Indicators of Performance. While focus of oversight may shift to 

demonstrations of performance (i.e., what capabilities can actually do), it may not be 

feasible to perform some tasks in an exercise or perform them fully. For example, the 

scope of an exercise will not allow for actually monitoring 20 percent of the plume 

exposure EPZ population over 12 hours. Where necessary, proxy quantitative indicators 

of capability may be substituted for performance indicators. Indicators of potential 

challenges, such as 511 information on number of days on which major evacuation 

routes experienced delays, may also be considered if obtaining the information can be 

streamlined. 

4.2.6 Other Considerations

• Diversity of Scenarios. Performance demonstrations would continue to occur under a 

variety of scenarios, to avoid rote response and the possibility of “gaming” the oversight 

system. Performance demonstrations may include additional scenarios or variety in 

exercise design, including, but not limited to, exercising real weather conditions, 

exercising at different times of day, or reacting to a sudden radiological release without a 

gradual escalation of events. 

• Transition. The new regimen would replace the existing regimen and its requirements.3

Some transitional testing or pilot phase with selected licensees and State and local 

OROs likely would be required before adoption of the new regimen. Such a pilot could 

be conducted under the “alternative approaches and methods” provisions described in 

FEMA’s REP Program Manual.

• State Input. Following HSEEP principles, the new regimen will encourage inclusion of 

State ORO self-critiques for after-action reports (AARs). Depending on the level of 

performance in a given demonstration, analysis and corrective action may remain 

internal to the ORO, an input from the ORO for FEMA review, or a FEMA-led corrective 

action plan with State ORO input. States already perform some self-assessment for 

FEMA in ALCs and, in the all-hazards context, State Preparedness Reports (SPRs).

Additionally, OROs that identify a weakness in performance during the demonstration of 

a capability may be able to re-start the performance demonstration without penalty. 

 3 This would be in contrast to the initially proposed approach for onsite oversight in the NRC’s “Elements 

of a Performance Based Emergency Preparedness Regulatory Regimen” (ADAMS Accession Number 

ML080440163), p. 3. The initial thought was that a new performance-based approach for onsite oversight 

could be made voluntary. However, maintaining two regimens in parallel permanently could be 

burdensome for FEMA to implement. It could be confusing for OROs seeking to comply. It could also 

undermine public confidence in “reasonable assurance” determinations under at least one if not both of 

the regimens.

17

5. ELEMENTS OF RERP AND THEIR RISK SIGNIFICANCE

5.1 RADIOLOGICAL EMERGENCY RESPONSE LOGIC

Defining a performance-based oversight system for RERP requires an understanding of what 

tasks the ORO must perform to protect public health and safety. This section describes a 

simplified, high-level logic for radiological emergency response to frame RERP oversight. 

Radiological emergency response relies on recognition of an emergency by the licensee. Under 

Appendix E to 10 CFR 50, “nuclear power reactor licensees shall establish and maintain the 

capability to assess, classify, and declare an emergency condition within 15 minutes after the 

availability of indications to plant operators that an emergency action level has been exceeded 

and shall promptly declare the emergency condition as soon as possible following identification 

of the appropriate emergency classification level.” There are four emergency classification 

levels per NRC Bulletin 2005-02:

• Notification of Unusual Event. Events are in process or have occurred that indicate a 

potential degradation of the level of safety of the plant or indicate a security threat to 

facility protection. No releases of radioactive material requiring offsite response or 

monitoring are expected unless further degradation of safety systems occurs.

• Alert. Events are in process or have occurred that involve an actual or potential 

substantial degradation of the level of safety of the plant or a security event that involves 

probable life-threatening risk to site personnel or damage to site equipment because of 

intentional malicious dedicated efforts of a hostile act. Any releases are expected to be 

limited to small fractions of the Environmental Protection Agency’s (EPA) Protective 

Action Guideline (PAG) exposure levels.

• Site Area Emergency. Events are in process or have occurred that involve an actual or 

likely major failure of plant functions needed for protection of the public, or involve 

security events that result in intentional damage or malicious acts that could lead to the 

likely failure of, or prevent effective access to, equipment needed for the protection of 

the public. Any releases are not expected to result in exposure levels that exceed EPA 

PAG exposure levels beyond the site boundary.

• General Emergency. Events are in process or have occurred that involve actual or 

imminent substantial core degradation or melting with potential for loss of containment 

integrity or an actual loss of physical control of the facility due to compromised security.

Releases can be reasonably expected to exceed EPA PAG exposure levels offsite for 

more than the immediate site area.

Under Appendix E to 10 CFR 50, at IV.D.3, licensees also must have the capability to notify 

responsible State and local governmental agencies (i.e., the ORO) within 15 minutes after 

declaring an emergency. If the EPA PAG is expected to be exceeded, the notification must be 

accompanied by a protective action recommendation to the ORO.

The ORO must be able to receive the licensee notification and understand the nature of the 

emergency declared by the licensee. This is the beginning of offsite radiological emergency 

response, which proceeds in conceptual phases as shown in Table 1 on the next page.

Exposure concerns and appropriate protective actions evolve over the course of the response, 

as shown in Table 2 on the next page. However, radiological response is initially focused on 

avoiding the population’s exposure to external radiation from any plume, and from inhalation of 

material in that plume.
18

Table 1. Phases of a Radiological Release

(EPA 1992: 1-2)

Early Phase

(During Event to a Few Days After 

Event)

Intermediate Phase

(Follows Early Phase, After Incident 

Brought Under Control)

Late Phase

(Ends When Remediation Is 

Complete)

• PADs will be made with only 

preliminary situational 

understanding and data

• Actions should be taken quickly 

and modified as additional data 

become available

• Considerations: Plume 

exposure, short-term exposure 

to deposited materials, 

inhalation exposure

• Typically overlaps with early and 

late phases

• Considerations: Additional 

relocation or removal of public 

vs. allowing to return home, food 

and drinking water

• No longer “emergency 

response”

• Considerations: cleanup and 

recovery, decontamination, 

return and reentry

Table 2. Protective Actions and Exposure Pathways of Concern, by Phase

(EPA 1992: 1-4)

Phase Potential Exposure Pathways Possible Protective Actions*

Early External Radiation from Facility Sheltering, Evacuation, Access 

Control

Early External Radiation from Plume Sheltering, Evacuation, Access 

Control

Early Inhalation of Material in Plume

Sheltering, Administration of 

Potassium Iodide (KI),+ Evacuation, 

Access Control

Early/Intermediate Contamination of Skin and Clothes Sheltering, Evacuation, 

Decontamination of Persons

Early/Intermediate/Late External Radiation and Ground Deposition of 

Activity

Evacuation, Relocation, 

Decontamination of Land and 

Property

Intermediate/Late Ingestion of Contaminated Food and Water Food and Water Controls

Intermediate/Late Inhalation of Re-Suspended Activity Relocation, Decontamination of Land 

and Property

* While these are logically possible protective actions to reduce exposure via a given pathway, some are more 

relevant in practice. For example, access control would be the primary means to protect against external radiation 

from the facility. Decontamination of persons would be the primary means to address contamination of skin and 

clothes. Evacuation may serve to avoid exposure through ground deposition of activity, but generally relocation and 

decontamination are most relevant once ground deposition has been assessed.

+ Note that 50 CFR 10.47(b)(10) requires “consideration” of KI as a “supplement” to protective actions involving 

sheltering and evacuation, “as appropriate.” The NRC does not require or recommend KI, but recognizes the 

prerogative of States to decide this question.

Whatever the licensee’s recommendation, responsibility for making the protective action 

decision—PAD—rests with the ORO. Because the ORO has this responsibility, the ORO must 

be able to independently assess the situation in order to decide what measures are most 

appropriate to protect the public health and safety from excessive radiation exposure. The two 

primary options in the early phase are evacuation or shelter-in-place, although States may 

supplement these with KI to protect against uptake of radioactive iodine in the thyroid.1 Major 

considerations include: 

• Incident Progression and Plant Conditions. An emergency’s classification may evolve.

The Notice of Unusual Event can become an Alert if “further degradation of safety 

systems occurs,” and that degradation could progress to failure, and then loss of 

 1 Evacuation and shelter-in-place are not mutually exclusive. In some scenarios, it may be appropriate for 

some segments of the population to shelter in place first and then evacuate. For example, a plume may 

travel too quickly for evacuation to be accomplished in advance of arrival. People may shelter in place, 

then evacuate to reduce exposure to deposited materials.
19

containment integrity. If notified of a condition other than General Emergency, OROs are 

faced with deciding whether any precautionary actions should be implemented and 

communicated for any part of the potentially affected population.2 OROs rely on 

information from licensees regarding current and expected plant conditions, and the 

likely rate of change in those conditions.

• Time until Arrival of Plume vs. Evacuation Time Estimate (ETE). Evacuation will protect 

the public from receiving a significant dose from any airborne release if it can be 

completed before arrival of the plume. If evacuation of an area cannot be completed in 

time to avoid an unsafe dose, shelter-in-place may be preferable initially.

• Duration of Release vs. Shelter Protective Factors. Sheltering will reduce the gamma 

exposure rate from deposited materials, but it is not a suitable protective action for this 

pathway for a long-duration exposure. Sheltering will eventually be followed by 

evacuation out of the EPZ for the affected population in the plume exposure pathway.

Any PAD must be supported by instructions to the population segments expected to implement 

the PAD, as well as by the mobilization of responders who assist in its implementation (e.g., to 

support traffic management, access control, etc.).

Once people are in a place safe from external radiation from the plume or inhalation of material 

in the plume, radiological emergency response focuses on attending to their needs and 

managing their risks of additional exposure and contamination, to stabilize the situation. Areas 

of safety—and restricted areas—may be better defined over time as radiological monitoring 

information becomes available. Over time, radiological emergency response efforts focus on 

reclaiming more of the impacted area for unrestricted use, where achievable, and shift to 

community recovery. 

5.2 RADIOLOGICAL EMERGENCY RESPONSE TASKS AND RISK SIGNIFICANCE

Figure 4, on the next page, shows the high-level tasks described in the foregoing section and 

an assessment of their risk significance for public health and safety.

The most risk-significant tasks for offsite radiological emergency response focus on avoiding the 

population’s exposure to the plume and the material it contains. Collectively, these tasks can be 

highly time-sensitive, depending on the scenario. Successful performance of these tasks 

reduces the challenge of managing subsequent risks of exposure and contamination, such as 

through access control or decontamination of individuals.

Managing those subsequent risks, along with reception and screening of the evacuated 

population, have medium risk significance. They address marginal exposure risks after the 

population is safe from the danger posed by the plume and material deposited on the ground.

Late phase actions focused on cleanup and recovery have low risk significance because 

immediate health and safety needs of the population have been addressed at this stage. Return 

 2 NUREG/CR-6953 concluded that “precautionary efforts during Site Area Emergency are prudent.” See

pp. ix-x, 25, 64. The study noted that some instances could warrant early closure of schools, parks, 

government facilities, etc., at the Site Area Emergency, and early notification of the general population 

within the 10-mile EPZ to prepare for evacuation.
20

Figure 4. Radiological Emergency Response Tasks and Risk Significance

to the impacted area is not a requirement for individual physical health and safety, nor is 

broader community recovery, although both may be important psychologically.

This tiering of tasks by risk significance for public health and safety informs which tasks receive 

more focused and frequent oversight under a revised oversight system, as will be discussed in 

the next chapter. The remainder of this chapter provides more description of these high-level 

tasks, grouped by their risk significance. 

5.3 HIGH RISK SIGNIFICANCE TASKS

• Receive Licensee Notification of Emergency. This response task requires OROs to have 

sufficient communications systems in place to receive messages from the NPP in the 

event of an emergency. OROs must also have sufficient staffing at all times to be able to 

receive and process information through those communications channels. The ORO 

should have a notification and warning point that receives the information. The 

notification and warning point may be a 911 Center, a duty officer, a designated 

communications center, or any other officially designated channel that can receive and 

disseminate information.

• Understand the Notification. ORO decision-makers, as well as the ORO notification and 

warning point, should be sufficiently trained in REP to understand the current emergency 

classification system (Notification of Unusual Event, Alert, Site Area Emergency, 

General Emergency), and the potential implications of an emergency notification. Based 

upon existing plans and procedures, OROs must also be able to take the appropriate 

next steps to notify other responders and begin incident response.

• Notify the Appropriate Officials. The notification and warning point begins incident 

response by contacting previously identified decision-makers from State and local 

OROS. The decision-makers involved may vary depending upon existing plans and 
21

procedures. In the event that a primary contact cannot be reached, secondary or tertiary 

contacts should be established to enable incident response to proceed in a timely 

manner.

• Assess the Situation. Once necessary decision-makers and OROs have been activated, 

an identified technical advisor or public health expert (who is trained to interpret the 

information being provided from the facility) will take the lead in processing information 

received from the plant, as well as information observed by the OROs, and synthesize it 

into an accurate assessment of the situation given the current information. OROs must 

be able to produce a projection of the plume based upon wind and weather conditions, 

and use this plume projection to identify the at-risk population. OROs should also 

demonstrate an established capability to monitor the environment outside the facility 

boundary for any radiation that may have been released above ground. The technical 

advisor must demonstrate the ability to develop a protective action recommendation 

based on an understanding of information available. 

• Make a Protective Action Decision. OROs must decide the best way to protect the public 

given the nature of the incident and the resources available to them. The licensee 

provides protective action recommendations to the ORO, which may then be accepted 

or revised by the ORO. Protective action decision-making typically centers on identifying 

which zones within the 10-mile EPZ to evacuate and which to shelter-in-place. Plant 

conditions, weather and wind direction, and ETEs for the potentially affected area all 

factor into the decision. Use of KI is another consideration for protective action decisionmaking, when applicable.

• Mobilize Response Components. In assessing the situation and determining required 

protective actions, OROs should gain a better understanding of the response 

components that will need to be involved in implementing the PAD and follow-on 

activities. OROs will need to mobilize firefighters, police, emergency medical technicians 

(EMTs), and other response elements. Sufficient response personnel should be 

available to fulfill requirements of the PAD. Responders may be called to a variety of 

subtasks, including traffic control points and activation of reception facilities and 

decontamination stations. Mobilization may be concurrent with or following protective 

action decision-making. 

• Notify the Public. Once a PAD has been determined, the public must be informed of the 

steps that they should take in order to ensure their own safety. Notification of the public 

is a responsibility of the State and local OROs. OROs must be able to alert and 

disseminate a prompt message to the EPZ population that there is an emergency (e.g., 

with sirens and Emergency Alert System (EAS) messages). The method of 

dissemination is less important than achieving coverage of the population. Any such 

prompt message must be followed quickly by a clear, coherent, and complete message 

providing instructions for implementing the PAD. A Joint Information Center should be 

established so that the licensee and ORO may collaborate and deliver a unified 

message. 

• Implement Protective Action Decision. State and local OROs should implement the PAD 

in an efficient and timely manner. ORO primary responsibilities for facilitating evacuation 

include traffic management and ensuring transportation for persons requiring assistance. 

The ORO may seek to confirm implementation of the PAD. PAD implementation should 

account for all persons in the 10-mile EPZ, including emergency workers, non-Englishspeaking population, persons with access and functional needs, and persons under the 

care of others in facilities (e.g., schools, prisons, health care facilities, etc.). 
22

5.4 MEDIUM RISK SIGNIFICANCE TASKS

• Receive, Screen, and Support Evacuees. Once evacuation has been initiated, local 

OROs must ensure that evacuees’ basic needs are met. A first need is screening 

individuals to determine if decontamination is necessary; contaminated individuals are 

not allowed into public shelters. Counties should have sufficient access to shelters to be 

able to accommodate displaced individuals from the plume zone in the short term. Local 

OROs should also have sufficient stockpiles of food and water to feed evacuees in the 

short term. Where an ORO cannot meet these functions on its own, it should have 

resource sharing agreements in place with other jurisdictions.

• Manage Exposure and Contamination Risks. Local OROs must be prepared to 

decontaminate individuals and their belongings as they evacuate the EPZ. Local OROs 

must also be able to decontaminate response vehicles and equipment that exit a 

radiation control zone. State and local OROs must also restrict access to evacuated 

areas until they are deemed safe for reentry. Finally, OROs may seek to mitigate the 

potential for ingestion of contamination by having livestock placed on stored feeds and 

uncontaminated water, and restricting movement of animals and agricultural products. 

5.5 LOW RISK SIGNIFICANCE TASKS

• Make Area of Impact Safe for Return. This function calls for OROs to be able to 

determine, for specific sites and their intended uses, that an acceptable level of 

radioactivity has been achieved before the sites can be reopened for return. This may 

require site decontamination including the removal of topsoil. This response task will not 

be evaluated in the oversight regimen.

• Facilitate Community Restoration and Recovery. Once the goal of ensuring public health 

and safety has been met, the OROs must facilitate the return and recovery of displaced 

individuals and businesses. This may include financial assistance, construction 

assistance, or other assistance as needed to return a displaced population to normal 

functionality. The Federal Government may provide technical and other assistance 

during this time. This response task will not be evaluated in the oversight regimen.

A risk-informed oversight regimen allocates resources and attention on tasks that bear most 

directly on ensuring public health and safety. Return and restoration are important, but 

presuppose an already healthy and safe population that must make choices about long-term 

and less acute risks. Therefore, this framework does not provide for performance-based 

oversight of these tasks, beyond requiring their periodic consideration and discussion by OROs.

OROs evaluate these tasks internally and may address them more frequently than required.
23

6. PERFORMANCE-BASED OVERSIGHT SYSTEM FOR

RADIOLOGICAL EMERGENCY RESPONSE PROGRAM

If a risk-informed oversight system focuses on the RERP elements most critical to the protection 

of public health and safety, a performance-based oversight system focuses most on whether 

those elements do what is needed to protect public health and safety. This is different from 

whether there are plans to perform tasks, an adequate quantity and/or quality of personnel and 

equipment to perform tasks to the extent and level required, or an organization of those 

resources. These are elements of capability—being able to do. The question of interest for 

performance-based oversight is whether the capability performs. 

This chapter outlines a system for developing “reasonable assurance” that ORO RERP 

capabilities perform both reliably and timely, as appropriate. It addresses:

• Decomposition of High-Level Tasks. Tasks presented in Chapter 5 above are divided 

into subtasks to facilitate observation, analysis, and measurement.

• Methods, Frequency, and Conditions for Performance Demonstration. Methods for 

OROs to demonstrate subtask performance are described, where possible and 

appropriate. Given risk significance of the parent tasks, frequency of required 

demonstration is also defined preliminarily. Final determination of specific frequencies 

and methods should be subject to a more detailed risk assessment in the future, to flesh 

out the conceptual framework provided here. 

• Performance Indicators and Acceptable Performance. Preliminary measures are defined 

for acceptable or target performance of subtasks. Reliance on existing measures is 

discussed.

• Aggregate Timeliness of Performance. The report provides a tentative construct for 

rating overall performance on timeliness for protective actions. This is critical for an 

intelligible reasonable assurance determination.1

• Alternative, Proxy Indicators for Performance. For subtasks not amenable to 

performance demonstration, alternative metrics are proposed.

• Issue Handling and Differential Levels of Oversight. The report proposes how to identify 

and manage issues of less-than-acceptable performance, and how the proposal may 

enhance consistency of issue identification. The proposed system provides for 

gradations of unacceptable performance.

• Enhancement of Risk Focus. The report describes how focus on risk-significant 

elements should increase under a revised oversight process.

6.1 DECOMPOSITION OF TASKS INTO SUBTASKS

Each task identified in Chapter 5 of this paper consists of multiple subtasks. However, to 

streamline oversight, only subtasks necessary and critical for preservation of public health and 

safety are included in task “checklists.” The complete subtasks and checklists, including 

proposed performance indicators for the subtasks, are in Appendix A. Each subtask is 

organized in a similar manner to NEI 99-02, which discusses onsite facility preparation and 

response. Each subtask has a name, a definition, measurement criteria (including data reporting 

elements and clarifying notes), and an initial effort at calculating the measurement. 

 1 Projected dose could also be the basis for an overall assessment of performance, but that option is not 

developed in this paper.
24

Tasks and subtasks that occur in the early steps of the emergency response process tend to be 

simpler in scope. The focus of activity shifts from inward and management-focused to outward 

and community-focused as an incident progresses. As a result, emergency response 

performance becomes more complicated and difficult to measure.

6.2 METHODS AND FREQUENCY OF PERFORMANCE DEMONSTRATION

Performance is best tested in exercises and drills (and real-world response), using objective, 

measurable standards of performance where feasible. Scheduled exercises will continue to 

operate in eight-year cycles, incorporating a number of scenarios and conditions to test OROs’ 

performance of tasks. 

A key consideration for a performance-based oversight regimen is how often and under what 

conditions performance must be demonstrated in order to provide reasonable assurance that 

the ORO can be relied upon to perform the task adequately in a variety of emergency situations.

Success in a single demonstration may not be an adequate basis to judge. For example, such a 

demonstration may have involved the one person on one shift who is highly skilled in a task, 

rather than a less adept individual on another shift who may require training or review of a 

procedure. There will need to be multiple communications tests, drills, and exercises, at 

different times of day, and sometimes with limited or no notice, in order to provide a more 

accurate assessment of an ORO’s RERP performance. Emphasis would shift from large-scale 

exercises with long lead times and broad scope, to more limited functional exercises, taskfocused drills, and communications tests.

To determine the ideal minimum frequency for these performance demonstrations may require 

more detailed analysis. In the interim, this paper proposes that a biannual (i.e., twice-yearly) 

functional exercise requirement focused on PADs and public messaging will generate an 

adequate number of observations for reasonable assurance in these task areas, with additional 

communications tests and drills for field teams.

Generally, tasks with high risk significance would be tested more frequently than tasks with 

lower risk significance. For oversight purposes, this generates more observations on the tasks 

most critical to ensuring public health and safety. Yet as an additional benefit, it requires OROs 

to spend relatively more time testing and maintaining proficiency in the most risk-significant 

tasks, as well. This should increase the reliability of the ORO and thereby reduce risk.

Practically, the desired frequency of demonstration must be balanced with the cost of 

demonstrating a specific subtask. For example, a large registration or decontamination drill that 

requires significant resources and volunteer participation should occur less frequently than a 

simple communications drill of equal risk significance that might require only a phone call. This 

is of particular importance if one goal for implementation of any new oversight regimen is not to 

unduly increase the existing resource burden of oversight.

Table 3 on the next page shows a preliminary, qualitative approach for the determination of 

frequencies of subtasks, based on risk significance and resource requirements. Table 4 (on the 

following pages) shows the proposed methods of and frequency for demonstrating performance 

of the subtasks. Methods include communications and notifications tests in addition to drills 

(performance of tasks regardless of scenario) and exercises (performance of tasks within the 

context of a given scenario). Frequencies proposed in this table are preliminary and may merit 

more detailed study. A final determination should require capabilities to be exercised frequently 

enough to generate confidence that the ORO would be able to perform a subtask during a realworld emergency, and retain the skill between demonstrations.
25

Table 3. Proposed Frequency of Subtask Demonstrations Based on Resource 

Requirements and Risk Significance

Resource Requirements 

for Single Demonstration 

of Subtask

Risk Significance

High Medium Low

High Quadrennially Cycle Internally evaluated by 

ORO at ORO’s discretion

Medium Biannually Annually Internally evaluated by 

ORO at ORO’s discretion

Low Quarterly/Monthly Biannually/Quarterly Internally evaluated by 

ORO once per cycle

Table 4. Methods for and Frequency of Subtask Demonstrations

Task Subtask Demonstration Type Recommended 

Minimum Frequency

Receive Licensee 

Notification of 

Emergency

Primary Communications Comms Test Monthly

Secondary Communications Comms Test Monthly

24-Hour Warning Point Comms Test Monthly

Alternate Communications 

Center Exercise Quadrennially

Understand the 

Notification

Message Comprehension Drill (may be combined with 

comms test) Monthly

Secondary Official Message 

Comprehension

Drill (may be combined with 

comms test) Monthly

Notify the Appropriate 

Officials

Timely Notification Comms Test/Within 

Exercise Monthly

Functional Communications Comms Test Monthly

Assess the Situation

Radiological Expertise Within Exercise Biannually

Contact Licensee Within Exercise Biannually

Monitor Radioactivity Drill Quarterly

Weather Evaluation Within Exercise Biannually

Plume Mapping Within Exercise Biannually

Risk Mapping (Dose Projection) Within Exercise Biannually

Make a Protective Action 

Decision

Consult with Experts Within Exercise Biannually

Make Protective Action Decision Exercise Biannually

Road Mapping Within Exercise Biannually

Evacuation Mapping Within Exercise Biannually

Shelter-in-Place Mapping Within Exercise Biannually

KI Mapping Within Exercise Biannually

Mobilize Response 

Components

Contact Emergency 

Responders Comms/Notification Test Quarterly

Activate Responders Mobilization Drill Quadrennially

Response Support Drill or Within Exercise Quadrennially

Notify the Public

Mass Notification Exercise Biannually

Develop Follow-On Instruction 

Message Exercise Biannually

Notification of Non-EnglishSpeaking Population Exercise Biannually

Establish Joint Information 

Center Exercise Biannually

Special Needs Populations Exercise Biannually

Implement Protective 

Action Decision

Evacuation N/A N/A

Shelter-in-Place N/A N/A

KI Distribution N/A N/A

Emergency Worker Protection Exercise Biannually
26

6.3 PERFORMANCE INDICATORS AND ACCEPTABLE PERFORMANCE

6.3.1 Concepts for Performance Indicators

Following the determination of subtasks, objective performance measures were developed 

where possible for each subtask. These are available in Appendix A. Three types of measures 

were used:

• Activity or Process Measures. Process measures assess the performance of a process 

or activity (i.e., whether the ORO was able to perform a given task). For example, did the 

ORO’s communications center answer when contacted? 

• Output Measures. Output measures are assessments of the results of an activity or 

process that can be measured quantitatively. Quantitative output measures may be 

framed to give pass-fail determinations by asking if the ORO produced the expected 

output x for a given subtask. For example, did the ORO contact over x percent of 

farmers in the ingestion pathway EPZ with a stored feed advisory?

• Efficiency or Time-Based Measures. These measures assess how quickly an ORO was 

able to perform a subtask, or how much output the ORO was able to achieve in a unit of 

time. For example, did the ORO generate a message for delivery to the public within 15 

minutes of notification of an emergency? Or, was the ORO’s evacuee screening 

throughput in one hour adequate to screen x percent of the EPZ population in 12 hours?

Because the focus is on performance, the measures do not reference compliance with plan 

provisions. Measures focus solely on achieving a defined output or outcome. To address 

reliability, several of the measures are extended by requiring frequent measurement over time 

(e.g., monthly communications tests are successful over x percent of the time). Proposed 

thresholds are not included for all indicators, and in many cases the development of appropriate 

threshold levels will require additional study.

Receive, Screen, and 

Support Evacuees

Reception Center Operation Drill Annually

Radiological Monitoring Drill Annually

Register Evacuees Drill Annually

Congregate Care Drill Cycle

Manage Exposure and 

Contamination Risks

General Decontamination Drill Annually

Emergency Worker 

Decontamination Drill Annually

Evacuation Zone Access 

Control Drill Annually

Secure 

Contaminated/Restricted Zones Drill Cycle

Nuclear Facility Access Control 

Support Drill Cycle

Identify Agricultural 

Contamination Within Exercise Annually

Stored Feed Advisories Within Exercise Annually

Make Area of Impact 

Safe for Public Return N/A

Exploratory/Ungraded 

Tabletop Exercise

(internally evaluated by 

ORO)

Cycle

Facilitate Community 

Restoration and 

Recovery

N/A

Exploratory/Ungraded 

Tabletop Exercise

(internally evaluated by 

ORO)

Cycle
27

6.3.2 Modification of Existing Radiological Emergency Preparedness Program Criteria

FEMA’s REP Program Manual (FEMA 2013f) offers multiple measures and criteria to test 

compliance with the 16 existing planning standards. Most are not quantitative and rely on 

qualifiers such as “timely” or “appropriate.” Exceptions to this include:

• Assessment Subelement 1.d.1. At least two communications systems are available and 

at least one operates properly.

• Assessment Subelement 4.b.1. ORO can field at least two field monitoring teams.

• Assessment Subelement 5.a.1. Primary alert and notification covers essentially 100 

percent of the EPZ.

• Criterion 5.a.4. FEMA and the NRC recommend that OROs and operators establish 

means that will reach those in approved exception areas within 45 minutes once the 

initial decision is made by authorized offsite emergency officials to notify the public of an 

incident.

• Criterion 6.a.1. Staff responsible for the radiological monitoring of evacuees must 

demonstrate the capability to attain and sustain, within about 12 hours, a monitoring 

productivity rate per hour needed to monitor the 20 percent EPZ population planning 

base. The monitoring sequences for the first six simulated evacuees per monitoring 

team will be timed by the evaluators to determine whether the 12-hour requirement can 

be met.

• NUREG-0654 Criterion I.9. Each organization shall have a capability to detect and 

measure radioiodine concentrations in air in the plume exposure EPZ as low as 10-7

microcuries per cubic centimeter (μCi/cc) under field conditions.

Criteria from the REP Program Manual were adapted in two ways. 

First, some REP Program Manual criteria required only minor changes in wording and 

recombination to support a subtask. For example, a number of criteria address the ability to 

identify contamination in the environment. These include requirements for evaluators to review 

calibration logs, review use of approved equipment, ensure the correct amount of equipment, 

ensure the correct number of responders, and conduct tests suitable to measure radioiodine 

concentrations in the environment. While all of these indicators may be important, proposed 

measures focus on final performance. The adapted measure reads: 

ORO must be able to activate and provide field monitoring teams at random times over a 

given period of time. To measure this, a series of unannounced tests must be performed, in 

which the ORO is required to provide at least two offsite monitoring teams with capability, 

equipment, and knowledge to accomplish a reading. During demonstration, the ORO should 

be required to detect elevated levels of radioiodine concentrations in the air to an amount as 

low as 10-7 μCi/cc within one hour of notification, even with interference from noble gases 

and background radiation.

Secondly, and more commonly, quantitative measures were proposed for areas FEMA currently 

assesses qualitatively. These measures were also framed in terms of performance rather than 

compliance with plan provisions. These include many measures related to activities conducted 

in the Emergency Operations Center (EOC), such as mapping, procuring expert advice, or 

implementing resource agreements. 
28

An initial review of the REP Program Manual seeking ideas for both potential performance 

measures and capability indicators is in Appendix B. The proposed measures in Appendix A are 

a later evolution of this initial review; Appendix B documents consideration of the existing 

criteria.

6.4 AGGREGATE TIMELINESS OF PERFORMANCE

The tasks, subtasks, and performance indicators proposed would simplify the existing oversight 

process while refocusing it on performance. To the extent the tasks and subtasks represent the 

most necessary and critical actions to be performed for protecting public health and safety, 

failure to perform any task at a target level should trigger action, up to and including 

reconsideration of “reasonable assurance.” Section 6.6, below, describes how the proposed 

RERP oversight system would define and handle issues of less than acceptable performance 

on task areas.

However, subtask performance is measured in different ways (timeliness, reliability on multiple 

trials, binary yes/no output measures). Those measurements can be combined qualitatively to 

produce a bottom-line judgment of reasonable assurance. Yet without a single unifying metric, 

that bottom line may remain difficult to communicate effectively to the public and other 

stakeholders, including OROs and licensees seeking to enhance performance.

The bottom line for public health and safety in RERP is whether, and to what extent, RERP 

efforts can ensure members of the public avoid receiving any dose of radiation as a result of a 

radiological emergency or, failing that, any dose in excess of accepted PAGs regarding what 

would constitute an unhealthy dose. However, as suggested in Table 4 above and discussed 

more fully in Section 6.5 below, performance on the most important element of RERP—

implementation of the PAD—is the most difficult to measure. 

6.4.1 Construct of Problem

As discussed in Section 5.1, PADs depend on consideration of time. Figure 5 illustrates that for 

protective actions to be successful in avoiding exposure, the sequence from licensee notification 

of emergency, through making a PAD, notifying the public of the PAD, and implementing that 

PAD, must take less time than estimated for arrival of the plume at the zone for which protective 

actions are being considered.

Figure 5. Time for Protective Action Decision-Making and Notification of the Public
29

As shown in Figure 6, below, zones for development of protective actions within the EPZ are 

defined by rings of two, five, and ten miles from the plant, which are then segmented.

Source: Sorenson and Vogt (2006)

Although Figure 6 gives the “ideal” zone arrangement, real-world zones are defined by streets, 

landmarks, and other easy-to-communicate features.

2 Usually in a General Emergency, the 

appropriate PAD is to evacuate the two-mile ring and people living in the five-mile zone(s) 

downwind and slightly to either side of the projected path of the release. The need to evacuate 

portions of the EPZ beyond five miles is assessed as the incident progresses. This is called 

“keyhole” evacuation. However, NUREG-0654/FEMA-REP-1, Rev. 1, Supplement 3 indicates 

that in a rapidly progressing incident, shelter-in-place could be the appropriate initial strategy for 

the two-mile ring and five-mile zones where the ETE exceeds a predetermined duration. 

6.4.2 Assessing Timeliness of Protective Action Decision-Making and Public 

Notification

Assessing OROs’ aggregate timeliness would occur in exercises. Focus of observation would 

be on the time to make a PAD and the time to notify the public of the PAD for each zone of the 

EPZ potentially affected by the scenario.

In order for this not to be rote and automatic, scenarios must be varied and not known to players 

beforehand. In an eight-year cycle, scenarios would include at least one hostile-action-based 

incident, one scenario not progressing to a release (but potentially requiring a PAD), and one

rapidly progressing emergency. The other scenarios would involve varied source terms. 

Additionally, real weather would be used. Extent of play would involve only EOC personnel, not 

field elements, given the flexibility required. The focus would be on decision-making and crafting 

appropriate messages. Extent of play would need to encompass multiple hours (at least four) to 

account for possible changes in wind direction.

ORO players would have control over the light-shaded elements in Figure 5: their decision time 

and notification time to the public. OROs would make decisions for the two-mile, five-mile, and 

 2 For an example, see Coates and Hines (2012).

Figure 6. Emergency Response Planning Areas in EPZ
30

ten-mile rings. Decisions and messages would need to be appropriate. (Other subtask 

indicators and requirements would be addressed in the exercise, although the focus here is on 

timeliness.) 

The scenarios would determine time to release (although the exact time would not be known to 

players initially). Weather conditions would determine time from release to arrival of the plume.

Existing ETEs would be used for the potentially affected zones.

6.4.3 Options for Assessing Results

Timeliness of the PAD process would be assessed against the specific scenario conditions.

Success would involve having appropriate decision-making and notification occur with enough

time to leave a buffer in all scenario conditions. Ability to do this consistently would add to 

“reasonable assurance” regarding RERP; results could be combined with assessment of the 

licensee’s ability to meet its requirements for notification and protective action recommendations

in a timely manner, to aid in overall “reasonable assurance” regarding preparedness to protect 

public health and safety. 

A RIPB RERP oversight system could also include an overall timeliness metric for the critical 

path in ensuring public health and safety: the sequence of activities from receiving licensee

notification to implementing the PAD.

Although a standard “design basis” decision and public notification time for all sites and 

scenarios may be desirable, it is likely infeasible, particularly if it is to ensure decision-making 

and public notification can address the worst case. Currently NUREG-0654/REP-1, Rev. 1, 

recommends that planning not address a single accident sequence. Consider that such a 

design basis might consist of: 

• Time to Release. This could be one hour, as given for rapidly progressing emergencies in 

current guidance (see NUREG-0654/FEMA-REP-1, Rev. 1, Supplement 3, p. 9).

3

• Time to Arrival. Per the National Renewable Energy Laboratory’s Wind Maps, average 

annual wind speeds in the U.S. at 30 and 80 meters do not exceed 10 mph. Time to arrival 

at the two-to-five-mile ring would then be 15 minutes, and at the five-to-ten-mile ring would 

be 30 minutes.

• Worst-Case ETE. From a set of ETEs available in 1981, NUREG/CR-1856 gives a 

maximum permanent population evacuation time to evacuate the 10-mile EPZ under 

adverse conditions of approximately 16 hours (the 75th percentile case was six hours). The 

2007 NUREG/CR-6953 said these ETE ranges were still applicable.4

If this were the design basis, protective action decision-making and notification of the public 

could occur instantaneously—and initial guidance would still be to shelter-in-place, even for 

ETEs as low as 90 minutes. It may be possible to develop a 90th-percentile or 80th-percentile 

case for scenarios and ETEs rather than a worst case (i.e., completion of decision-making and 

 3 Lower times are possible if not considered likely; NUREG/CR-6953 used a source term with 40 minutes 

until release from the time a General Emergency was declared. These values are much lower than in the 

State of the Art Reactor Consequence Analyses for Peach Bottom and Surry, which had containment 

failure in an earthquake scenario beginning at about the eight-hour mark (NRC 2013a). 4 Review of just four randomly-selected ETEs submitted to the NRC in 2013 found a worst case of 

approximately six hours. This was for 100 percent clearance of the zone, rather than 90 percent, and in 

adverse winter conditions. In no case was an ETE given under 80 minutes for any zone, even for clearing 

only 90 percent of the zone population.
31

public notification in X time would meet requirements for 90 percent of release scenarios and 

ETEs), but that is beyond the scope of this effort.5

6.4.4 Limitations

The aggregate timeliness metric would need to supplement, not supplant, the other measures.

FEMA and the NRC would still need to consider quality of the PADs, ability to mobilize 

supporting resources (assumed away in this metric), and other factors, potentially including 

projected dose.6 However, timeliness is a key data point to be captured and communicated for 

reasonable assurance. 

6.5 ALTERNATIVE, PROXY INDICATORS FOR PERFORMANCE

As indicated in Table 4 and Appendix A, useful performance-based metrics for assessing 

protective action implementation—evacuation, shelter-in-place, and KI—in an exercise or drill 

environment are not available. It is not feasible to obtain large-scale public participation to test 

performance in these areas. Obtaining reasonable assurance on protective action 

implementation will require other indicators. Potential proxy indicators for performance are 

discussed below.

6.5.1 Evacuation

Some jurisdictions will have real-world evacuation experience, whether for hurricanes in the 

Southeast, wildfires in the West, or hazardous materials incidents throughout the United States.

These jurisdictions may claim credit for the real-world implementation of evacuation. Evaluation 

should address whether the evacuation was of a scale to approximate a RERP-related 

evacuation, and whether it was completed within a reasonable timeframe for a RERP-related 

evacuation (e.g., was it accomplished in the same or less time as the ETE for an equivalent 

area of the EPZ?). The NRC has studied the applicability of non-RERP evacuations to RERP in 

NUREG/CR-6864.

However, not all ORO jurisdictions will have relevant real-world evacuation experience in the 

course of a RERP evaluation cycle. An indicator is necessary to track, if not the ability to 

implement an evacuation, then whether there can be confidence in the approved ETE and its 

 5 This also suggests an alternative metric: the ORO’s response time would allow for successful protection 

against X percent of the risk. But for setting a threshold timeliness target, research on evacuation times 

for other types of incidents may help scope what is feasible. One example is Mills et al. (1995). This 

research was initially focused on cases comparable to radiological transportation accidents, but 

broadened in scope to capture sufficient data points. Only three of the 66 cases examined had an 

evacuation radius of two miles or more. The maximum evacuation time for the 66 incidents was 10 hours.

6 Although this paper emphasizes an overall timeliness metric, the NRC’s Deductive Quantification Index 

(DQI) method effectively captures many of the necessary measurements that would need to be made for 

a dose-based assessment system. The DQI method was developed and used in a proof of concept 

application for two sites with several accident sequences at each site. The analyses performed compare 

the potential consequences of accident scenarios when a radiological emergency response plan is fully 

and effectively implemented, and quantify the value of emergency preparedness in terms of dose that the 

public avoids as a direct result of the emergency preparedness program. The tool is able to quantify, 

given a basic accident scenario, the dose effects of a lack of PAD, untimely PAD, or incorrect PAD in 

terms of the population dose. The DQI method could be incorporated into RIPB RERP oversight to help 

evaluate and determine the significance, in terms of dose, of different levels of performance in making 

PADs. See NUREG/CR-7160.
32

underlying assumptions.7 Under Appendix E to 10 CFR 50, the NRC already requires licensees 

to track Census annual population estimates and to advise if estimated population increase 

between decennial censuses would affect the longest ETE value for any zone in the two- or fivemile rings or for the overall 10-mile EPZ to increase by 25 percent or 30 minutes, whichever is 

less.

It may be possible to construct an evacuation feasibility index to monitor trends affecting the 

ability to evacuate. Such an index could address not only population, but also availability of 

transportation, conditions of the road network, and congestion (capacity relative to normal 

demand) of the road network, and theoretical capacity. The American Highway Users Alliance 

(2006) developed an evacuation capacity index, but it examined 37 urban areas. Indices not 

specific to evacuation have also been constructed for urban areas. There is a travel time index

(Texas A&M Mobility Institute 2012), and the Federal Highway Administration has developed 

travel time reliability measures within its monthly congestion reporting—but again, for urban 

areas. Data availability and frequency of updates could be issues for such an index.

Surveys regarding evacuation behavior are possible, but surveys are recommended for 

developing ETEs, which are to be updated at least every 10 years.

6.5.2 Shelter-in-Place

Shelter-in-place is the least burdensome protective action to implement. The issues of interest 

are whether the need to shelter in place is communicated, whether it is received, and whether 

there is compliance. Communication of a shelter-in-place PAD (where appropriate) is tested 

under a Notify the Public subtask for Mass Notification, which proposes a survey to test whether 

notification is received. Such a survey could be expanded to ask whether the recipient of the 

notification would shelter in place or contribute to a shadow evacuation. 

6.5.3 Potassium Iodide Distribution

For KI, the main available indicator would be an inventory. The current oversight system has 

provided that 

quantities of KI available and storage locations(s) will be confirmed by physical 

inspection at the storage location(s) or through documentation of current inventory 

submitted during the exercise, provided in the ALC submission, and/or verified during an 

SAV. Available supplies of KI must be within the expiration date indicated on KI bottles 

or blister packs. As an alternative, the ORO may produce a letter from a certified private 

or state laboratory indicating that the KI supply remains potent, in accordance with U.S. 

Pharmacopoeia standards.8

This does not address performance of distribution, but only availability of KI for distribution.

6.6 ISSUE HANDLING AND DIFFERENTIAL LEVELS OF OVERSIGHT

Performance-based oversight requires clear, objective performance measures. “Objective” does 

not always mean “quantitative.” Some subtasks involve demonstration of multiple elements, but 

these can be addressed with a question tree using simple binary (“yes/no”) questions.

 7 See NUREG/CR-7002 for the ETE development process. 8 FEMA (2013f), demonstration guidance subelement 1.e., p. III-34. Note that this criterion applies to 

emergency workers, institutionalized individuals, and, only where stipulated by the jurisdiction’s plans and 

procedures, the public (including transients).
33

Using objective performance measures for RERP oversight enables consistent evaluations 

across all OROs. Many indicators in the current oversight regimen allow for subjective 

determinations. While subjective judgment cannot be eliminated, the current oversight system 

may magnify its potential effects by offering only a small range of potential outcomes when 

issues are identified. Those are areas requiring corrective action (ARCAs) and deficiencies.

ARCAs are relatively moderate assessments that do not indicate a significant reduction to public 

safety. ARCAs require some corrective action to be made, and require a subsequent redemonstration of a capability, either during the exercise, or at a future scheduled date.

Deficiencies are assessments such that there would be a significant degradation in public safety 

should an emergency at an NPP. As noted in Section 2.4, FEMA’s determination that the 

radiological emergency plan (or the ability to implement it) is inadequate initiates a 120-day 

window to take action on deficiencies to FEMA’s satisfaction. Within that time the State must 

either correct the deficiency or provide a timeline for doing so. If after 120 days (or the agreed 

upon timeline) the deficiencies are not corrected, FEMA notifies the Governor, the NRC, other 

agencies, and the public that FEMA has withdrawn its finding that offsite plans and 

preparedness provide “reasonable assurance” that appropriate protective measures can be 

taken in the event of a radiological emergency. This escalates the issue to the NRC. Upon 

receipt of FEMA’s withdrawal notification, the NRC considers a response under 10 CFR 

50.54(s)(2)(ii). If the NRC finds that the state of emergency preparedness does not provide 

reasonable assurance that adequate protective measures can and will be taken in a radiological 

emergency, and if after 120 days of this finding the deficiencies have not been remedied, the 

Commission will determine whether the reactor should be shut down until the deficiencies are 

corrected or whether other enforcement action is appropriate. The Commission shall take into 

account whether the licensee can demonstrate that the deficiencies are not significant, or that 

there are compensatory actions. Per 10 CFR 50.54(s)(3), the Commission is not constrained in 

taking “action under any other regulation or authority of the Commission or at any time other 

than that specified in this paragraph,” so the 120-day period does not necessarily apply.

If exercise assessments escalate to the level at which the NRC considers a plant shutdown, the 

basis must be clear and defensible. Because the current system lacks objective measures, 

there is room for substantial variations in judgment of how significant the assessment may be.

Table 5, on the next page, shows by FEMA Region the average number of ARCAs and 

deficiencies given per exercise, from a review of 288 AARs from 1999 to 2012. While it is 

possible the OROs’ performance varied by region, the results suggest that some FEMA regional 

personnel may be more likely to issue ARCAs and deficiencies than colleagues in other 

regions.9

Among the deficiencies cited, one involved a school that evacuated children upon declaration of 

an Alert rather than a Site Area Emergency as required in approved plans. Several deficiencies 

involved issuing alerts and notifications to the public just a few minutes after the 15-minute 

deadline (one of the few hard, quantitative measures in current oversight). In the former case, 

significance was unclear. In the latter case, while it constitutes unacceptable performance on a 

highly risk-significant task, it is not clear that entering a process involving the possibility of 

shutting down the plant is preferable to a lesser penalty.

 9 GAO (2013) has also noted, and the NRC has acknowledged, regional variation in NRC oversight 

findings. 
34

Table 5. Mean Number of ARCAs and Deficiencies by FEMA Region, 1999-2012

FEMA 

Region Mean Number of ARCAs per Exercise Mean Number of Deficiencies per Exercise

I 11.20 0.69

II 6.70 0.15

III 8.30 0.49

IV 1.03 0.06

V 2.04 0.07

VI 1.54 0.28

VII 2.20 0.40

VIII N/A N/A

IX 5.82 0.09

X 5.20 0.00

Mean 3.81 0.21

The NRC’s significance determination process for onsite issues provides for a greater range of 

options. It uses four color-coded levels for individual assessments, each indicating a stronger 

degradation in plant safety, and pertains to the seven cornerstones of the ROP, including onsite 

emergency preparedness:

• Green. Performance is within an expected performance level in which the related 

cornerstone (i.e., important areas of assessment such as emergency preparedness, 

mitigation systems, etc.) objectives are met, though minor reductions in safety margin 

may occur.

• White. Related cornerstone objectives are still being met with a minimal reduction in 

safety margin.

• Yellow. Related cornerstone objectives are being met but with a moderate reduction in

safety margin. 

• Red. There is a significant reduction in safety margin in the area measured by the 

performance indicator.

The NRC has a response (action) matrix for combining individual findings to determine an 

appropriate level of oversight response:

• Column I. All findings are green, and baseline oversight continues.

• Column II. There are no more than two white findings in different cornerstones. Staff

members hold a public meeting with utility management, corrective actions are required, 

and baseline inspections follow up on the corrective actions. 

• Column III. There are three white findings or a yellow finding; the safety margin for the 

cornerstone is considered minimally reduced. The senior regional leadership team holds

a public meeting with senior utility management; the utility conducts a self-assessment 

under NRC oversight, and the NRC conducts additional inspections focused on the 

cause of degraded performance.

• Column IV. There have been repetitive evaluations at Level III (three white findings or a

yellow), there are multiple yellow findings, or there is one red finding; there are 

longstanding unresolved issues or significant reduction in the safety margin. NRC 

headquarters’ Executive Director for Operations holds a public meeting with senior utility 

management; the utility develops a performance improvement plan under NRC 

oversight; there is an NRC team inspection; there is a Demand for Information, 

Confirmatory Action Letter, or Order.
35

• Unacceptable Performance. There is an unacceptable reduction in the margin of safety.

The plant is not permitted to operate; the Commission meets with senior utility 

management; there is an order to modify, suspend, or revoke the operating license.10

A performance-based oversight regimen for offsite response should also incorporate multiple 

levels of evaluative oversight for each subtask and task. Tables 6, 7, and 8, two pages below, 

give a proposed evaluation and oversight scheme. This is not intended to be a direct translation 

of the NRC’s Emergency Preparedness Significance Determination Process (IMC 0609, 

Appendix B) for onsite emergency preparedness.11

Portions of the NRC’s color scheme are used for consistency. While the green, white, and 

yellow assessment levels have been preserved, they have been devised to fit actions and 

consequences appropriate to offsite RERP and FEMA oversight. Instead of a “green” 

assessment being defined as a minor negative assessment, this framework proposes that 

“green” is equivalent to performing a subtask with no anticipated consequences for public health 

and safety. The “white” assessment level is retained as a level indicating a minor reduction in 

public health or safety, while the “yellow” assessment indicates a moderate reduction to public 

health or safety. The “red” assessment level has been dropped from consideration to avoid 

confusion, since in the emergency preparedness cornerstone, a “red” may only be given in the 

event of a real-world emergency response failure. Instead, an “orange” assessment level has 

been introduced for problems in performing high-risk tasks and subtasks of such degree that 

they do not allow for unqualified reasonable assurance that the ORO’s RERP capabilities would 

adequately protect public health and safety if needed. Each subtask in Appendix A has a target

or “green” level of performance. Gradations of unacceptable performance remain to be defined. 

Risk significance of a task determines what ratings may apply to the subtasks. Only subtasks for 

highly risk-significant tasks are subject to the “orange” assessment level, as these are the 

subtasks projected to have the most consequences for public health and safety. For mediumrisk tasks, the most severe assessment that can be given is a “yellow,” as even a complete 

failure to perform a task is not expected to lead to severe consequences for public health and 

safety. 

Tables 6, 7, and 8 on the following pages explain the proposed evaluation schema in greater 

detail. Evaluation builds from an individual demonstration of subtask performance, to subtask 

performance over time (e.g., two years), to task performance over time, to an overall 

“cornerstone” evaluation of offsite RERP.

As noted in Table 6 on the next page, OROs are allowed the opportunity to identify and rectify 

their mistakes or less-than-target performance in a timely manner, depending on level of 

performance. There are several ways this may be accomplished: 

• First, if OROs are able to self-identify a mistake in their demonstration of a task or subtask 

while the task is being demonstrated or immediately following a failed demonstration, they 

may be given the opportunity to restart the activity being performed, without penalty by 

evaluators. 

 10 See NRC (2013). Although the term “column” is not used there, it is used for the action matrix included 

in the IMC 0305 at p. F-1-1. 

11 NRC, “Emergency Preparedness Significance Determination Process.” Available at: 

http://pbadupws.nrc.gov/docs/ML1200/ML120090270.pdf (accessed October 27, 2013). See also NRC,

“Technical Basis for Emergency Preparedness Significance Determination Process.” Available at: 

http://pbadupws.nrc.gov/docs/ML1228/ML12284A512.pdf (accessed October 27, 2013).
36

Table 6. Proposed Evaluative Schema for Performance-Based RERP Oversight, Single

 Demonstration of Subtask

Scope of 

Evaluation

Green White Yellow Orange

Single 

Demonstration of 

Subtask+

Subtask has 

generally been 

performed in a 

satisfactory manner. 

Where issues have 

arisen in an exercise 

or drill, they were 

identified first by the 

ORO and redemonstrated 

immediately to meet 

the green 

performance 

measure.

Subtask performance 

has fallen tolerably 

short of the target 

demonstration 

criteria. Marginal 

decreases to public 

health and safety 

may be expected in 

the event of a 

radiological 

emergency. 

How performance will 

be corrected and 

improved is an 

internal matter for the 

ORO.

Subtask performance 

has fallen short of the 

demonstration criteria 

to a degree that 

would likely result in 

degradation of public 

health and safety in 

the event of a

radiological 

emergency. 

The State ORO is 

required to provide 

FEMA with a root 

cause analysis and 

proposed corrective 

action prior to the 

next scheduled 

subtask 

demonstration (or in 

a specially scheduled 

demonstration before 

the end of the twoyear period if no 

scheduled 

opportunities remain).

The orange 

assessment level is 

reserved only for 

subtasks of tasks 

with high risk 

significance. Orange 

is assessed when 

subtask performance 

is at a level likely to 

result in significant 

degradation of public 

health and safety in 

the event of a 

radiological 

emergency.

If a subtask is 

assessed as orange, 

FEMA will lead a root 

cause analysis 

(requiring meetings, 

interviews, and 

potentially the review 

of capability elements 

such as plans, 

training records, 

equipment status, 

etc.), and develop a 

corrective action plan 

for the ORO to follow. 

The ORO will redemonstrate the 

subtask within a 

specified period of 

time. + Some notification-related subtasks are not assessed in a single demonstration, but only in terms of cumulative 

performance.

• Second, if OROs are given a marginally less-than-target yet tolerable (white) assessment, 

how they correct performance is their concern. 

• Third, if OROs are given a moderately negative (yellow) assessment, they may develop a 

corrective action plan allowing the OROs to work to fix any problems, and re-demonstrate 

them, without long-term ramifications for the licensee. The corrective action plan must, 

however, be documented and shared with FEMA.

However, as shown in Tables 6-8, an orange assessment of a subtask potentially may lead to 

serious consequences, up to and including FEMA’s withdrawal of its reasonable assurance 

determination. The ORO faces more comprehensive oversight, as orange represents significant 

concerns that an ORO will not be able to provide for public health and safety in the event of a 

radiological emergency. Federal staff develop and monitor implementation of a performance 

improvement plan, to ensure that OROs take specific steps to enable improved performance

and then demonstrated that improved performance. If performance does not improve, FEMA 

refers the matter to the NRC. As is currently the case, the NRC has discretion in what it elects to 

do in response, up to suspending plant operations. However, this more systematic escalation 

process should ensure serious attention.
37

Table 7. Proposed Evaluative Schema for Performance-Based Oversight, Cumulative

 Biennial Subtask and Task Performance

Scope of 

Evaluation

Green White Yellow Orange

Cumulative 

Biennial Subtask 

Performance

At least 75% of 

subtask 

demonstrations were 

assessed green.* 

There were no yellow 

or orange subtask 

assessments.

At least 75% of 

subtask 

demonstrations were 

assessed white or 

green. Fewer than 

25% of subtask 

demonstrations were 

assessed yellow.

Any yellow 

assessment was 

cleared.Λ

There was no orange 

subtask assessment.

At least 25% of 

subtask 

demonstrations were 

assessed yellow or 

orange. There was 

no more than one 

orange assessment, 

and it was cleared. Λ

Subtask performance 

was assessed 

orange, and the 

orange assessment 

was not cleared. Λ Or 

subtask performance 

was assessed orange 

more than once, 

cleared or not.

Cumulative 

Biennial Task 

Performance

For a given task, no 

biennial subtask 

assessment was 

yellow or orange. 

One quarter or fewer 

of biennial subtask 

assessments were 

white. 

For a given task, no 

biennial subtask 

assessment was 

yellow or orange. 

More than one 

quarter but fewer 

than half of biennial 

subtask assessments 

were white.

For a given task, at 

least one biennial 

subtask assessment 

was yellow, or more 

than half of biennial 

subtask assessments 

were white.

For a given task, at 

least one biennial 

subtask assessment 

was orange, or more 

than half of biennial 

subtask assessments 

were yellow.

* For subtasks assessed only in terms of cumulative performance, green may be higher (e.g., 90% successful 

attempts at notification).

Λ A yellow or orange subtask assessment in a single demonstration is “cleared” if all subsequent demonstrations of 

the subtask are assessed white or green. 

The overall assessment criteria in Table 8 (next page) could incorporate a timeliness evaluation 

as discussed in Section 6.4. For example, green may also require meeting a timeliness 

threshold in 90 percent of demonstrations, or demonstrating timeliness such that greater than 

90 percent of risk is addressed. 

The proposed evaluation scheme addresses both issues mentioned previously. By introducing 

an expanded range of determinations and response options along with a systematic means for 

combining the individual assessments, each task and the overall ORO program may be 

evaluated systematically and appropriately, with a focus on resolving performance issues.

Additionally, by tying evaluations to metrics that have a definite quantification or threshold that 

must be met, potential subjectivity should be reduced. Introducing performance-based, 

quantifiable metrics and tying these to a graduated evaluation system offers promise for 

improving the overall consistency of RERP oversight, while providing a reasonable assurance 

that public health and safety needs are being met.

Software could support and enhance a systematic evaluation process. Appendix C examines 

potential applicability of an existing software tool to the proposed evaluation process.
38

Table 8. Proposed Evaluative Schema for Performance-Based Oversight, Cumulative

 Biennial ORO RERP “Cornerstone” Performance

Scope of 

Evaluation

Green White Yellow Orange

Cumulative 

Biennial ORO 

RERP 

“Cornerstone” 

Performance

No task was 

assessed yellow or 

orange, and there 

was no more than 

one white high-risk 

task.

No task was 

assessed orange. No 

high-risk task was 

assessed yellow, and 

there was no more 

than one yellow task 

assessment. There 

were no more than 

two white high-risk 

tasks, and no more 

than four white tasks 

overall.

No task was 

assessed orange. At 

least one high-risk 

task was assessed 

yellow, but no more 

than three tasks were 

yellow overall. 

At least one task was 

assessed orange, or 

at least four tasks 

were assessed 

yellow.

FEMA notifies the 

NRC and initiates a 

comprehensive 

review of ORO plans, 

organization, 

equipment, training, 

and exercises to 

develop a 

performance 

improvement plan for 

the ORO’s RERP, 

delivered within a 

specified period of 

time (e.g., 120 days). 

The ORO will have a 

specified period of 

time to implement all 

requirements of the 

performance 

improvement plan, 

monitored by FEMA. 

The ORO must 

demonstrate at least 

white-level 

performance in all 

subtasks of tasks that 

led to the 

performance 

improvement plan in 

that time. If not, 

FEMA withdraws 

“reasonable 

assurance” and 

refers the matter to 

the NRC.

6.7 ENHANCEMENT OF RISK FOCUS

The RERP oversight scheme proposed herein has several features to ensure a focus on risksignificant elements:

• Performance Focus. The oversight system is designed to focus on actual performance.

Task performance contributes more directly to reducing risk than do enablers of task 

performance, whose contribution to risk reduction is more difficult to gauge.

• Task Stratification by Risk Significance. Tasks have been identified as high, medium, and 

low risk. The basis for this determination is the tasks’ contribution to a defined public 

health and safety goal of dose avoidance. The determination of risk significance has 

implications for oversight. For example, yellow ratings on three high-risk tasks will receive 

a higher level of follow-on oversight than yellow ratings on medium- or low-risk tasks. As 

previously mentioned, only high risk tasks can be subject to an orange assessment. 
39

Further, low risk tasks are not formally evaluated; they have no defined subtasks (meaning 

they can never be rated even yellow under the proposed scheme). Oversight and 

evaluation resources are then allocated away from low risk tasks.

• Subtask Selection. Only subtasks deemed necessary and critical are included for 

evaluation. Oversight and evaluation resources are not spread over an extensive set of 

subtasks and criteria that may not be critical to performance of risk-significant tasks.

• Gradation of Assessments against Objective Measures. There is or will be a scale for 

unacceptable performance of all subtasks that have performance measures. The scale 

allows for more middle ground in addressing unacceptable performance before escalating 

the involvement of oversight resources to high levels that may not be warranted by the 

risks posed.

Whether the proposed oversight system will necessarily enhance focus on risk-significant 

elements compared to the current system is difficult to prove absent additional resource 

information for both.

As noted above in Section 6.6, a review of FEMA-identified deficiencies from 1999 to 2012 

showed some deficiencies that were not necessarily risk-significant, or at least not to the point 

of considering the shutdown of a reactor. However, this was anecdotal. For fuller consideration, 

Table 9 on the next page groups all deficiencies given from 1999 to 2012 into several broad 

categories, gives examples for each grouping, and offers a preliminary determination on the risk 

significance of these deficiencies within the proposed performance-based oversight system, 

based on the risk significance of the analogous task.

Table 9 shows that in most instances the current oversight system has identified deficiencies for 

areas that are significant to protecting public health and safety in radiological emergency.

Problems with alert and notification of the public were most common, followed by a failure to 

provide or transmit PADs. These deficiencies, should they occur in a real- world incident, would 

all result in members of the public being unaware of an emergency or actions they should take 

to protect themselves. 

However, deficiencies were assigned for deviations from plans where the demonstrable impact 

on public health and safety was not clear. For example, as noted previously, a school chose to 

evacuate children when an Alert was declared, contrary to plans. In another, a surveillance 

helicopter claimed it would not fly into the EPZ given the release, contrary to plans. It is unclear

why these were deficiencies rather than ARCAs. Thus there is a potential to focus resources on 

issues of uncertain, and potentially low, risk significance.

Also as noted previously, deficiencies were assigned for alert and notification delays of a few 

minutes. While this could be significant, particularly in a rapidly progressing emergency, it may 

not warrant raising the possibility of shutting down the reactor. Again, there is potential for 

misallocating resources relative to risk.

Given its performance focus, stratification of tasks by risk, selection of only necessary and 

critical subtasks, and gradation of assessments, the proposed RERP oversight process should 

serve to enhance the focus of ORO and oversight resources on risk-significant elements.
40

Table 9. Number of Deficiencies by Type, 1999-2012

Type of Deficiency Number 

Given Example(s) Significance Determination

Alert and Notification 

Problems – Responder 

Error

17

Sending faxes instead of notification to start 

sirens, taking more than 15 minutes to start 

sirens

High Significance

PADs for Responders Not 

Transmitted 7

Responders not directed to take KI, 

responders do not receive notification to take 

KI

High Significance

PADs for Public Not 

Transmitted 7 Public not advised of evacuation routes, public not advised to take KI High Significance

Communication Errors 6

Incorrect messaging released, inability to 

contact and activate necessary personnel, 

not reaching out to confirm receipt of EAS 

messages

High Significance

Deviation From Plans 5

Releasing children from school in violation of 

plans, helicopter refusing to fly given release 

in violation of plans

Indeterminate

Alert and Notification 

Problems – Mechanical 

Error

4 Digi-Cart system did not work, lack of backup 

power source High Significance

Failure to Develop PADs 

to Adequately Protect 

Public

3

Failure to identify special needs individuals 

requiring transportation, evacuation orders 

sending evacuees into plume path

High Significance

Access Control Problems 1 Failure to establish access control in 

evacuation zone in timely manner Moderate Significance
41

7. CONSISTENCY WITH PROPOSED ONSITE PERFORMANCEBASED OVERSIGHT

The NRC’s 2008 “Elements of a Performance-Based Emergency Preparedness Regulatory 

Regimen” (ADAMS Accession Number ML080440163) offers a preliminary proposal for a 

voluntary performance-based emergency preparedness regulatory regimen for licensees. The 

offsite RERP oversight regimen proposed here follows the spirit of that proposal and is largely 

consistent with it. Areas of similarity include:

 Focus on Performance in Lieu of Compliance. The NRC seeks to “focus oversight and 

licensee efforts on actual performance competencies [demonstrated] during drills and 

exercises, rather [than] compliance issues.” Variations in plans, training attendance, and 

response organization “would not matter so long as performance was acceptable.” 

Reasonable assurance determinations would focus on demonstration of competencies 

rather than compliance. The same focus on performance underlies the proposal for 

offsite RERP oversight.

 Goals. Stated goals for the NRC’s performance-based regimen are similar to the ones 

identified in Chapter 3 of this paper. (However, the NRC’s proposal includes goals for 

State and local officials, whereas this paper is silent on goals for licensees.) For 

example, both regimens are concerned with ensuring immediate notification of 

emergency conditions from licensee to ORO and rapidly alerting and informing the public 

of PADs.

 Scenario Variation and Secrecy. The NRC’s proposal specifies that licensees must not 

use the same scenario in sequential evaluated exercises, and that emergency teams 

should not know scenario specifics in advance of an exercise. Section 6.4 of this paper 

calls for scenarios to “be varied and not known to players beforehand” in testing the 

ability to make timely PADs and communicate them to the public. 

 De-emphasis of Large Biennial Exercises. The NRC’s proposal acknowledges that some 

competencies are not amenable to testing in large biennial exercises. Section 4.2 of this 

paper calls for testing critical tasks through frequent functional exercises (such as for 

testing PAD development and communications as described in section 6.4) and drills, 

moving away from a focus on large-scale biennial exercises. 

 Performance Standards. Both documents call for performance standards (objective 

measures) for each element to be demonstrated. This paper has proposed measures for 

subtasks (see Appendix A).

 Differential Levels of Oversight. Both documents note that increased oversight should 

only result from crossing clear thresholds of performance or failures in significant 

corrective actions.

 Concern for Communicating Reasonable Assurance. Both papers posit that use of 

objective performance standards or measures will improve understanding of reasonable 

assurance.

There is a difference in what is to be evaluated. The NRC offers multiple proposed metrics 

focused on activating centers and making them operational. This report generally treats facilities 

as an enabler of performance, not a focus of evaluation in themselves. The exception is a 

proposed subtask on activating an alternate communications center, given the importance of 

ensuring reliability for this function. 

Overall, however, the performance-based oversight systems proposed by the NRC and in this 

report are consistent and potentially complementary. 

43

8. INTEGRATION WITH ALL-HAZARDS INITIATIVES

The Nation’s laws and policies for emergency preparedness have evolved piecemeal, generally 

in response to specific incidents. Some incidents led to hazard-specific policy. For example, the 

1979 Three Mile Island emergency led to current regulations for radiological emergency 

preparedness both at and around NPPs.

Yet, after other incidents, Congress concluded there are unacceptable risks to focusing 

resources and attention on a single, preeminent hazard. After Hurricane Andrew in 1992, 

Congress eliminated the Federal Civil Defense Act requirement that FEMA-funded State and 

local preparedness efforts be “consistent with, contribute to, and not detract from” nuclear attack 

preparedness; policy became simply to prepare for “hazards.” After Hurricane Katrina in 2005, 

Congress corrected a perceived focus on terrorism-specific preparedness through the PostKatrina Emergency Management Reform Act (PKEMRA). PKEMRA required the President to 

develop a national preparedness system “to prepare the Nation for all hazards, including natural 

disasters, acts of terrorism, and other man-made disasters.” Presidential Policy Directive 8

(PPD-8) of 2011 assigns responsibilities to Executive Branch agencies for implementing these 

all-hazards national preparedness system requirements. PPD-8 also reaffirms the all-hazards 

National Incident Management System (NIMS) required by Homeland Security Presidential 

Directive 5 (HSPD-5) of 2003.

While details may change over time, any new RIPB construct for oversight of offsite RERP will 

need to operate within a broader, all-hazards preparedness policy framework. State and local 

OROs will be responsible for satisfying both RIPB RERP requirements and all-hazards 

requirements.

This chapter describes how the RIPB oversight concept for offsite RERP can integrate with key 

all-hazards preparedness policy initiatives. Elements of the proposed RIPB RERP framework 

are or could be consistent with all-hazards initiatives. However, RERP would not meet its 

hazard-specific intent through such generic preparedness requirements as currently exist.

Further, the proposed RIPB RERP framework will not help ensure significantly greater 

consistency of onsite NPP emergency preparedness (EP) with all-hazards initiatives than do 

current arrangements.

This chapter will:

• Discuss what integration of RERP and all-hazards initiatives means and what it should 

achieve,

• Describe key Federal all-hazards initiatives and how the Federal Government influences 

compliance,

• Examine how the proposed RIPB RERP oversight construct could work within the allhazards policy context, and

• Assess whether onsite NPP EP compliance with all-hazards initiatives is likely to 

improve in the proposed RIPB RERP oversight construct.
44

8.1 OBJECTIVES FOR INTEGRATION

This section addresses what integration of RIPB RERP and all-hazards initiatives should 

achieve. Providing reasonable assurance of public health and safety in the event of a 

radiological emergency at an NPP is foremost. Integration is valuable, but secondary.

8.1.1 Meaning of “Integration”

To integrate is either to combine two or more elements into a functioning, unified whole or to 

incorporate one or more elements into a larger unit. If all-hazards preparedness is the “larger 

unit,” then there are two basic options for integrating RIPB RERP and all-hazards preparedness 

initiatives:

(1) Develop crosswalks and linkages between RIPB RERP and all-hazards preparedness 

initiatives so that each supports the other. RIPB RERP and all-hazards preparedness 

initiatives would be consistent with and supportive of one another, but not lose their 

identity within the whole of preparedness efforts.

(2) Subsume the RIPB RERP construct within all-hazards preparedness initiatives. Specific 

RIPB RERP requirements would be eliminated in favor of all-hazards preparedness 

requirements. All-hazards preparedness requirements would have to provide reasonable 

assurance of OROs’ ability to protect public health and safety from a radiological release 

beyond the boundaries of an NPP. 

As will be discussed in Sections 8.2 and 8.3, the first option applies for most all-hazards 

initiatives.

8.1.2 Rationale for Integration

There are two main arguments for integration of all-hazards initiatives with offsite RERP: 

compliance and efficiency.

The compliance argument involves respect for law, regulation, and policy. Where there is no 

conflicting legal requirement, law, regulation, and policy should be followed. Ignoring the 

requirements of any one law, regulation, or policy—picking and choosing—undermines 

compliance with law, regulation, and policy generally.

The efficiency argument is that it takes fewer resources or less effort to do something one way

rather than two ways. Working with one set of concepts, terms, procedures, and standards for 

RERP and another for other hazards is inherently inefficient, if the same concept, term, 

procedure, or standard could adequately address both RERP and all-hazards concerns.

8.1.3 Dimensions of Integration

Integration of RIPB RERP and all-hazards initiatives could take place in multiple areas:

• Categorization and Reporting of Results, or consistency in how performance outputs are 

organized and presented.

• Performance, or consistent definition of how or what performance occurs through 

standards and targets. 

• Inputs to Performance, with consistent approaches and terminology in the plans, 

organizational constructs, equipment, training, and exercises that enable performance.
45

In a strict performance-based oversight construct, these are irrelevant to oversight until 

and unless performance is judged inadequate. 

• Funding Inputs, allowing all-hazards funding streams to be used for RERP needs or 

meeting an all-hazards (including RERP) need through any RERP funding stream.

• Risk Prioritization, ensuring that RERP hazards factor into all-hazards risk assessment 

and management, and even that other hazards (e.g., as initiating events or 

complications to response) factor into RERP. 

FEMA and the States have already made efforts in several of these areas, as will be discussed 

in Section 8.3.

8.1.4 Objectives for Integration

Integration of all-hazards preparedness initiatives in the RIPB constructs for offsite RERP and 

onsite NPP EP must satisfy the following objectives:

• Integration will not reduce the NRC’s ability to provide reasonable assurance of public 

health and safety in the event of a radiological emergency at an NPP.

• No RIPB RERP or NPP EP performance target will be made less stringent for the sake 

of all-hazards integration.

• Integration of any all-hazards requirements into RIPB RERP requirements will not 

increase the scope of RIPB RERP oversight, or industry contributions for that oversight, 

beyond RERP-specific concerns.

• Integration of any all-hazards requirements into RIPB RERP requirements will not 

increase total costs for OROs.

• Where there is no performance or efficiency reason not to be consistent with all-hazards 

initiatives, such consistency will be encouraged through the oversight process.

• Both licensees and OROs will be free to incorporate all-hazards requirements, concepts, 

and/or terminology as long as they can meet their performance targets under RIPB 

RERP or NPP EP oversight constructs.

• Where a licensee or ORO exhibits a performance deficiency, any remedies required by 

Federal entities will incorporate applicable all-hazards requirements, concepts, and/or 

terminology.

8.2 INTEGRATION CONTEXT: ALL-HAZARDS INITIATIVES, PROCESSES, AND 

SYSTEMS

This section defines all-hazards initiatives and identifies the key ones, describes means 

available to influence compliance, and summarizes the key initiatives.

8.2.1 Scope of All-Hazards Initiatives

For this report, “all-hazards initiatives” are policies from the President, the Department of 

Homeland Security (DHS), or FEMA about preparing to manage incidents resulting from 
46

hazards of any type: natural, accidental, or deliberate.1,2 The term excludes policies focused on 

a single hazard, such as FEMA’s REP Program Manual or Executive Order 13636 of February 

12, 2013, “Improving Critical Infrastructure Cybersecurity.”

There are two Presidential directives that best fit this definition:3

• HSPD-5, “Management of Domestic Incidents,” which establishes the requirement for 

NIMS to provide for interoperability and compatibility among Federal, State, and local 

capabilities. NIMS includes a core set of concepts, principles, terminology, and 

technologies covering the incident command system (ICS); multi-agency coordination 

systems; unified command; training, qualifications, and certification; identification and 

management of resources (including systems for classifying types of resources); and the 

collection, tracking, and reporting of incident information and incident resources.

• PPD-8, “National Preparedness,” which requires a National Preparedness Goal defining 

core capabilities needed to prepare for the greatest risks facing the Nation. PPD-8 also 

requires a National Preparedness System of guidance, programs, and processes to help 

the Nation meet the National Preparedness Goal through planning, organization, 

equipment, training, and exercises that build core capabilities, as well as regular 

assessment of national capabilities. 

Congress requires in PKEMRA that FEMA lead implementation of both of these initiatives: 

• “The [FEMA] Administrator ... shall ensure ongoing management and maintenance of 

the National Incident Management System” (paragraph 509(b)(1)).

• “The President, acting through the [FEMA] Administrator, shall develop a national 

preparedness system to enable the Nation to meet the national preparedness goal”

(subsection 643(a)).

8.2.2 Means to Influence Compliance

FEMA does not have authority to direct State, local, or private sector entities.4 However, FEMA 

does have various tools at its disposal to influence State and local OROs to adopt or comply 

with its policies. These include:

 1 NRC (2012), paragraph 4.3.1, refers to “integration of White House and DHS/FEMA initiatives into the 

RERP and NPP EP programs” and “DHS/FEMA initiatives, processes, and systems that affect State, local 

and NPP EP programs.” 

2 Paragraph 602(a)(1) of the Robert T. Stafford Disaster Relief and Emergency Assistance Act, Public 

Law 93-288, as amended (42 USC 5195a) defines “hazard” as “an emergency or disaster resulting from–

(A) a natural disaster; or (B) an accidental or man-caused event.” 

3 Other directives were considered. PPD-21, Critical Infrastructure Security and Resilience, does refer to 

“all hazards.” However, it focuses on Federal responsibilities. HSPD-8, National Preparedness, was 

superseded by PPD-8. Executive Order 13603, National Defense Resources Preparedness, addresses 

resource claimancy for the “full spectrum of emergencies.” Authorities from this Executive Order and the 

Defense Production Act have been delegated to FEMA to help States and private sector entities place 

“rated” (priority) orders for contracts, but the Executive Order contains no State, local, or private sector 

preparedness requirement. Sources for review of Presidential directives include compilations listed under 

References.

4 FEMA (2011) describes even the current Radiological Emergency Preparedness (REP) Program as a 

“voluntary program.” As noted in 44 CFR 352, States may “decline or fail” to prepare or exercise offsite 

radiological emergency plans. FEMA could make a determination based on plans available to FEMA 

under 44 CFR 350.3(f), not necessarily a radiological emergency response plan submitted under the 44 
47

• Financial Stimulus – Preparedness Assistance. In Fiscal Year 2013 FEMA will provide 

more than $354 million in State Homeland Security Program grants, almost $559 million 

in Urban Area Security Initiative grants, and more than $332 million in Emergency 

Management Performance Grant (EMPG) funding. States and localities are not obligated 

to take these funds, but if they do they become subject to the conditions of the grants. All 

three of these programs require that grantees “ensure and maintain adoption and 

implementation of NIMS” in compliance with HSPD-5 (see FEMA 2013a, p. 12, and 

FEMA 2013b, p. 17). Further, Urban Areas and States must maintain and update a 

Threat and Hazard Identification and Risk Assessment (THIRA) annually, and their 

emergency operations plan (EOP) every two years as conditions of grant funding. They 

must also develop and submit a multi-year training and exercise plan (TEP) annually, 

following HSEEP guidance. State grantees must submit an annual SPR. Finally, the 

grants must build and sustain the core capabilities described in the National 

Preparedness Goal.

• Financial Stimulus – Disaster Assistance. Where required by law, FEMA has made 

completion of a preparedness activity a condition for receiving an additional amount of 

disaster assistance. The Disaster Mitigation Act of 2000 amended the Stafford Act so 

that States and local jurisdictions could receive additional post-disaster mitigation 

funding if they had approved hazard mitigation plans.

• Direct Technical Assistance. FEMA may conduct workshops and facilitate a grantee’s 

efforts to develop plans or undertake other homeland security and emergency 

management efforts. FEMA’s National Preparedness Directorate publishes a catalog of 

available technical assistance (FEMA 2013c).

• Tools. FEMA provides free software tools to accomplish some homeland security and 

emergency management tasks. This helps standardize processes and products as users 

opt for the free tools over alternative approaches. One widely used tool is HAZUS 

(Hazards U.S.), software for estimating potential losses from earthquakes, floods, and 

hurricanes. (See FEMA 2013j.)

• Templates. Another way to standardize State and local work products without a direct 

requirement is to offer templates. State and local users save time by using the 

templates, and FEMA sees consistent products as more users opt for the template 

approach. FEMA’s HSEEP offers templates for exercise documents, exercise planning 

meeting presentations, and more. (See DHS 2013, DHS 2013a, and DHS 2013b.)

• Guidance. FEMA provides doctrine and guidance on a range of subjects, offering advice 

and best practice for accomplishing emergency management tasks. Examples include 

Comprehensive Preparedness Guide (CPG) 101, version 2.0, Developing and 

Maintaining Emergency Operations Plans (FEMA 2010a).

• Reports and Research Findings. FEMA can influence actions simply by providing 

information. Under PKEMRA, FEMA provides annual reports to Congress on the 

Nation’s preparedness. FEMA does not identify individual States, but as Figure 7 on the 

next page shows, States can compare their individual SPR ratings to aggregates 

 

CFR 350 approval process. As noted in 44 CFR 350.3(c)(1), the NRC can also issue a license without a 

FEMA-approved plan if “there exists a State, local or utility plan which provides assurance that public 

health and safety is not endangered by the operation of the facility.” However, FEMA notes that 

participation in the 44 CFR 350 plan approval process “demonstrates to the public that a site has met 

specific planning and preparedness criteria.” See FEMA (2011), pp. 54, 682. 
48

State and Territorial Preparedness Perspectives: Environmental Response/Health and Safety

Figure 7. Example of State Preparedness Data in National Preparedness Report

Source: FEMA (2013d)

presented in the National Preparedness Report (NPR) (FEMA 2013d). Subsection 

611(e) of the Stafford Act also authorizes the FEMA Administrator to “study and develop 

emergency preparedness measures designed to afford adequate protection of life and 

property, including research and studies as to the best methods of treating the effects of 

hazards.” This can include both technical reports and dissemination of lessons learned.

Of these tools, grants are FEMA’s primary lever. Table 10 on the next page shows the relative 

contribution of Federal (primarily FEMA) grants and nuclear industry fees to selected State 

emergency management agency budgets. Preparedness grants, even if some funding passes 

through to local government, are a significant portion of the budget.5 State and local OROs must 

be sensitive to all-hazards requirements attached to the funding. 

FEMA has limited ability to influence licensees. Each year, FEMA sets the fee it will charge 

licensees for RERP oversight under authority of 44 CFR 354. However, the fee has been for 

FEMA services “to facilitate offsite radiological emergency planning and preparedness,” which is 

understood as distinct from all-hazards preparedness.6,7 FEMA’s other major initiative with the 

private sector, PS-PREP, has been to endorse preparedness standards for private sector 

business continuity and emergency preparedness programs, and to recognize companies that 

meet them. The program is voluntary. (See FEMA 2012b.)

 5 Examples are from a review of recent State emergency management agency budgets for 31 States with 

NPPs. Budget presentations vary by State. In some cases the emergency management budget is 

subsumed within a larger Department of Public Safety or Department of Military Affairs budget. In some 

cases both preparedness and disaster assistance are combined in a single Federal Aid line item. In some 

States radiological emergency preparedness funding goes to a Department of the Environment or a 

Department of Health responsible for radiation control activities in the State. The budget figures shown 

here taken from: 

http://www2.illinois.gov/gov/budget/Documents/Budget%20Book/FY%202014/FY14EnactedTotalBudgetb

yLineItem.xls, http://admin.state.nh.us/budget/Budget2014-2015/GovernorsBudgetBill.pdf#02-23, 

http://media.obm.ohio.gov/OBM/Budget/Documents/operating/fy-14-15/bluebook/budget/SectionD_DPS.pdf, http://www.portal.state.pa.us/portal/server.pt/document/1320332/2013-

14_governors_executive_budget_cd_pdf, and 

http://finance.vermont.gov/sites/finance/files/pdf/Vantage/FY%202014%20Executive%20Budget%20Reco

mmendations.pdf (accessed October 27, 2013). 6 44 CFR 353, Fee for Services in Support, Review and Approval of State and Local Government or 

Licensee Radiological Emergency Plans and Preparedness. Available at: 

http://www.gpo.gov/fdsys/pkg/CFR-2011-title44-vol1/pdf/CFR-2011-title44-vol1-part353.pdf (accessed 

October 27, 2013).

7 44 CFR 354, Fee for Services to Support FEMA’s Offsite Radiological Emergency Preparedness 

Program. Available at: http://www.gpo.gov/fdsys/pkg/CFR-2011-title44-vol1/pdf/CFR-2011-title44-vol1-

part354.pdf (accessed October 27, 2013).
49

Table 10. Relative Contribution of Grants and Nuclear Industry Fees to Selected

 State Emergency Management Agency Budgets

(Multiple sources*)

Illinois New 

Hampshire Ohio Pennsylvania Vermont

Budget

Year 2014 enacted 2012 actuals 2013 estimated 2012-2013 

available 2013 as passed

Total

Budget

$502,148,400

(100%)

$11,062,395

(100%)

$131,786,175

(100%)

$645,844,000

(100%)

$18,591,123

(100%)

Preparedness 

Grants

$300,351,100

(59.81%)

$4,856,462

(43.90%)

$77,966,477

(59.16%)

$204,937,000

(31.73%)

$15,842,606

(85.22%)

Disaster

Grants

$127,000,000

(25.29%)

$2,427,923

(21.95%)

$27,707,636

(21.02%)

$191,100,000

(29.59%)

$0

(0%)

Radiological 

Emergency 

Preparedness 

Fees/ 

Assessments

$23,677,600

(4.72%)

[6 sites]

$1,334,672

(12.06%)

[1 site]

$1,415,945

(1.07%)

[2 sites]

$2,050,000 

(0.32%)

[5 sites]

$2,321,510

(12.49%)

[1 site]

* Sources include: Illinois Office of Management and Budget (2013), New Hampshire Department of 

Administrative Services (2013), Ohio Office of Management and Budget (2013), Pennsylvania Office of the 

Budget (2013), and Vermont Department of Finance and Management (2013).

8.2.3 Key All-Hazards Initiatives for Integration

To assess potential integration of the proposed RERP RIPB construct with all-hazards 

initiatives, the rest of this chapter will center on two broad initiatives: the National Preparedness 

System and NIMS. These are the key initiatives as based on the White House and legislative 

interest described in Section 8.2.1, and FEMA’s grant focus as discussed briefly in Section 

8.2.2. Summaries follow.

National Preparedness System. Capabilities are “the means to accomplish a mission, 

function, or objective based on the performance of related tasks, under specified conditions, to 

target levels of performance.” PPD-8 required a National Preparedness Goal to define the core 

capabilities necessary to prepare for the incidents that pose the greatest risk to the security of 

the Nation. It required a National Preparedness System to guide building and maintaining those 

core capabilities.

The National Preparedness Goal (DHS 2011a) identified 31 core capabilities across five mission 

areas: prevention, protection, mitigation, response, and recovery. (Section 8.3.9 will discuss the 

response-related core capabilities in more detail.) 

The National Preparedness System (FEMA 2011b) identified six components for achieving 

preparedness:

(1) Identifying and Assessing Risk. Preparedness requires identifying threats and hazards, 

including projected consequences. A Strategic National Risk Assessment (SNRA) 

informs the National Preparedness Goal (see DHS 2011b). State and local FEMA 

grantees must perform a THIRA, as noted in Section 3.2. So must FEMA Regions, 

working with Federal partners.

(2) Estimating Capability Requirements. In their THIRAs, State and local grantees must also 

develop requirements for delivering each of the 31 core capabilities. FEMA guidance is 
50

to combine the largest relevant impact (e.g., 50 fatalities) with either a timeliness or 

effectiveness standard (e.g., within 72 hours, or with 100 percent completion) to define a 

target requirement (FEMA 2013h).

(3) Building and Sustaining Capabilities. Defining target requirements allows grantees to 

identify gaps and the means to address them. The means include funding, organizing 

(both internally and through mutual aid), equipping, and training.

(4) Planning to Deliver Capabilities. The National Preparedness System calls for plans to 

document how capabilities will be delivered. Each mission area has a framework to 

assign responsibilities across the “whole community” (Federal, State, territorial, tribal, 

local, private sector, and public) for delivering capabilities. An example is the National 

Response Framework, formerly the National Response Plan. There are Federal 

Interagency Operational Plans detailing how the Federal Government implements its 

responsibilities under each framework, as well as department- and agency-level plans as 

needed. Grantees are encouraged to use the planning approach outlined in CPG 101 to 

guide their planning.

(5) Validating Capabilities. Exercises and assessments are used to test whether targets are 

being met. The National Exercise Program (NEP) is the “principal exercise mechanism 

for examining national preparedness and measuring readiness” (FEMA 2013e). HSEEP 

offers doctrine and templates for design, development, conduct, and evaluation of 

exercises. Finally, a Comprehensive Assessment System (CAS) that includes the NPR 

collects and analyzes data about capabilities to report progress on building and 

sustaining required levels of capability. 

(6) Reviewing and Updating. The National Preparedness System includes periodic reviews 

of risk conditions, capability development, and exercise and assessment results to adjust 

priorities, goals, and objectives if needed.

Section 8.3 will discuss potential integration of these all-hazards preparedness elements with 

the proposed RIPB RERP construct.

NIMS. PPD-8 and National Preparedness System documents stress the importance of NIMS, 

which was required by HSPD-5. NIMS provides standardized, scalable organizational structures 

for incident management (see Figure 8 on the next page). 8 This helps collaboration—across 

the public and private sectors, among different levels of government, and across disciplines. 

NIMS also focuses on common terminology to underpin communications and collaboration.

Common terminology extends to resource management, where NIMS relies on “resource 

typing” (definitions and standards for certain resources or combinations of resources) as well as 

evolving training and qualifications standards for people to hold positions within standard NIMS 

organizational structures for an incident. As shown in Table 11 on the next page, NIMS also 

relies on standard forms for communications required to support NIMS processes, such as 

development and issuance of incident action plans. 

 8 Not every position is required depending on the incident. See DHS (2008) for more on NIMS principles 

such as reliance on incident action plans, standard forms, and common terminology.
51

Figure 8. NIMS ICS Generic Incident Organizational Structure

Source: FEMA IS-200.b (undated)

Table 11. Selected Standardized Forms under NIMS ICS

(DHS 2008: 128) 

Number Purpose

ICS 201 (p. 1) Incident Briefing Map

ICS 201 (p. 2) Summary of Current Actions

ICS 201 (p. 3) Current Organization

ICS 201 (p. 4) Resources Summary

ICS 202 Incident Objectives

ICS 203 Organization Assignment List

ICS 204 Assignment List

ICS 205 Incident Radio Communications Plan

ICS 206 Medical Plan

ICS 207 Incident Organization Chart (wall mounted)

ICS 209 Incident Status Summary

ICS 210 Status Change

ICS 211 Incident Check-In List

ICS 213 General Message

ICS 215 Operational Planning Worksheet

ICS 215A Hazard Risk Analysis

Section 8.3 will discuss potential integration of NIMS organizational, resource typing, and 

position qualification requirements with the proposed RIPB RERP construct.
52

8.3 ALL-HAZARDS INTEGRATION POTENTIAL: OFFSITE

This section reviews potential for integrating elements of the National Preparedness System 

and NIMS into a RIPB RERP oversight construct. Key factors assessed are whether and how 

the current RERP oversight system integrates all-hazards elements, the degree to which allhazards elements support RERP (and vice versa), how well all-hazards elements support 

performance-based oversight, and whether all-hazards elements would conflict with 

performance-based oversight of offsite RERP efforts.

Generally, the all-hazards elements are inputs to performance rather than standards of 

performance. They therefore neither support nor conflict with performance-based oversight, 

which focuses on whether State and local OROs meet objective performance outcomes.

8.3.1 Risk Prioritization: Threat and Hazard Identification and Risk Assessment

Radiological emergencies involving commercial NPPs have factored into risk assessments 

supporting all-hazards preparedness. The SNRA included a scenario in which an accident led to 

reactor core damage and release of radiation. Both the 2012 and 2013 editions of FEMA’s CPG 

201 THIRA guidance mentioned “radiological release” as one hazard to consider. Of four States 

with NPPs and 2012 THIRA documents online, two specifically mentioned the hazard. 

The THIRA process differs from the probabilistic risk assessments favored by the NRC. The 

THIRA is consequence-focused. FEMA’s THIRA guidance encourages development of a “metascenario” that combines the most stressing aspects of each hazard in order to set capability 

targets for addressing the worst impacts.

However, as a risk assessment the THIRA process is not directly germane to RERP or 

performance-based oversight. The main offsite RERP interest in other hazards is whether State 

and local OROs can perform adequately if confronted with both a radiological emergency and 

another incident at the same time. This goes to setting performance and capability targets, 

discussed next.

In summary, there is currently a moderate level of integration between RERP and all-hazards 

risk prioritization efforts such as THIRA. However, it is one-way: RERP considerations inform 

some THIRAs, but THIRAs do not inform RERP. 

8.3.2 Performance Targets (National Preparedness Goal, Threat and Hazard 

Identification and Risk Assessment)

State and local jurisdictions are using the THIRA process to set capability targets because the 

National Preparedness Goal does not—at least not specific and measurable targets. Table 12

on the next page provides an example for one core capability. The numbered targets further 

specify tasks, but do not define any standards of performance.9

The National Preparedness Goal acknowledges this. It notes that the targets will serve as the 

basis for development of performance measures to track progress. It also states that the 

“capability targets serve as strategic targets and will be vetted and refined, taking into 

 9 The National Academy of Public Administration (2011) has leveled this criticism: “The Panel finds that 

the Goal document does not meet the need for clearly defined, measurable outcomes for preparedness 

capabilities. ... Without additional clarification by DHS/FEMA, the states and urban areas will need to 

continue to establish capability levels for themselves which may or may not satisfy the national interest. 

The Panel strongly recommends that [DHS/FEMA] develop more specific or additional capability targets 

...” (p. viii). 
53

Table 12. Example of Capability Targets in the National Preparedness Goal

(DHS 2011a)

Mass Care Services:

Provide life-sustaining services to the affected population with a focus on hydration, 

feeding, and sheltering to those who have the most need, as well as support for 

reunifying families.

1. Move and deliver resources and capabilities to meet the needs of disaster survivors, including individuals with 

access and functional needs and others who may be considered at-risk.

2. Establish, staff, and equip emergency shelters and other temporary housing options (including accessible 

housing) for the affected population.

3. Move from congregate care to non-congregate care alternatives and provide relocation assistance or interim 

housing solutions for families unable to return to their pre-disaster homes.

consideration risk information and resource requirements, during the planning process 

established through PPD-8.” The THIRA process is one element of this planning.

From the few examples available online, States have used the THIRA process to develop 

measurable targets, often focusing on initiating or completing an activity within a certain amount 

of time. However, the targets are not necessarily compatible or standardized. Table 13 on this

and the following page gives example targets from three States for three different capabilities 

potentially applicable to RERP. Over time, FEMA could begin compiling and deconflicting the 

targets to develop and apply national performance measures, but this has not occurred.

None of the States with THIRAs online specifically cited RERP considerations as the basis for a 

performance target. For full integration with RIPB RERP, ideally the THIRA process will come to 

involve mandatory inclusion of a RERP-relevant scenario by States with RERP planning 

responsibilities. These would be used to help set capability targets. Targets for a given 

capability could be stricter than those required for RERP, depending on characteristics of other 

hazards. As long as the capabilities and capability targets addressed all RIPB RERP 

requirements, and the all-hazards THIRA capability targets were equal to or more stringent than 

any accepted RIPB RERP performance target, RIPB RERP could rely on meeting all-hazards 

performance targets to give reasonable assurance of preparedness for RERP requirements.

However, this is not currently the case. The examples given do not reflect expectations for 

response to a significant release. The THIRA focus on States and selected urban areas also 

means its targets may not scale to RERP implementation requirements for local, non-urban 

OROs around NPPs.

Table 13. Selected Capability Targets from Three States’ 2012 THIRAs

(Multiple sources*)

Mass Care Services: Provide life-sustaining services to the affected population with a focus on hydration, 

feeding, and sheltering to those who have the most need, as well as support for reunifying families.

• Mobilize resources within 72 hours of an incident to provide life-sustaining services to the affected human 

and animal populations consisting of an estimated 650,000 people and 96,300 animals in need of shelter, 

potable water, sanitation disposal, secure medical treatment facilities, mental health treatment, functional 

needs assistance, veterinary services, and support to reunify families.

• Inspect 2,600 potential temporary shelter structures within the first 72 hours after an incident prior to placing 

them into operation.

• Provide assistance to communities with shelter, food, water, health care, and functional and behavioral needs 

within 24 hours of an event or incident.
54

• Within 72 hours, move and deliver resources and capabilities to meet the needs of disaster survivors, 

including individuals with access and functional needs and others who may be considered to be at risk.

• Within 72 hours, establish, staff, and equip emergency shelters and other temporary housing options 

(including accessible housing), feeding operations, bulk distribution centers, and volunteer reception and 

donations centers for the affected population.

• Within 30 days, move from congregate care to non-congregate care alternatives and provide relocation 

assistance or interim housing solutions for families unable to return to their pre-disaster homes.

Critical Transportation: Provide transportation (including infrastructure access and accessible 

transportation services) for response priority objectives, including the evacuation of people and animals, 

and the delivery of vital response personnel, equipment, and services into the affected areas.

• Identify, within 24 hours of an incident, staging areas and other support locations for use by federal, state, 

local, and private partners. 

• Establish lines of supply into the impacted zone within 72 hours using seaports, airports, railroads, and roads 

in order to establish emergency power and fuel lines of supply. 

• Inspect, reestablish, and maintain land, sea, pipeline, and air routes within the first 72 hours after an incident 

in order to move first responders into the area affected by the incident and to safely sustain 650,000 persons 

in the vicinity of the incident or evacuate them out of the impacted zone, including up to 12,500 persons 

seriously injured and 43,000 persons with injuries not requiring hospitalization.

• Within 12 hours of the incidents, establish physical access through appropriate transportation corridors and 

within six hours of receiving a request deliver required resources to save lives and to meet the needs of 

disaster survivors for an event impacting 13,209 sq. miles and damaging or destroying 1,659 transportation 

lifelines.

• Type, stage, and provide transportation resources to responders/evacuees within one hour of request.

Public Information and Warning: Deliver coordinated, prompt, reliable, and actionable information to the 

whole community through the use of clear, consistent, accessible, and culturally and linguistically 

appropriate methods to effectively relay information regarding any threat or hazard and, as appropriate, 

the actions being taken and the assistance being made available.

• Provide warning of the expected impacts during an unfolding event (such as a flood) for 800,000 people, 

including messaging on how to take appropriate actions. 

• Coordinate local, state, and federal public information strategy within 12 hours after an incident to 

communicate information such as location and activation schedule of shelters and Family Assistance Centers 

as well as information related to missing persons and fatalities.

• Establish communications within 24 hours after an incident to update the populace at regular intervals in all 

counties, tribal areas, and surrounding states, as well as federal partners, on response efforts and other 

information (e.g., details of public/individual assistance programs) relevant to the incident.

• Employ multiple information dissemination platforms to include private media outlets, public address systems, 

and social media to distribute guidelines for additional assistance available to the public after the close of the 

immediate operational response.

• Within two hours of an incident and at the end of every operational period afterwards, inform all affected 

segments of society by all means necessary, including accessible tools, of critical lifesaving and lifesustaining information to expedite the delivery of emergency services and aid the public to take protective 

actions.

• Within two hours of an incident and at the end of every operational period afterwards, deliver credible 

messages to inform ongoing emergency services and the public about protective measures and other lifesustaining actions and facilitate the transition to recovery.

• Provide unified, timely, and consistent warning and guidance to public and responders within one hour for 

urgent situations and four hours for all others.

* Sources: California Governor’s Office of Emergency Services (undated), Missouri Office of Homeland Security 

(2012), and New Hampshire Homeland Security and Emergency Management (2012)
55

8.3.3 Funding (Grants)

Federal funds must be used for the purposes for which they were appropriated, and grants must 

be used for eligible costs. All-hazards grants have a broader purpose than RERP, but a purpose 

that could include RERP uses.

However, many States have funding specifically for RERP from fees levied on NPP licensees.

For example, Pennsylvania charges each of its five sites annual fees of $200,000 for a 

Radiological Emergency Planning and Preparedness Fund account and $150,000 for a 

Radiological Emergency Response Fund account within the General Fund. These funds are 

managed by the Pennsylvania Emergency Management Agency.10

States must maintain a foundational all-hazards capability and address other hazard-specific 

needs. Some of these hazards may not have dedicated sources of funding. Since most States 

with NPPs can cover RERP-specific needs with assessments on the licensees, States will 

generally use all-hazards grant funding for non-RERP purposes.

The dividing line between RERP and non-RERP all-hazards purposes may not always be clear.

For example, Pennsylvania law for its Radiation Emergency Response Program funding 

requires “development and maintenance of a current list of handicapped persons requiring 

assistance” (Pennsylvania Code 116.5(a)(2)(v)). This has benefits beyond RERP-related 

incidents. Similarly, RERP benefits from a foundation of all-hazards preparedness, including 

ensuring there is a professional emergency management staff to coordinate response if needed, 

as EMPG program funds help do. Yet RERP-specific funding at least covers RERP minimum 

requirements; all-hazards grants are then unlikely to replace or supplant that funding.

From a performance-based oversight perspective, the funding source is immaterial. However, to 

the extent licensees are the beneficiaries of ensuring adequate State and local ORO RERP 

performance, licensees will continue supporting the costs of RERP. The nuclear industry has 

expressed concern about including all-hazards matters in current RERP-related requirements in 

part because of this funding support.11

Funding considerations may be the most significant obstacle to integration. Currently funding is 

largely “siloed.” Efforts funded from either all-hazards or RERP-specific funds can have 

crossover benefits. However, funding is generally focused on compliant inputs and activities, not 

achieving a target level of performance. Under a purely performance-based RERP oversight 

system, there may be disagreement over whether licensees should fund an all-hazards activity 

as a corrective action, even if it could help jurisdictions meet related RERP-specific performance 

targets. Specifically, there may be disagreement over whether the expected impact on 

performance can be attributed to the proposed input/activity, and over the scope needed to 

achieve that impact.

 10 This is in addition to an annual fee of $550,000 per site for the Radiation Protection Fund managed by 

the Pennsylvania Department of Environmental Protection (see Pennsylvania Department of 

Environmental Protection 2010). Note also that an additional $300,000 is appropriated to the Radiological 

Emergency Planning and Preparedness Fund, beyond the $200,000 per site contribution. 

11 Regarding proposed changes to FEMA’s REP Program Manual, one NEI reviewer wrote: “Many 

changes or additions to this program manual are imposing new requirements and expectations that reach 

far beyond planning and responding to the unique aspects of a radiological event at a NPP (i.e., a “REP 

event”) ... subjecting OROs to evaluation against these generic emergency response elements” (FEMA

2011a, p. 92). 
56

8.3.4 Performance Input: Plans (CPG 101) 

All-hazards emergency operations plans must still account for requirements specific to single 

hazards. For example, a generic function of providing public information and warning may rely 

on some of the same organizations and resources to deliver messages regardless of hazard.

However, the timeline, specific target audience, and pre-scripted messages for hurricane 

evacuation would be different from those for a radiological release at an NPP. 

The National Response Framework accounts for this through “incident annexes.” The annexes 

describe specific requirements and arrangements for given hazards, yet can reference the allhazards elements of the framework rather than duplicate them as in a separate plan. CPG 101, 

FEMA’s guidance on planning (FEMA 2010a), features this hazard-specific incident annex 

approach but does not require it. In fact, CPG 101 notes that local communities “may find it 

appropriate to address specific hazards or threats in completely separate and stand-alone 

plans. In this case, the EOP must specifically reference those plans and provide a brief 

summary of how the EOP is to be coordinated with the stand-alone plans.”

FEMA also recognizes in CPG 101 that there are legal and regulatory requirements associated 

with some hazards. However, one State commenter on proposed REP Program Manual 

changes said: “CPG 101 the document contradicts the planning needs and evaluation criteria 

for NPP and is unsuitable for use in NPP planning.” FEMA responded that offsite RERP 

requirements are more specific, but that CPG 101 has useful information (see FEMA 2011a, p. 

53). 

From the all-hazards standpoint, FEMA leaves States and locals free to address RERP-specific 

requirements in any format they choose. Table 14, below, shows that over half the 31 States 

with NPPs use stand-alone plans to meet RERP requirements, although there may be 

summaries or other “bridging documents” in the all-hazards plans. Although many States have 

moved from stand-alone plans to hazard-specific annexes in all-hazards plans, planning 

integration is relatively limited.12 None of the 31 States relies solely on “generic” all-hazards

planning provisions to address RERP requirements.

Table 14. State Approaches to RERP Requirements in All-Hazards Planning

Separate RERP Plan

(referenced)

Separate RERP Plan and 

Bridging Document(s) in AllHazards Plan

RERP Annex or Annexes in AllHazards Plan

12 6 13

California, Connecticut, Georgia, 

Iowa, Illinois, Massachusetts, 

Mississippi, Missouri, Nebraska, 

New Jersey, Pennsylvania, 

Tennessee

Alabama, Arizona, Florida, 

Kansas, Louisiana, 

New York

Arkansas, Maryland, Michigan, 

Minnesota, New Hampshire, North 

Carolina, Ohio, South Carolina, 

Texas, Virginia, Vermont, 

Washington, Wisconsin

From a performance-based oversight perspective, plans would be immaterial as long as 

performance is acceptable. However, objective RERP performance measures would allow 

FEMA to correlate planning approaches and specific planning provisions to successful RERP 

performance. This could help justify any FEMA-driven corrective actions for adopting all-hazards 

guidance to support RERP efforts, where State and local ORO performance requires FEMAdetermined corrective action under the proposed RIPB RERP framework.

 12 For example, Minnesota consolidated its formerly separate plan into its all-hazards plan in 1995, 

according to the 2001 Minnesota Emergency Operations Plan.
57

8.3.5 Performance Input: Organization (National Incident Management System Incident 

Command System)

Supplement 4 to NUREG-0654 reiterates that NIMS ICS is not required for OROs who do not 

receive Federal preparedness assistance (p. 3). While FEMA has said it will not evaluate OROs 

on NIMS ICS compliance during REP exercises (FEMA 2011a, pp. 54-55, 91-93, 96-98, 419, 

520-521, 693, and 703), the 2013 REP Program Manual does seek consistency with NIMS ICS 

in plans “for those OROs that have adopted NIMS.” Specifically, their concepts of operations 

must be consistent with NIMS core doctrine, their organizational charts must use the five main 

organizational elements of NIMS ICS (Command, Planning, Operations, Logistics, and Finance 

and Administration), and they must identify who will carry out the five NIMS ICS functions

(FEMA 2013f, pp. II-6, II-7, and II-9). States have cautioned against requiring RERP plans to 

include more NIMS ICS content than was required for their Governors to certify local plans as 

NIMS compliant (FEMA 2011a, p. 54). 

Not all State-level RERP stakeholders consider that NIMS ICS benefits RERP. One commenter 

on NUREG-0654 Supplement 4 opposed how the State’s NIMS implementation added 

organizational layers between the State Radiation Control Officer and the State's Emergency 

Management decision makers, rather than enabling them to speak directly (FEMA 2011, p. 

242). It may be possible to develop a suggested NIMS ICS-compliant organization for NPP 

radiological emergencies that resolves this issue. In any case, FEMA describes the benefit of 

NIMS ICS to RERP as aiding mutual aid and interoperability: “The integration of NIMS/ICS into 

ORO emergency plans/procedures for NPPs will provide greater consistency across response 

jurisdictions and facilitate integration of response elements during an incident that affects a 

NPP” (FEMA 2013f, pp. II-5 to II-6).

Despite current efforts to integrate NIMS ICS into offsite RERP oversight, for purely 

performance-based oversight purposes NIMS ICS remains an input to performance. Personnel 

and resources are organized under NIMS ICS to accomplish something. Performance-based 

oversight is focused on whether the accomplishment occurs, not how it was organized. NIMS 

ICS organization is therefore not material to performance-based oversight.

8.3.6 Performance Input: Equipment (National Incident Management System Resource 

Typing)

Current RERP-related policy does not specifically address NIMS resource typing efforts, which 

address equipment or teams of personnel and their associated equipment. FEMA’s National 

Integration Center has developed several typed resource definitions, but few elements are 

specific to radiological incidents (FEMA 2013i). Table 15 on the next page shows radiological 

detection requirements for Hazardous Materials Entry Teams. There are no current definitions 

for teams of technical specialists and advisors on radiological issues.

Typing resources most likely to be needed for response to a radiological emergency 

surrounding an NPP could be useful for mutual aid and eventually in defining any performance 

targets related to mobilization (i.e., the ability to deploy some quantity of a typed capability by a 

given time). However, it is not necessary for specifying performance standards. Resources are 

organized to be able to accomplish something. Performance-based oversight is focused on 

whether the accomplishment occurs, not the specific resources used in accomplishing it. NIMS 

ICS resource typing is therefore not directly material to performance-based oversight.
58

Table 15. NIMS Typed Resource Definition with Radiological Detection Capability

(FEMA 508-4, 2005)

Resource: Hazmat Entry Team

Category: Hazardous Materials Response (ESF #10) Kind: Team

Minimum Capabilities: Type I Type II Type III Type IV Other

Component Metric

Team Sampling:

Capturing, 

Labeling, 

Evidence 

Collection

Same as Type 

III plus:

(WMD Chem 

Bio)

Special 

resources may 

be required for 

air sample 

collection.

Same as Type 

III plus:

(Unknown 

Industrial 

Chemicals)

Known and 

unknown 

industrial 

chemicals

standard 

evidence 

collection 

protocols.

Ability to sample 

liquids and 

solids.

(Known 

Industrial 

Chemicals)

Known 

industrial 

chemicals 

standard 

evidence 

collection 

protocols for 

each include: 

capturing and 

collection, 

containerizing 

and proper 

labeling, and 

preparation for 

transportation 

and 

distribution, 

including 

standard 

environmental 

sampling 

procedures for 

lab analysis.

Consistent with 

established 

chain of 

custody 

protocols.

Radiation 

Monitoring/ 

Detection

Identify and 

establish the 

exclusion 

zones after 

contamination 

spread (this 

does include 

identification of 

some, but not 

all, 

radionuclides).

Ability to 

conduct 

environmental 

and personnel 

survey.

Ensure all 

members of 

survey teams 

are equipped 

with 

accumulative 

self-reading 

instruments 

(dosimeters).

Same as Type 

III plus:

(Alpha 

detection)

Basic criteria 

include 

detection and 

survey 

capabilities for 

alpha, beta, and 

gamma.

(Beta 

detection, 

Gamma 

detection)

The ability to 

accurately 

interpret 

readings from 

the radiation 

detection 

devices and 

conduct 

geographical 

survey search 

of suspected 

radiological 

source or 

contamination 

spread.

Basic criteria 

include 

detection and 

survey 

capabilities for 

beta and 

gamma.
59

8.3.7 Performance Input: Training (National Incident Management System Qualification 

Systems)

NIMS relies on ensuring that a term means the same thing across jurisdictions and disciplines.

This extends to NIMS position titles. For the titles to mean the same thing means that anyone 

occupying a NIMS position must be “qualified” in that position. Qualification means both 

completing formal training and gaining related experience, including experience in progressively 

more complex incidents. Usually a “position task book” defines the training and experience 

requirements for a position.

The NIMS Training Program provides NIMS training expectations surrounding a core curriculum 

and different levels of incident complexity. FEMA states that its guidance is not absolute; most 

of the guidance uses “should” rather than “must” (DHS 2011, p. 16). None of the NIMS core 

curriculum specifically addresses RERP, although it would be possible to tailor NIMS training to 

RERP concerns through use of RERP-related examples.

Likewise, current RERP policy does not specifically address NIMS training. FEMA’s 2013 REP 

Program Manual addresses requirements for “radiological emergency response training” for 

different elements of the ORO. The requirements are for plans and procedures to “discuss” 

training these different ORO groups; no content is specified beyond notifications and radiation 

protection (FEMA 2013f, p. II-138). One State commenter has noted that “there is no 

established training curriculum, no requirement regarding the organizations that must be 

trained, and no requirement about how many personnel must be trained” (Engelhart 2013). 

NIMS training supports NIMS organization, and the potential benefit to RERP is to facilitate 

mutual aid and augmentation for OROs. However, like the NIMS organizational construct and 

NIMS resource typing, NIMS training is an input to performance and neither directly supports 

nor conflicts with a performance-based oversight construct for RERP. ORO personnel required 

to use NIMS will be required to have some level of NIMS training regardless of the RERP 

oversight construct.

8.3.8 Performance Input: Exercise Doctrine (Homeland Security Exercise and 

Evaluation Program)

Similar to NIMS, HSEEP doctrine describes a common approach—terminology, processes, and 

document templates—for managing exercise programs and designing, developing, conducting, 

and evaluating individual exercises. HSEEP is not about compliance. HSEEP doctrine is 

“flexible, scalable, [and] adaptable ... . Exercise practitioners are encouraged to apply and 

adapt HSEEP doctrine to meet their specific needs” (DHS 2013, p. Intro-2). Per NUREG 

0654/FEMA-REP-1, Rev. 1, Supplement 4 (pp. 3-4), HSEEP’s key features are the following:

(1) Exercise scheduling through Training and Exercise Program Workshops documented in 

multi-year TEPs,

(2) Use of certain documents for exercise conduct, developed according to certain planning 

meeting milestones,

(3) An AAR and appended improvement plan (IP) in a certain format (organized by core 

capabilities, etc.), and

(4) Tracking of corrective actions identified in the AAR/IP.

As noted in Section 8.2.2, FEMA grantees must develop TEPs and “the use of HSEEP is 

strongly encouraged” when developing AAR/IPs (FEMA 2013b, p. 61). FEMA Directive 123-15 

of January 16, 2009 also required all FEMA programs, including REP, to use HSEEP (Spence 

2013). Accordingly, FEMA and its grantees that are OROs have made significant progress in 
60

integrating HSEEP doctrine in the current RERP oversight framework. For example, of 12 

States with NPPs whose TEPs are online, all but one mentions the REP program and REP 

exercises.13 FEMA directed its regional REP staff to begin using HSEEP and conducted three 

integration exercises to test application of HSEEP processes to REP. FEMA concluded that 

there “was no need to modify” how REP exercises were conducted in order to integrate HSEEP 

doctrine. There were, in fact, some benefits to following HSEEP guidelines on exercise 

documents, such as the exercise plan and controller/evaluator handbook (FEMA 2010). The 

REP program has continued using HSEEP doctrine since 2009, including the revised and 

simplified HSEEP doctrine of 2013. 

For performance-based oversight, HSEEP guidance may help develop consistent performance 

demonstrations. It is not necessary—as FEMA noted, using HSEEP doctrine did not alter how 

exercises were conducted—but it does not conflict with performance-based oversight. (Section 

8.3.10 below addresses HSEEP evaluation processes.)

8.3.9 Categorization and Reporting: Core Capabilities

Oversight of response preparedness requires categories of response activities to align direction, 

guidance, and reporting. The National Preparedness Goal defines 14 categories—of a total of 

31 core capabilities—as applicable specifically to the response mission and “necessary to save 

lives, protect property and the environment, and meet basic human needs after an incident has 

occurred” (see Table 16, next page). FEMA and its grantees use the core capabilities to set 

performance targets, devise plans, evaluate exercise performance, and assess the status of 

preparedness.

FEMA has mapped assessment areas from its current RERP oversight system to the 31 core 

capabilities—and had mapped them to the core capabilities’ predecessor, the target capabilities 

list (TCL), with its 37 capabilities. This was part of FEMA’s effort to incorporate HSEEP 

evaluation procedures into RERP oversight. That FEMA was able to redefine preparedness 

categories from 37 to 31, and that RERP oversight was able to adapt to the change, shows that 

categorization does not affect the underlying substance of homeland security and emergency 

management. Similarly, a crosswalk between task areas for the proposed RIPB RERP construct 

and the core capabilities is feasible. Table 17 (following Table 16 on the next page) gives a 

crosswalk between the 14 response core capabilities and both the current RERP assessment 

areas and the task areas in the proposed RIPB RERP construct. The only response-related 

core capabilities not addressed are Fatality Management and Mass Search and Rescue 

Operations, given the nature of the hazard, and Planning, given that both the assessment areas 

and the RIPB RERP task areas focus on performance demonstrations.14

Since mapping one construct to another is feasible, the proposed RIPB RERP construct could 

simply have used the core capabilities. It did not do so in order to preserve some specificity. The 

all-hazards core capability definitions are broad; they require another layer of RERP-specific 

content (i.e., a set of RERP-specific performance targets) to be meaningful. For example, the 

Critical Transportation core capability combines evacuation and movement of response 

resources. Further, the all-hazards categorization may change in the future. By working instead 

from the specific requirements for protecting public health and safety in a radiological 

emergency at an NPP, the proposed RIPB RERP construct preserves those terms and 

requirements for integration with any all-hazards categorization.

 13 Vermont’s 2012 TEP does not, but the document seems to be a draft though marked “final.” Vermont’s 

2010 TEP did mention REP exercises. The other 11 States are: Arkansas, Florida, Louisiana, Michigan, 

Minnesota, Missouri, Nebraska, Pennsylvania, South Carolina, Washington, and Wisconsin.

14 Table 17 draws from the crosswalk at FEMA (2013f), pp. IV-19 to IV-20.
61

Table 16. Response Core Capabilities from the National Preparedness Goal

(DHS 2011a)

Core Capability Objective

Planning

Conduct a systematic process engaging the whole community in development of 

executable strategic, operational, and/or community based approaches to meet defined 

objectives

Public Information and 

Warning

Deliver coordinated, prompt, reliable, and actionable information to the whole 

community through the use of clear, consistent, accessible, and linguistically 

appropriate methods to effectively relay information regarding any threat or hazard and 

actions being taken/assistance being made available

Operational 

Coordination

Establish and maintain a unified and coordinated operational structure and process that 

appropriately integrates all critical stakeholders and supports the execution of core 

capabilities

Critical Transportation

Provide transportation for response priority objectives, including the evacuation of 

people and animals, and the delivery of vital response personnel, equipment, and 

services to the affected areas

Environmental 

Response/Health and 

Safety

Ensure the availability of guidance and resources to address all hazards, including 

hazardous materials, acts of terrorism, and natural disasters, in support of responder 

operations and the affected communities

Fatality Management 

Services

Provide fatality management services, including body recovery and victim identification, 

working with state and local authorities to provide temporary mortuary solutions, sharing 

information with Mass Care Services for the purpose of reunifying family members and 

caregivers with missing persons/remains

Infrastructure Systems Stabilize critical infrastructure functions, minimize health and safety threats, and restore 

and revitalize systems and services 

Mass Care Services

Provide life-sustaining services to the affected population with a focus on hydration, 

feeding, and sheltering to those with the most need, as well as support for reunifying 

families

Mass Search and 

Rescue Operations

Deliver traditional and atypical search and rescue capabilities, including personnel, 

services, animals, and assets to individuals in need, with the goal of saving the greatest 

number of lives in the shortest time possible

On-Scene Security and 

Protection

Ensure a safe and secure environment through law enforcement and related security 

and protection operations for people and communities located within affected areas in 

the impact area and all response forces

Operational 

Communications

Ensure the capacity for timely communications in support of security, situational 

awareness, and operations by any and all means available, among and between 

affected communities in the impact area and all response forces.

Public and Private 

Services and 

Resources

Provide essential public and private services and resources to the affected population 

and surrounding communities, to include emergency power to critical facilities, fuel 

support for emergency responders, and access to community staples (grocery stores, 

pharmacies, and banks) and fire and other first response services

Public Health and 

Medical Services

Provide lifesaving medical treatment via emergency medical services and related 

operations, and avoid additional disease and injury by providing targeted public health 

and medical support and products to all people in need within the affected area

Situational Assessment Provide all decision makers with decision relevant information regarding the nature and 

extent of the hazard, any cascading effects, and the status of the response
62

Table 17. Crosswalk of Response Core Capabilities, Current REP Program Manual

 Assessment Areas, and Tasks in the Proposed RIPB RERP Oversight 

 Construct

Common Response

PlanningPublic Information and WarningOperational CoordinationCritical TransportationEnvironmental Response/ Health and SafetyFatality Management ServicesInfrastructure SystemsMass Care ServicesMass Search and Rescue OperationsOn-Scene Security and ProtectionOperational CommunicationsPublic and Private Services and ResourcesPublic Health and Medical ServicesSituational Assessment

FEMA REP Program Manual 

Assessment Areas

Emergency 

Operations 

Management

X X X X X X X

Protective Action 

Decision-Making X X X X X

Protective Action 

Implementation X X X X X

Field 

Measurement and 

Analysis

X X X X

Emergency 

Notification and 

Public Information

X X X

Support 

Operations/

Facilities

X X X X

Tasks in Proposed RIPB RERP Construct

Receive Licensee 

Notification of 

Emergency

X

Understand the

Notification X

Notify Appropriate 

Officials X X

Assess the 

Situation X

Make a Protective 

Action Decision X

Mobilize 

Response 

Components

X X

Notify the Public X X

Implement 

Protective Action 

Decision

X X X X X X

Receive, Screen, 

and Support 

Evacuees

X X X X

Manage Exposure 

and Contamination 

Risks

X X X

Make Area of 

Impact Safe for 

Public Return

X

Facilitate 

Community 

Restoration and 

Recovery

X
63

In sum, integration of the core capabilities as categories for organizing and reporting results of 

performance demonstrations occurs in the current RERP oversight system, and could be 

accommodated in the proposed RIPB RERP construct. Results generated under the current 

assessment areas or under the proposed RIPB RERP task areas can be categorized in terms of 

core capabilities. This is an overlay that neither supports nor conflicts with RERP content and 

performance-based oversight.

8.3.10 Categorization and Reporting: Exercise Evaluation (Homeland Security Exercise 

and Evaluation Program)

HSEEP structures exercise evaluation around exercise evaluation guides, or EEGs. For each 

core capability to be assessed under a given exercise’s objectives, the EEG lists one or more 

“organizational capability targets” for the core capability, and one or more “critical tasks” for 

each organizational capability target. The critical tasks give evaluators points of review for 

assessing whether organizational capability targets are met. Figure 9 on the next page provides 

a sample EEG.

FEMA has developed a set of HSEEP EEGs for RERP oversight. FEMA used its assessment 

areas as the organizational capability targets, and its specific demonstration criteria as the 

critical tasks. The proposed RIPB RERP oversight construct could use the same procedure, 

with task areas (or subtask areas) as the organizational capability targets, and specific 

performance targets as the critical task descriptions. 

Under HSEEP, evaluators grade organizational capability targets and the overall core capability 

on a four-step rating scale. As shown in Table 18 (on the page following Figure 9), this fourstep scale is compatible with, if not exactly equivalent to, four-step evaluation scales under both 

current RERP oversight and proposed RIPB RERP oversight. Results under one system’s rating 

scale could be translated to the other system’s rating scale for reporting and comparison 

purposes.

Some RERP stakeholders have noted that HSEEP has a “no-fault” evaluation approach that is 

incompatible with RERP’s requirements, and that HSEEP calls for more self-evaluation and selfdriven corrective action than RERP currently does. The idea in HSEEP is to focus on potential 

corrective actions, not blame individuals and organizations for inadequate performance.15

 15 Current HSEEP doctrine (DHS 2013) uses the term “no-fault” only once, in describing evaluation of 

tabletop exercises. It also uses the term “non-attribution environment” once. The terms are not defined or 

discussed. See pp. 2-5 and 5-4. 
64

EXERCISE EVALUATION GUIDE

Exercise Name: [Insert exercise name]

Exercise Date: [Insert exercise date]

Organization/Jurisdiction:

[Insert organization or jurisdiction]

Venue:

[Insert venue name]

Response

Exercise Objective: [Insert exercise objective]

Core Capability: Operational Communications

Ensure the capacity for timely communications in support of security, situational awareness, and operations by any and all means available, among and 

between affected communities in the impact area and all response forces.

Organizational Capability Target 1: [Insert customized target based on plans and assessments]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Source(s): [Insert name of plan, policy, procedure, or reference]

Organizational Capability Target 2: [Insert customized target based on plans and assessments]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Source(s): [Insert name of plan, policy, procedure, or reference]

Organizational Capability Target 3: [Insert customized target based on plans and assessments]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Critical Task: [Insert task from frameworks, plans, or SOPs]

Source(s): [Insert name of plan, policy, procedure, or reference]

Organizational 

Capability 

Target

Associated Critical Tasks Observation Notes and 

Explanation of Rating

Target 

Rating

[Insert 

Organizational 

Capability 

Target 1 from 

page 1]

• [Insert Organizational Capability 

Target 1 Critical Tasks from page 

1]

[Insert 

Organizational 

Capability 

Target 2 from 

page 1]

• [Insert Organizational Capability 

Target 2 Critical Tasks from page 

1]

[Insert 

Organizational 

Capability 

Target 3 from 

page 1]

• [Insert Organizational Capability 

Target 3 Critical Tasks from page 

1]

Final Core Capability Rating

Evaluator Name ________________________________________________ Ratings Key

Evaluator E-mail ________________________________________________ P – Performed without Challenges

S – Performed with Some Challenges

M – Performed with Major Challenges

U – Unable to be Performed

Phone _________________________________________________________

[PROTECTIVE MARKING, AS APPROPRIATE]

Homeland Security Exercise and Evaluation Program (HSEEP)

Figure 9. Example of an HSEEP EEG

Source: DHS (2013a)
65

Table 18. Rating Scales for HSEEP and RERP Exercise Evaluations

System Rating Rating Definition HSEEP

P –

PERFORMED 

WITHOUT 

CHALLENGES

Targets and tasks were completed in a manner that achieved the 

objective(s) and did not negatively impact the performance of other 

activities. Performance of this activity did not contribute to additional health 

and/or safety risks for the public or for emergency workers, and it was 

conducted in accordance with applicable plans, policies, procedures, 

regulations, and laws.

S –

SOME CHALLENGES

Targets and tasks were completed in a manner that achieved the 

objective(s) and did not negatively impact the performance of other 

activities. Performance of this activity did not contribute to additional health 

and/or safety risks for the public or for emergency workers, and it was 

conducted in accordance with applicable plans, policies, procedures, 

regulations, and laws. However, opportunities to enhance effectiveness 

and/or efficiency were identified.

M –

MAJOR CHALLENGES

Targets and tasks were completed in a manner that achieved the 

objective(s), but some or all of the following were observed: demonstrated 

performance had a negative impact on the performance of other activities; 

contributed to additional health and/or safety risks for the public or for 

emergency workers; and/or was not conducted in accordance with 

applicable plans, policies, procedures, regulations, and laws.

U –

UNABLE TO PERFORM

Targets and tasks were not performed in a manner that achieved the 

objective(s).

REP

(Current RERP Oversight)

MET

The participating ORO performed all activities for the criterion to the level 

required in the Extent-of-Play Agreement, with no Deficiencies or Areas 

Requiring Corrective Action assessed in the current exercise for that 

criterion and no unresolved prior Areas Requiring Corrective Action.

PLAN ISSUE This refers to an identified inadequacy in the organization’s emergency 

plan/ procedures, rather than in the organization’s performance.

AREA REQUIRING 

CORRECTIVE ACTION

(ARCA)

This refers to an observed or identified inadequacy of organizational 

performance in an exercise that is not considered, by itself, to adversely 

impact public health and safety.

DEFICIENCY

This refers to an observed or identified inadequacy of organizational 

performance in an exercise that could cause a finding that offsite 

emergency preparedness is not adequate to provide reasonable assurance 

that appropriate protective measures can be taken in the event of a 

radiological emergency to protect the health and safety of the public living 

in the vicinity of an NPP.

Proposed RIPB RERP 

Oversight Construct

GREEN Performance was at a level with no anticipated negative consequences for 

public health and safety. (Generally a quantitative target.)

WHITE Performance was at a level that could result in a minor reduction to public 

health or safety. (Generally a quantitative target.)

YELLOW Performance was at a level that could result in a moderate reduction to 

public health or safety. (Generally a quantitative target.)

ORANGE

Performance was at a level not allowing reasonable assurance that the 

ORO’s RERP capabilities would adequately protect public health and safety 

if needed. (Generally a quantitative target.)

8.3.11 Categorization and Reporting: Exercise Evaluation (National Exercise Program)

The NEP is the primary exercise program for measuring national preparedness. With the 

revised NEP Base Plan and NEP Implementation Plan, FEMA redesigned the NEP to capture 

information about Federal senior leaders’ priorities from a cross-section of exercises around the 

Nation, rather than rely on one large national-level exercise and a handful of tabletop exercises 

to identify key preparedness issues for senior leaders. In the new NEP, FEMA develops a 

quarterly Rolling Summary Report identifying trends from exercises accepted into the NEP.

FEMA maps this information to the core capabilities.
66

FEMA has not publicly released a NEP Rolling Summary Report or even a listing of exercises 

included within the NEP. At least one REP exercise was selected for the NEP; many more could 

be to help the NEP have a large sample size for trend analysis. Yet in any case, the NEP uses 

findings from the individual exercises to assess trends. Results of any individual RERP-related 

exercise would already be available within the RIPB RERP oversight process; thus the NEP and 

its macro-level trend reporting would not add to RIPB RERP oversight.

8.3.12 Categorization and Reporting: Assessments (National Preparedness 

Report, State Preparedness Report)

States receiving FEMA all-hazards grants must provide an annual SPR. The SPR requires an 

assessment of planning, organization, equipment, training, and exercise effectiveness in each of 

the 31 core capabilities. FEMA uses the SPR results in the NPR, showing the Nation’s progress 

towards meeting the National Preparedness Goal.

The 2013 NPR is silent on RERP. Nuclear and radiological matters are mentioned only in 

connection with Department of Defense capabilities for response to terrorism involving 

chemical, biological, radiological, nuclear, and explosive elements. There is no comparison of 

RERP States to non-RERP States in the aggregate, for example. With an RIPB RERP oversight 

construct, FEMA would have a source of objective, quantitative metrics that could be mapped to 

the core capabilities, offering additional data points on the Nation’s preparedness for the NPR.

However, reporting of all-hazards assessment results—particularly aggregate results not 

focused on a given State or site—will neither contribute to nor detract from performance-based 

oversight of RERP. Like RERP planning requirements, reporting of RIPB RERP results must be 

addressed, but whether that reporting is separate or is integrated within an all-hazards 

document (e.g., as an appendix) will not affect RIPB RERP results.

8.4 ALL-HAZARDS INTEGRATION POTENTIAL: ONSITE

Current integration of all-hazards initiatives with onsite emergency preparedness is extremely 

limited. Adoption of a new RIPB RERP oversight construct offsite is unlikely to change that.

None of the all-hazards initiatives discussed above is a requirement for private sector licensees.

The legal authorities underpinning the initiatives focus on what Federal agencies must do, 

including requiring compliance as a condition of receiving Federal preparedness assistance. 

FEMA, in implementing the initiatives, exercises no authority over the private sector.

The all-hazards initiative most relevant to licensee NPP EP efforts is NIMS, because licensees 

will interact with OROs in emergencies and OROs will use NIMS organization and terminology.

HSPD-5, the source of the NIMS requirement, creates no private sector requirement. HSPD-5 

says NIMS is meant to “provide a consistent nationwide approach for Federal, State, and local 

governments to work effectively and efficiently together” (emphasis added). HSPD-5 requires 

the NRC to use NIMS/ICS in its own response plans, but not to make NIMS a regulatory 

requirement for licensees.

The NRC’s guidance (NSIR/DPR-ISG-01) acknowledges NIMS without requiring it: 

Because ORO responders to an NPP event will conduct and coordinate their response 

efforts in accordance with NIMS/ICS concepts, licensee plans and procedures should 

establish the protocols and interfaces that will allow OROs to effectively support onsite 

activities, consistent with incident command concepts established in State and local 

emergency response plans. ... Personnel in ERO [emergency response organization] 
67

positions with ORO interface responsibilities should be familiar with incident command 

concepts, position titles, and terminology consistent with State and local emergency 

response plans and procedures. These ERO personnel should have a listing of ORO 

incident command terms and definitions to ensure that these terms are clearly 

understood during communications with ORO personnel.

In its Federal Register Notice revising emergency preparedness rules, the NRC explained why it 

did not require licensees to adopt NIMS/ICS (76 FR 72574-72575). The NRC cited specific 

performance outcomes to be achieved, such as “prompt communications” with the ORO, and 

said its regulation was adequate to ensure that outcome. Further, the NRC raised the possibility 

that NIMS/ICS requirements could change, requiring the NRC to issue new rules for licensee 

compliance. By focusing on the performance outcome, the NRC has given licensees freedom to 

adopt NIMS/ICS (or not), and the ability to adapt.

For its part, NEI (2010) has encouraged licensees to be familiar with NIMS/ICS: “Lead ERO 

facility managers, and liaisons to the ICP [incident command post], should be broadly familiar 

with the concepts and principles of [NIMS] and [ICS], as practiced by their State and local 

authorities. ... This is not to suggest that NIMS/ICS training be required for any ERO member.”

As the NRC considers new post-Fukushima rulemaking on onsite emergency capabilities for 

severe accidents, the NEI has maintained that “essential attributes of command and control 

should be defined at the guidance level. Licensees should retain the flexibility to address these 

attributes within the context of their existing emergency preparedness and response capabilities 

and applicable Owners Group guidance.” This suggests the industry is not eager to adopt 

NIMS/ICS and associated training requirements. In this climate, it seems equally unlikely that 

the industry would pursue NIMS resource typing for its FLEX mutual aid resources.

Adoption of an RIPB RERP oversight construct will not lead directly to greater all-hazards 

compliance and integration in NPP EP efforts. The RIPB RERP construct focuses on offsite, 

State and local preparedness to achieve defined outcomes, as gauged in performance 

demonstrations. It creates no new authority over licensee NPP EP efforts and no new 

requirement for licensee NPP EP efforts.

Indirectly, the use of objective, quantifiable performance metrics in RIPB RERP may help 

develop evidence to support regulation, guidance, or voluntary industry standards. It may 

become possible to show, for example, a correlation between higher ORO performance and 

licensee adoption of NIMS ICS, should a number of licensees choose to adopt NIMS ICS.

8.5 ALL-HAZARDS INTEGRATION PROSPECTS: SUMMARY

Table 19 on the next page summarizes the review of integration potential in Section 8.3.

The proposed RIPB RERP oversight construct could most easily and usefully integrate HSEEP, 

refining FEMA’s already substantial integration of HSEEP in RERP oversight. Otherwise, 

prospects are not promising. As long as RERP has more specific, stand-alone requirements

supported by separate funding, there is little driving the part (RERP) to assimilate the whole (allhazards). As noted in this chapter, all-hazards performance targets could become at least as 

stringent as RERP performance targets if a more comprehensive—and therefore burdensome—

THIRA process required their inclusion for any State with an NPP (or with a portion of a 10-mile

EPZ): RERP requirements would become the default unless another hazard imposed more 

difficult requirements. Yet even then, RERP-specific oversight would be needed to ensure at 

least the RERP-specific mandates and requirements were met.
68

Table 19. Factors for Assessing Integration Prospects of All-Hazards Initiatives with

 RIPB RERP

All-Hazards Initiative 

or Element

Current Level 

of Integration

(+)

Potential 

RERP Utility 

to AllHazards 

Initiative

(+)

All-Hazards 

Initiative’s 

Potential 

Utility to 

RERP

(+)

Supports 

PerformanceBased 

Oversight

(+)

Conflicts with 

PerformanceBased 

Oversight

(-)

Risk Prioritization 

(THIRA) Medium Medium Low Low Low

Performance Targets Low Medium Medium Medium Low

Funding (Grants) Low Medium Medium Low Low Performance Input

Plans 

(CPG 101) Low Medium Low Low Low

Organization 

(NIMS ICS) Medium Low Medium Low Low

Equipment 

(NIMS 

Resource 

Typing)

Low Low Medium Low Low

Training (NIMS 

Qualification 

Systems)

Low Low Medium Low Low

Exercise 

Doctrine 

(HSEEP)

High Low Medium Medium Low

Categorization and 

Reporting

Core 

Capabilities Medium Low Low Low Low

Exercise 

Evaluation 

(HSEEP)

High Low Low Medium Low

Exercise 

Evaluation 

(NEP)

Medium Medium Low Low Low

Assessments 

(NPR/SPR) Low Medium Low Low Low

Most striking in Table 19 is that nothing in the all-hazards initiatives conflicts with performancebased oversight, even if an initiative is not directly supportive of performance-based oversight. 

All-hazards initiatives have tended to focus on performance inputs and processes, not on 

defined performance outcomes. Because a performance-based approach is focused on 

assessing outcomes rather than prescribing how to achieve outcomes, it increases freedom to 

integrate all-hazards requirements, concepts, and terminology. Leaving such input and process 

concerns in the ORO (or licensee) “band” of discretion may actually lead to increased 

integration as State and local OROs pursue overall efficiencies.

Where the ORO loses discretion because of performance deficiencies, FEMA may still be 

motivated to promote adoption of all-hazards concepts. Use of consistent, objective, quantitative 

performance metrics in an RIPB RERP construct could help FEMA develop evidence for which 

all-hazards concepts may best aid RERP performance. Such evidence becomes critical in a 

truly performance-based system. If requirements are defined in terms of performance outcomes, 

all inputs to performance are moot unless they are shown to improve performance outcomes.

Research to marshal such evidence and mechanisms to disseminate evidence may be an 

important adjunct to RIPB oversight.

FEMA (and State and local OROs) may need that evidence as long as licensees separately 

fund RERP oversight. Licensees will want the funds they provide applied to a RERP purpose. At 

the Federal level, Congress authorizes the collection for RERP purposes. Satisfying a RERP 
69

purpose may have all-hazards benefits, but licensees will ask that satisfying an all-hazards 

purpose with licensee funding satisfy a RERP purpose.

Even with evidence, FEMA should temper making adoption of any all-hazards practice a 

requirement in corrective actions with the recognition that all-hazards concepts, terms, and 

practices are not stable. The switch from the Target Capabilities List (TCL) to Core Capabilities 

is one example. CPG 101 guidance on planning, CPG 201 guidance on THIRAs, and the NEP

all were revised between one and four years after issuance. 

By the time the NRC and FEMA are ready to consider adopting an RIPB RERP oversight 

construct for OROs, all-hazards concepts and requirements may have become stable. Defined 

targets for all-hazards performance outcomes may exist that meet or exceed the requirements 

of radiological emergencies. Use of all-hazards concepts, terminology, and processes may be 

shown to have a direct bearing on meeting the performance targets. And oversight of State and 

local all-hazards preparedness may somehow become quasi-regulatory in its own right. If that 

happens, integration may mean replacing the RIPB RERP construct with an all-hazards one.

Yet until then, integration will mean much the same as it does now: looking for improved 

interfaces between RERP and all-hazards preparedness.

71

9. IMPLEMENTATION CONSIDERATIONS

Earlier chapters have outlined a rationale for considering an RIPB RERP oversight regimen, 

provided a conceptual framework for that regimen, and examined how this RIPB RERP 

oversight framework might coexist with all-hazards preparedness initiatives. This chapter briefly 

addresses practical aspects of making the concept a reality, should the NRC and FEMA ever 

choose to do so.

9.1 POTENTIALLY INCREASED RESOURCE BURDENS FOR FEDERAL OVERSIGHT

A purely performance-based oversight system will require more frequent demonstrations of 

performance, to provide reasonable assurance that OROs can reliably meet performance 

targets. Although State requirements for non-Federally-evaluated RERP exercises vary, the 

required minimum demonstration frequencies proposed in Section 6.2 exceed some States’ 

requirements for exercises and drills.

The Federal Government—FEMA—may choose not to evaluate all of these exercises directly.

However, to offset the loss of information from regular reviews of plans and other inputs, FEMA 

will need to observe and evaluate more demonstrations directly than with the current biennial 

exercise focus. Whether this increased evaluation workload for demonstrations is met through 

FEMA staff, FEMA contractors, or through a FEMA-coordinated network of peer evaluators, this

element of FEMA’s oversight burden will increase.1

It is likely the increase in Federal oversight burden for demonstrations will offset any savings 

from reduced compliance monitoring of plans.

It is also possible that routine non-site-specific activities could increase, should FEMA pursue a 

program of research to develop evidence and “lessons learned” to support corrective action 

recommendations.

Qualitatively, then, the level of resources required for routine oversight would likely be more 

than under the current system. 

Without firm data, how much more can only be a rough order of magnitude estimate. Consider 

the following unverified, illustrative assumptions as the basis of a rough order of magnitude 

estimate:

(1) Half (50 percent) of FEMA’s FY2013 $37.3 million REP program budget, or $18.7 

million, was for site-specific biennial exercise activity;

(2) Half of the remaining budget (i.e., $9.3M) is saved from reduced compliance activities 

(plan reviews, determining adequacy of plans);

(3) No additional savings accrue as former guidance and technical assistance activities 

transition to research on lessons learned for corrective action purposes; and

 1 In its fiscal year 2013 Congressional justification for the REP program budget, FEMA planned to 

evaluate approximately 30 biennial exercises and requested budget authority of $37.3 million for 

collections from licensees. A portion of this funding supports 170 full-time equivalent personnel. Flat fee 

and exercise-specific costs or labor hour breakdowns are not provided in the document (or in annual 

Federal Register notices setting the hourly rate to be charged to licensees for site-specific biennial 

exercise activities). See FEMA (2012).
72

(4) The site-specific exercise activity will double (i.e., require another $18.7 million) under 

the proposed oversight regimen.2

Under these illustrative assumptions, FEMA’s oversight budget would need to increase by one 

quarter, or more than $9 million.

9.2 IMPACT OF IMPLEMENTATION ON OROS

With increased demonstration requirements, the resource burden would increase for at least 

some OROs if the proposed RIPB RERP oversight construct were implemented.

OROs may over time seek to include relevant RERP performance targets in THIRAs, as a basis 

for enhancing all-hazards capabilities to better support RERP. 

However, OROs will face transition costs of their own before any new RIPB RERP oversight 

regimen can be implemented. State laws and regulations on fee collections from licensees and 

their appropriate uses may need to change to harmonize with Federal requirements. (States 

may elect to maintain requirements for performance “inputs” such as plans, training, 

procurement of specialized equipment, etc., in the service of meeting Federal performance 

targets.)

Further, depending on implementation, States and their publics may face a period of uncertainty 

for “reasonable assurance” in offsite RERP. Performance-based oversight is retrospective. It 

relies on past performance for reasonable assurance regarding future performance. Initially 

there may be no past performance in terms of the new performance targets that provide the new 

basis for reasonable assurance. Also, plan compliance will have been defined as an insufficient 

(or at least, outmoded) basis for reasonable assurance going forward. Messaging the transition 

carefully will be important. The message must be that the new regimen is intended to achieve a 

greater degree of and more objective assurance regarding RERP; this does not invalidate

previous reasonable assurance determinations.

9.3 REQUIRED CHANGES IN EXISTING REGULATIONS AND GUIDANCE

Shifting from a plans-based to a purely performance-based oversight approach for RERP would 

be a radical change. Putting a new regulatory framework in place would be a significant 

undertaking after more than 30 years of incremental change and institutionalization of the 

current framework. This section discusses some elements needing change, without proposing 

specific regulatory language.

9.3.1 10 CFR 50.47, Emergency Plans, and 10 CFR 50 Appendix E 

These NRC regulations define emergency preparedness on the basis of plans and planning 

standards, which apply both to licensees and to OROs. FEMA’s role is defined as making 

determinations on “whether State and local emergency plans are adequate and whether there is 

 2 Note that under the current regimen the focus is on a biennial exercise, and under the proposed 

regimen the focus is on a biannual functional exercise: a quadrupling, not doubling, of major 

demonstrations. The assumption here is that State and local OROs may be able to evaluate some 

exercise and demonstration activity already conducted separately under State auspices, and count it 

towards the required frequency of demonstrations. FEMA would send smaller-than-usual FEMA observer 

presence to at least a sample of these, to monitor the ORO evaluation and corrective action process.
73

reasonable assurance that they can be implemented.” 10 CFR 50, Appendix E, IV.F.2.c requires 

that licensees conduct exercises biennially with OROs.

To support the proposed RIPB RERP construct, new language must emphasize that the NRC’s 

“reasonable assurance that adequate protective measures can and will be taken in the event of 

a radiological emergency” will, at least where OROs are concerned, be derived from FEMA 

findings on reliably meeting standards of performance in key aspects of radiological emergency 

response. Regulation may refer to consensus standards for ORO performance endorsed by the 

NRC and FEMA, rather than define the performance targets themselves in regulation. This will 

provide some flexibility.

New regulatory language must also modify the biennial full participation exercise requirement to 

promote more frequent ORO demonstrations. The NRC and FEMA should synchronize licensee 

and ORO required demonstrations to the extent possible. 

9.3.2 44 CFR 350, Review and Approval of State and Local Emergency 

Response Plans and Preparedness

To implement the proposed RIPB RERP construct, FEMA’s regulation on RERP plan reviews 

must change. As with 10 CFR 50.47, language must put the focus on “reasonable assurance 

that adequate protective measures can and will be taken in the event of a radiological 

emergency.” States would not request review and approval of plans, but rather a review of ORO 

performance against standards referenced in the revised 10 CFR 50.47. References to 

NUREG-0654 such as at 44 CFR 350.5(a) would be unnecessary.

Current language on the plan review and approval process would need to reflect the 

performance review process described in Section 6.2. Language may also clarify the scope of 

corrective actions FEMA would propose to improve significantly deficient performance.

9.3.3 44 CFR 351, Radiological Emergency Planning and Preparedness

This regulation defines Federal department and agency roles in assisting State and local 

governments with RERP. Language would require minor modifications to deemphasize support 

for plan reviews and emphasize technical assistance to improve RERP performance.

9.3.4 44 CFR 352, Commercial Nuclear Power Plants: Emergency 

Preparedness Planning

This regulation addresses how reasonable assurance will be given if OROs decline or fail to 

participate in RERP planning and exercises. FEMA may arrange for Federal support to the 

licensee in preparing offsite plans, and may then review the Federal commitments to see if a 

finding of reasonable assurance for the NRC is warranted. 

Since this regulation makes an explicit assumption that OROs will implement at least their allhazards plans in an actual incident, it is not necessarily a high priority under the current 

oversight regimen. 

However, for consistency FEMA should explore adopting a performance focus in this regulation. 

Determinations would be based on whether Federal departments and agencies could, in lieu of 

State and local OROs, meet the identified performance standards.
74

9.3.5 44 CFR 354, Fee for Services

This regulation defines how FEMA assesses fees to NRC licensees. The current biennial 

exercise requirement figures prominently. Evaluation of plans is also mentioned.

New language would need to reflect requirements of the proposed RIPB RERP oversight 

construct. This would include additional exercise requirements. It could include additional 

language on lessons learned and corrective action development, although “technical 

assistance” is already cited as a use of the fees.

9.3.6 NUREG-0654/FEMA-REP-1, Rev. 1 and Guidance Generally

NUREG-0654 provides criteria for plan reviews. Plan reviews may be necessary under the 

proposed oversight regimen, but only if performance is significantly deficient or technical 

assistance is requested (on the assumption that a revised plan would improve performance).

Under the proposed construct, NUREG-0654 and its supplements would not be a more detailed 

set of “input” requirements to meet. As noted above, references to NUREG-0654 in regulations 

like 44 CFR 350 would be removed. Under the proposed performance-based oversight 

construct, very little guidance would be prescriptive. Most publications would instead focus on 

research and lessons learned about what practices and activities (plan provisions, training, etc.) 

have led to improved performance, based on meeting or exceeding objective performance 

targets.

Publications would also include the consensus standards, with performance targets and 

thresholds for different levels of degraded performance.

Beyond the requirements given by the standards, prescriptive guidance would be limited to 

defining valid conditions for demonstrations and associated evaluations, as well as reporting 

requirements. A successor publication to the FEMA Rep Manual could consolidate these 

requirements. 

9.4 NOTIONAL IMPLEMENTATION PROCESS AND MILESTONES

Careful, deliberative rulemaking takes time. Revising its emergency preparedness regulations 

took the NRC more than five years, from presenting a proposed rule to the Commissioners in 

2006 to issuing the final rule in 2011. The NRC and FEMA are collaborating on revisions to 

NUREG-0654 that likely will not be issued until 2017. In the case of the proposed RIPB RERP 

construct, the NRC and FEMA would need to work together to coordinate a package of changes 

to rules and guidance, only some of which were touched upon in Section 9.3.

Further, these are not incremental changes. Stakeholders discussed in Section 4.1 will need 

ample opportunity to comment. All parties wish to avoid unintended negative consequences or 

unreasonable burdens. Given the complexities, the NRC and FEMA will need to pilot the 

concept during rule development to test for issues. Stakeholders will also need ample transition 

time between the issuance and effective date of a final rule (or rules).

Finally, as the concept hinges on definition of performance standards, stakeholder input on the 

concept will require draft or strawman performance standards early in the process.

Table 20 on the following pages provides milestones for a notional implementation process 

based on these considerations. The sequence is more important than the notional timeframes. 
75

The notional timeframes may be optimistic depending on the level of stakeholder concern. One 

draft and round of comments could be eliminated to accelerate the process, if needed.

Table 20. Notional Implementation Process and Milestones for the RIPB RERP 

 Oversight Construct

Milestone Purpose Notional Timeframe 

for Completion

Provide SECY Paper / Decision 

Memorandum (and staff briefing)

Obtain NRC/FEMA leadership consent to 

proceed and form joint project team

2015q1

Issue Notice of Inquiry in Federal 

Register

Outline concept and obtain initial stakeholder 

views (in writing) on issues raised in this paper: 

• Clarity of reasonable assurance

• Benefit vs. cost of performance-based 

approach

• Elements of RERP and their risk 

significance

• Proposed performance indicators and 

target values

• Frequency of demonstration

• Alternative, proxy indicators for RERP 

elements not amenable to demonstration

• Differential oversight and intervention

2015q2

Adjudicate Comments Refine concept and prepare for gateway 

decision

2015q3

Hold Public Meeting Regarding Notice of 

Inquiry and Comments Received

Refine concept and prepare for gateway 

decision

2015q3

Revise Concept at Staff Level into 

Proposed Rule(s) [draft Notice(s) of 

Proposed Rulemaking (NPRM)]

Refine concept and prepare for gateway 

decision

2016q1

Research and Draft Initial Strawman 

Performance Standards (NUREG)

Engage experts (labs or relevant standardsmaking bodies) to develop proposed 

performance targets, threshold levels for lessthan-target performance, and proposed 

demonstration guidance

Obtain key information needed to judge 

cost/benefit of proposed construct

Develop clarifications based on interactions with 

drafters of strawman “exposure draft” standards

2016q3

Draft NPRM Refine rule language 2016q4

Gateway: Submit NPRM to Leadership 

for Approval to Release

Obtain NRC/FEMA leadership commitment to 

proceed

2016q4

Issue NPRM for 60-Day Comment Period Obtain public comment to refine rule language 2017q1

Conduct Supplemental Public Meetings Obtain public comment to refine rule language 2017q1

Draft Pilot Implementation Materials, 

Recruit ~3 Sites/OROs for Pilot, and 

Conduct Training

Prepare to pilot the RIPB RERP oversight 

process as an “alternative approach” for OROs 

at selected sites

2017q2

Conduct Pilot Obtain practical lessons learned from concept 

implementation

2018q2

Evaluate Pilot and Present 

Findings/Lessons at Stakeholder 

Conferences

Present lessons learned as part of outreach on 

concept

Obtain additional feedback on concept

2018q3

Draft Supplemental Notice of Proposed 

Rulemaking (SNPRM)

Refine rule language 2018q4
76

Publish SNPRM for 75-Day Comment 

Period

Obtain public comment to refine rule language 2018q4

Conduct Supplemental Public Meetings Obtain public comment to refine rule language 2019q1

Draft Interim Final Rule Refine rule language based on comments

Inform finalization of performance standards

2019q2

Gateway: Submit Interim Final Rule to 

Leadership for Approval to Release

Afford leadership opportunity to address any 

final concerns 

2019q2

Finalize Standards Prepare for final rule 2019q3

Issue Final Rule Finalize rule language 2019q4

Develop Implementation Guidance and 

Training

Endorse final standards for program

Complete guidance (e.g., revised REP Program 

Manual)

2020q3

Develop Website(s) for Public Reporting 

of Results and Portal for Analysis of 

Lessons Learned

Develop ability to associate RERP oversight 

results with other site information at Reactor 

Oversight Project site

Develop (or leverage existing) portal to compile 

AARs, analyses of AARs, and research into 

evidence supporting corrective actions for 

performance

2020q4

Complete Initial Post-Rule Training and 

Outreach

Complete cycle of outreach to stakeholder 

groups and OROs

Complete training of exercise planners and 

evaluators

2021q4

Make Final Rule Effective Provide an ample transition period for 

States/OROs to adjust

2022q1
77

10. CONCLUSIONS

Since the Three Mile Island accident, oversight of offsite radiological emergency preparedness 

has evolved into a complex oversight regime, with the original 16 planning standards 

supplemented by more than 190 assessment criteria. While these planning standards and the 

oversight system built upon them have served to provide defense-in-depth protection to the 

American public from a radiological disaster for more than 30 years, there is room to reduce the 

oversight burden on OROs by refocusing on what OROs must do, not what they must say in 

their plans. 

This paper has outlined an oversight system for offsite RERP that would be more risk-informed

and performance-based. The system is based not on a list of things to include in plans, but on a 

conceptual model of the basic tasks that must be performed to ensure public health and safety 

in a radiological emergency. Subtasks have objective and outcome-based metrics for gauging 

performance, with clear thresholds for acceptability. Levels of less-than-target performance are 

associated with different levels of oversight, corrective action, and re-demonstration in order to 

allocate resources appropriately against risks.

The proposed offsite RERP oversight system is consistent with proposals for performancebased oversight of licensees. In both, the focus on objective measures of performance rather 

than subjective determinations of compliance should offer more meaningful reasonable 

assurance determinations to the public, and more flexibility for emergency preparedness efforts 

to licensees and OROs. 

Further, by focusing on performance outcomes rather than performance inputs, the proposed 

RIPB RERP oversight construct would enable greater integration of RERP and all-hazards 

initiatives for OROs. Nothing in the oversight construct itself would impede adopting an allhazards format or approach, such as for plans, because oversight would focus on achieving 

outcomes. Also, use of objective performance measures could enable research and studies to 

find correlations between specific all-hazards requirements or approaches and RERP 

performance outcomes.

There are issues. A performance-based oversight system could require more resources than 

under the current system in order to fully reap its benefits.

There are also gaps. How to demonstrate the implementation of protective actions, especially 

evacuation, remains unclear. The minimum frequency of demonstration needed to provide 

reasonable assurance on subtask performance remains more relative than precisely specified. 

Degrees of unacceptable performance have not been defined or validated for all subtasks. 

Resource costs and savings of the proposed approach are not quantified. Additional work needs 

to be done in all these areas. 

However, a conceptual framework now exists in sufficient detail to guide further development of 

an RIPB RERP oversight concept. Ideally, this report has provided enough detail to suggest the 

additional work is worth doing. 

79

11. REFERENCES

11.1 LAW AND REGULATION

Post-Katrina Emergency Management Reform Act, Pub. L. 109-295, 120 Stat. 1394 (2006). 

Available at: http://www.gpo.gov/fdsys/pkg/PLAW-109publ295/pdf/PLAW-109publ295.pdf

(accessed October 27, 2013).

Robert T. Stafford Disaster Relief and Emergency Assistance Act, as amended, 42 U.S.C. § 

5121 et seq. Available at: http://www.fema.gov/media-library-data/1383153669955-

21f970b19e8eaa67087b7da9f4af706e/stafford_act_booklet_042213_508e.pdf (accessed 

October 27, 2013).

U.S. Code of Federal Regulations, “Domestic Licensing of Production and Utilization Facilities,” 

Part 50, Chapter I, Title 10, “Energy.” 

U.S. Code of Federal Regulations, “Emergency Planning and Preparedness for Production and 

Utilization Facilities,” Appendix E to Part 50, Chapter I, Title 10, “Energy.” 

U.S. Code of Federal Regulations, “Review and Approval of State and Local Radiological 

Emergency Response Plans and Preparedness,” Part 350, Chapter I, Title 44, “Emergency 

Management and Assistance.”

U.S. Code of Federal Regulations, “Radiological Emergency Planning and Preparedness,” Part 

351, Chapter I, Title 44, “Emergency Management and Assistance.”

U.S. Code of Federal Regulations, “Commercial Nuclear Power Plants: Emergency 

Preparedness Planning,” Part 352, Chapter I, Title 44, “Emergency Management and 

Assistance.”

U.S. Code of Federal Regulations, “Fee for Services in Support, Review and Approval of State 

and Local Government or Licensee Radiological Emergency Plans and Preparedness,” Part 

353, Chapter I, Title 44, “Emergency Management and Assistance.” Available at: 

http://www.gpo.gov/fdsys/pkg/CFR-2011-title44-vol1/pdf/CFR-2011-title44-vol1-part353.pdf

(accessed October 27, 2013).

U.S. Code of Federal Regulations, “Fee for Services to Support FEMA’s Offsite Radiological 

Emergency Preparedness Program,” Part 354, Chapter I, Title 44, “Emergency Management 

and Assistance.” Available at: http://www.gpo.gov/fdsys/pkg/CFR-2011-title44-vol1/pdf/CFR2011-title44-vol1-part354.pdf (accessed October 27, 2013).

11.2 COMPILATIONS OF PRESIDENTIAL DOCUMENTS

Federation of American Scientists, “Presidential Directives and Executive Orders,” undated. 

Available at: http://www.fas.org/irp/offdocs/direct.htm (accessed October 27, 2013).

National Archives and Records Administration, Office of the Federal Register, “Executive 

Orders,” undated. Available at: https://www.federalregister.gov/executive-orders (accessed 

October 27, 2013). 
80

11.3 NUCLEAR REGULATORY COMMISSION REFERENCES

76 FR 72560, “Enhancements to Emergency Preparedness Regulations; Final Rule,” Federal 

Register, Vol. 76, No. 226, November 23, 2011, pp. 72560-72600. Available at: 

http://www.gpo.gov/fdsys/pkg/FR-2011-11-23/pdf/2011-29735.pdf (accessed October 27, 2013).

ADAMS Accession Number ML080440163, “Elements of a Performance Based Emergency 

Preparedness Regulatory Regimen,” 2008. Available at: 

http://pbadupws.nrc.gov/docs/ML0804/ML080440163.pdf (accessed October 27, 2013).

Bulletin 2005-02, “Emergency Preparedness and Response Actions for Security-Based Events,”

July 18, 2005. Available at: http://www.nrc.gov/reading-rm/doc-collections/gencomm/bulletins/2005/bl200502.pdf (accessed October 27, 2013).

IMC 0305, “Operating Reactor Assessment Program,” June 13, 2012. Available at: 

http://pbadupws.nrc.gov/docs/ML1208/ML12089A066.pdf (accessed October 27, 2013).

IMC 0308, Attachment 3, Appendix B, “Technical Basis for Emergency Preparedness 

Significance Determination Process,” December 19, 2012. Available at: 

http://pbadupws.nrc.gov/docs/ML1228/ML12284A512.pdf (accessed October 27, 2013).

IMC 0609, Appendix B, “Emergency Preparedness Significance Determination Process,”

February 24, 2012. Available at: http://pbadupws.nrc.gov/docs/ML1200/ML120090270.pdf

(accessed October 27, 2013).

NRC 2009, “Backgrounder on Emergency Preparedness at Nuclear Power Plants,” January 

2009. Available at: http://www.nrc.gov/reading-rm/doc-collections/fact-sheets/emerg-plan-prepnuc-power-bg.html (accessed October 27, 2013).

NRC 2012, “Risk Informed and Performance Based Oversight of Radiological Emergency 

Response Programs,” Request for Proposal NRC-HQ-12-R-07-0051, October 1, 2012. Available 

at: 

https://www.fbo.gov/index?s=opportunity&mode=form&id=34908dc0442e2d49a3e13f114ae7d8f

0&tab=core&_cview=1 (accessed October 27, 2013).

NRC 2013, “Detailed Reactor Oversight Process Description,” May 20, 2013. Available at: 

http://www.nrc.gov/reactors/operating/oversight/rop-description.html (accessed October 27, 

2013).

NRC 2013a, “State-of-the-Art Reactor Consequence Analyses (SOARCA),” July 15, 2013. 

Available at: http://www.nrc.gov/about-nrc/regulatory/research/soar.html (accessed October 27, 

2013).

NSIR/DPR-ISG-01, “Emergency Planning for Nuclear Power Plants,” Interim Staff Guidance, 

November 2011.

NUREG-0654/FEMA-REP-1, Rev. 1, “Criteria for Preparation and Evaluation of Radiological 

Emergency Response Plans and Preparedness in Support of Nuclear Power Plants,” November 

1980. Available at: http://pbadupws.nrc.gov/docs/ML0404/ML040420012.pdf (accessed October 

27, 2013).

NUREG-0654/FEMA-REP-1, Rev. 1, Supplement 3, “Criteria for Preparation and Evaluation of 

Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power 
81

Plants: Guidance for Protective Action Strategies,” November 2011. Available at: 

http://pbadupws.nrc.gov/docs/ML1130/ML113010596.pdf (accessed October 27, 2013).

NUREG-0654/FEMA-REP-1, Rev.1, Supplement 4, “Criteria for National Preparedness Initiative 

Integration, Exercise Enhancement, and Backup Alert and Notification Systems,” October 2011. 

Available at: http://www.fema.gov/pdf/about/divisions/thd/FEMA-REP-1%20Rev-1%20Supp4%20Oct%202011.pdf (accessed October 27, 2013).

NUREG/CR-6864, Vol. 1, “Identification and Analysis of Factors Affecting Emergency 

Evacuations,” January 2005. Available at: 

http://pbadupws.nrc.gov/docs/ML0502/ML050250245.pdf (accessed October 27, 2013).

NUREG/CR-6953, Vol. 1, “Review of NUREG-0654, Supplement 3, ‘Criteria for Protective 

Action Recommendations for Severe Accidents,’” December 2007. Available at: 

http://www.nrc.gov/reading-rm/doc-collections/nuregs/contract/cr6953/v1/ (accessed October 

27, 2013).

NUREG/CR-7002, “Criteria for Development of Evacuation Time Estimate Studies,” November 

2011. Available at: http://pbadupws.nrc.gov/docs/ML1130/ML113010515.pdf (accessed October 

27, 2013).

NUREG/CR-7160, “Emergency Preparedness Significance Quantification Process: Proof of 

Concept,” June 2013. Available at: http://pbadupws.nrc.gov/docs/ML1316/ML13164A285.pdf

(accessed October 27, 2013).

SECY-00-0077, “Modifications to the Reactor Safety Goal Policy Statement,” Commission 

Paper, March 30, 2000. Available at: http://pbadupws.nrc.gov/docs/ML0036/ML003694247.html

(accessed October 27, 2013).

SECY-06-0200, “Results of the Review of Emergency Preparedness Regulations and 

Guidance,” Commission Voting Record, January 8, 2007. Available at: 

http://www.nrc.gov/reading-rm/doc-collections/commission/cvr/2006/2006-0200vtr.pdf

(accessed October 27, 2013).

11.4 OTHER REFERENCES

American Highway Users Alliance, “Emergency Evacuation Report Card,” 2006. Available at: 

http://www.highways.org/wp-content/uploads/2006/10/Evacuation_Study-FINAL.pdf (accessed 

October 27, 2013).

Apostolakis, G., “Application of Risk Assessment and Management to Nuclear Safety,” 

September 20, 2012. Available at: http://energy.gov/sites/prod/files/2013/12/f5/Apostolakis.pdf

(accessed October 27, 2013).

Arkansas Department of Emergency Management, “State of Arkansas Multi-Year Training and 

Exercise Plan 2011–2014,” January 20, 2011. Available at: 

http://www.adem.arkansas.gov/ADEM/Divisions/Preparedness/Exercise/Documents/R6-TEPW236_Arkansas_2011-2014_TEP_FINAL.pdf (accessed October 27, 2013).
82

Bear, M.L., Ohio Emergency Management Agency, “Comments on the future revision of 

NUREG-0654,” January 23, 2013. Available at: 

http://www.regulations.gov/#!documentDetail;D=FEMA-2012-0026-0020 (accessed October 27, 

2013). 

California Governor’s Office of Emergency Services, “Threat and Hazard Identification and Risk 

Assessment: Summary of FY 2012 Process and FY 2012 Capability Targets,” undated. 

Available at: http://www.calema.ca.gov/infrastructureprotection/Pages/THIRA.aspx (accessed 

October 27, 2013).

Coates, C. and R. Hines, “Innovations of the MN REP Program,” Presentation to the 2012 

National Radiological Emergency Preparedness Conference, April 2012. Available at: 

http://www.nationalrep.org/2012Presentations/Session%2025_Minnesota%20REP%20Initiative

s_Coates&Hines.pdf (accessed October 27, 2013).

Congressional Research Service, “State Authority to Regulate Nuclear Power: Federal 

Preemption Under the Atomic Energy Act,” R41984, September 6, 2011. Available at: 

https://www.hsdl.org/?view&did=718958 (accessed October 27, 2013). 

Department of Homeland Security, 2008, “National Incident Management System” [Core 

Document], December 2008. Available at: 

http://www.fema.gov/pdf/emergency/nims/NIMS_core.pdf (accessed October 27, 2013).

Department of Homeland Security, 2011, “National Incident Management System: Training 

Program,” September 2011. Available at: 

http://www.fema.gov/pdf/emergency/nims/nims_training_program.pdf (accessed October 27, 

2013).

Department of Homeland Security, 2011a, “National Preparedness Goal,” September 2011. 

Available at: http://www.fema.gov/media-library/assets/documents/25959 (accessed October 

27, 2013).

Department of Homeland Security, 2011b, “The Strategic National Risk Assessment (SNRA),” 

December 2011. Available at: http://www.dhs.gov/strategic-national-risk-assessment-snra

(accessed October 27, 2013).

Department of Homeland Security, 2013, “Homeland Security Exercise and Evaluation Program

(HSEEP)” [revised doctrine], April 2013. Available at: 

https://www.llis.dhs.gov/HSEEP/Documents/homeland-security-exercise-and-evaluationprogram-hseep (accessed October 27, 2013).

Department of Homeland Security, 2013a, “Exercise Evaluation Guide – Response –

Operational Communications,” April 2013. Available at: 

https://www.llis.dhs.gov/HSEEP/Documents/eeg-response-operational-communications

(accessed October 27, 2013).

Department of Homeland Security, 2013b, “Homeland Security Exercise and Evaluation 

Program (HSEEP) 2013,” undated. Available at: https://www.llis.dhs.gov/hseep (accessed 

October 27, 2013).
83

Engelhart, T., Wisconsin Division of Emergency Management, “NUREG-0654/FEMA-REP-1, 

Rev.1, Comments on Scope of Future Revisions to ‘Criteria for Preparation and Evaluation of 

Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power 

Plants,’” January 31, 2013. Available at: http://www.regulations.gov/#!documentDetail;D=FEMA2012-0026-0027 (accessed October 27, 2013).

Environmental Protection Agency, “Manual of Protective Action Guides and Protective Actions 

for Nuclear Incidents,” Guides and Actions,” 1992. Available at: 

http://www.epa.gov/radiation/docs/er/400-r-92-001.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, “Radiological Emergency Preparedness: Exercise 

Evaluation Methodology,” Federal Register, Vol. 66, No. 112, June 11, 2001, pp. 31342-31362.

Available at: http://www.gpo.gov/fdsys/pkg/FR-2001-06-11/pdf/01-14637.pdf (accessed October 

27, 2013).

Federal Emergency Management Agency, “Typed Resource Definitions: Fire and Hazardous 

Materials Resources,” FEMA 508-4, July 2005. Available at: 

http://www.fema.gov/pdf/emergency/nims/fire_haz_mat.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2010, “Integration: HSEEP and REP Exercise 

Programs,” presentation to Pennsylvania Annual Emergency Management Conference, 

September 2010. Available at: 

https://www.portal.state.pa.us/portal/server.pt?open=space&name=Dir&psname=SearchResult

&psid=1&cached=true&in_hi_userid=2&control=OpenSubFolder&subfolderID=158929&DirMode

=1# (accessed October 27, 2013).

Federal Emergency Management Agency, 2010a, “Developing and Maintaining Emergency 

Operations Plans,” CPG 101, Version 2.0, November 2010. Available at: 

http://www.fema.gov/media-library-data/20130726-1828-25045-

0014/cpg_101_comprehensive_preparedness_guide_developing_and_maintaining_emergency

_operations_plans_2010.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2011, “All-hazards Response and Preparedness 

Assessment Tool,” April 2011. Available at: http://www.fema.gov/medialibrary/assets/documents/21635 (accessed October 27, 2013).

Federal Emergency Management Agency, 2011a, “REP Program Manual and Supplement 4 

Comment Adjudication Report – Organized by Topic,” October 2011. Available at: 

http://www.fema.gov/pdf/about/divisions/thd/RPM-Supp4_Comments_by_Topic.pdf (accessed 

October 27, 2013).

Federal Emergency Management Agency, 2011b, “National Preparedness System,” November 

2011. Available at: http://www.fema.gov/media-library-data/20130726-1855-25045-

8110/national_preparedness_system_final.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2012, “Radiological Emergency Preparedness: 

Fiscal Year 2013 Congressional Justification,” October 2012. Available at: 

http://www.fema.gov/pdf/about/budget/11c_fema_radiological_emergency_preparedness_progr

am_dhs_fy13_cj.pdf (accessed November 11, 2013).

Federal Emergency Management Agency, 2012a, “Notice of Request for Comments on the 

Scope of Future Revisions to ‘Criteria for Preparation and Evaluation of Radiological Emergency 

Response Plans and Preparedness in Support of Nuclear Power Plants’ (NUREG-0654/FEMA-
84

REP-1, Rev.1),” Federal Register, Vol. 77, No. 210, October 30, 2012, pp. 65700-65701.

Available at: http://www.gpo.gov/fdsys/pkg/FR-2012-10-30/pdf/2012-26578.pdf (accessed 

October 27, 2013).

Federal Emergency Management Agency, 2012b, “About PS-PREPTM,” December 7, 2012. 

Available at: http://www.fema.gov/about-ps-preptm (accessed October 27, 2013). 

Federal Emergency Management Agency, 2013, “Boston Marathon Bombings: The Positive 

Effect of Planning and Preparation on Response,” Lesson Learned, 2013. Available at:

https://www.llis.dhs.gov/sites/default/files/Boston%20Marathon%20Bombings%20Positive%20E

ffects%20of%20Preparedness_0.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2013a, “FY 2013 Emergency Management 

Performance Grants (EMPG) Program Funding Opportunity Announcement (FOA),” 2013.

Available at: http://www.fema.gov/media-library-data/20130726-1916-25045-

8793/fy13_empg_foa_final.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2013b, “FY 2013 Homeland Security Grant Program 

(HSGP) Funding Opportunity Announcement (FOA),” 2013. Available at: 

http://www.fema.gov/media-library-data/20130726-1916-25045-6176/fy_2013_hsgp_foa.pdf

(accessed October 27, 2013).

Federal Emergency Management Agency, 2013c, “Technical Assistance Catalog: Preparedness 

& Program Management Technical Assistance,” 2013. Available at: http://www.fema.gov/medialibrary-data/20130726-1828-25045-9269/technical_assistance_catalog.pdf (accessed October 

27, 2013).

Federal Emergency Management Agency, 2013d, “National Preparedness Report,” March 30,

2013. Available at: http://www.fema.gov/media-library-data/20130726-1916-25045-

3721/npr2013_final.pdf (accessed October 27, 2013).

Federal Emergency Management Agency, 2013e, “National Exercise Program,” May 22, 2013. 

Available at: http://www.fema.gov/national-exercise-program (accessed October 27, 2013).

Federal Emergency Management Agency, 2013f, “Radiological Emergency Preparedness 

Program Manual,” June 2013. Available at: http://www.fema.gov/medialibrary/assets/documents/32780 (accessed October 27, 2013).

Federal Emergency Management Agency, 2013g, “National Incident Management System,” 

June 4, 2013. Available at: http://www.fema.gov/national-incident-management-system

(accessed October 27, 2013).

Federal Emergency Management Agency, 2013h, “Threat and Hazard Identification and Risk 

Assessment Guide,” CPG 201, Second Edition, August 2013.

Available at: http://www.fema.gov/media-librarydata/8ca0a9e54dc8b037a55b402b2a269e94/CPG201_htirag_2nd_edition.pdf (accessed 

October 27, 2013).

Federal Emergency Management Agency, 2013i, “National Integration Center Resource 

Management,” September 5, 2013. Available at: http://www.fema.gov/resource-management

(accessed October 27, 2013). 
85

Federal Emergency Management Agency, 2013j, “Hazus: The Federal Emergency 

Management Agency’s (FEMA’s) Methodology for Estimating Potential Losses from Disasters,” 

October 23, 2013. Available at: http://www.fema.gov/hazus (accessed October 27, 2013).

Federal Emergency Management Agency, “ICS for Single Resources and Initial Action 

Incidents,” Independent Study Course IS-200.b, undated. Available at: 

http://emilms.fema.gov/IS200b/ICS01summary.htm (accessed October 27, 2013). 

Federal Highway Administration, “Travel Time Reliability,” undated. Available at: 

http://ops.fhwa.dot.gov/publications/tt_reliability/brochure/ttr_brochure.pdf (accessed October 

27, 2013).

Florida Division of Emergency Management, “Multi-Year Training and Exercise Plan 2013-

2015,” undated. Available at: http://www.tbrpc.org/rdstf/pdfs/mytep/FL_MYTEP_2013_15.pdf

(accessed October 27, 2013).

Government Accountability Office, “Nuclear Power: Analysis of Regional Differences and 

Improved Access to Information Could Strengthen NRC Oversight,” GAO-13-743, September 

2013. Available at: http://www.gao.gov/products/gao-13-743 (accessed October 27, 2013).

Illinois Office of Management and Budget, “Fiscal Year 2014 Enacted Total Budget by Line 

Item,” 2013. Available at: 

http://www2.illinois.gov/gov/budget/Documents/Budget%20Book/FY%202014/FY14EnactedTota

lBudgetbyLineItem.xls (accessed October 27, 2013).

Klinger, J.G., Illinois Emergency Management Agency, “Comments on NUREG-0654/FEMAREP-1, Rev.1, Comments on Scope of Future Revision,” January 3, 2013. Available at: 

http://www.regulations.gov/#!documentDetail;D=FEMA-2012-0026-0015 (accessed October 27, 

2013).

Louisiana, “Multiyear Training and Exercise Plan 2012-2013,” undated. Available at: 

http://gohsep.la.gov/exercise/LAMYTEP2012_2013FINAL.pdf (accessed October 27, 2013).

Michigan, “State of Michigan/UASI Training and Exercise Plan 2014-2016,” undated. Available 

at: http://www.michigan.gov/documents/msp/MI_2008-2010_Plan_final_228774_7.pdf

(accessed October 27, 2013).

Mills et al., “Study of Evacuation Times Based on General Accident History,” SAND94-2714, 

1995. Available at: 

https://web.archive.org/web/20130221145017/https://radtran.sandia.gov/docs/SAND94-

2714.pdf (accessed October 27, 2013). 

Minnesota Division of Emergency Management [now Homeland Security and Emergency 

Management], “State of Minnesota Emergency Operations Plan,” December 31, 2001. Available 

at: http://transitsafety.volpe.dot.gov/training/Archived/EPSSeminarReg/CD/documents/OHIO_DOT/MinnesotaE

OP.doc (accessed October 27, 2013).

Minnesota Homeland Security and Emergency Management, “2013-2015 State of Minnesota 

Multiyear Training and Exercise Plan (TEP),” undated. Available at: 

https://dps.mn.gov/divisions/hsem/training/Documents/MN-TEP-2013-2015-final.pdf (accessed 

October 27, 2013).
86

Missouri Office of Homeland Security, “2012-2014 Multiyear Training and Exercise Plan,” 

October 20, 2010. Available at: 

http://www.dps.mo.gov/dir/programs/ohs/documents/resources/2012-2014exerciseplan.pdf

(accessed October 27, 2013).

Missouri Office of Homeland Security, “2012 Threat and Hazard Identification and Risk 

Assessment (THIRA),” December 2012. Available at: 

http://www.dps.mo.gov/dir/programs/ohs/documents/2012-mo-THIRA.pdf (accessed October 

27, 2013). 

Mulligan, P.A., New Jersey Department of Environmental Protection, “Comments on NUREG0654/FEMA-REP-1, Rev.1, Comments on Scope of Future Revision,” January 30, 2013.

Available at: http://www.regulations.gov/#!docketBrowser;rpp=25;po=0;dct=PS;D=FEMA-2012-

0026 (accessed October 27, 2013).

National Academy of Public Administration, “Improving the National Preparedness System: 

Developing More Meaningful Grant Performance Measures,” October 2011, p. viii. Available at: 

http://www.napawash.org/wp-content/uploads/2012/06/11-07.pdf (accessed October 27, 2013).

National Renewable Energy Laboratory, “Wind Maps,” undated. Available at: 

http://www.nrel.gov/gis/wind.html (accessed October 27, 2013).

Nebraska, “Nebraska Multi-Year Training and Exercise Plan 2012--2015,” November 2012. 

Available at: http://www.nema.ne.gov/pdf/training/state-training-calendar.pdf (accessed October 

27, 2013).

Nelson, A. P., Nuclear Energy Institute, letter to C.G. Miller, U.S. Nuclear Regulatory 

Commission, May 9, 2008. Available at: 

http://pbadupws.nrc.gov/docs/ML0813/ML081340523.pdf (accessed October 27, 2013).

New Hampshire Department of Administrative Services, “Governor’s Operating Budget for 

Fiscal Years ending June 30, 2014-2015,” February 14, 2013. Available at: 

http://admin.state.nh.us/budget/Budget2014-2015/GovernorsBudgetBill.pdf#02-23 (accessed 

October 27, 2013).

New Hampshire Homeland Security and Emergency Management, “State of New Hampshire 

Threat and Hazard Identification and Risk Assessment,” September 2012. Available at: 

http://www.nh.gov/safety/divisions/hsem/HazardMitigation/documents/2013-appendix-f.pdf

(accessed October 27, 2013).

Nuclear Energy Institute, 2009, “Regulatory Assessment Performance Indicator Guideline,” NEI 

99-02 Revision 6, October 2009. Available at: 

http://pbadupws.nrc.gov/docs/ML0929/ML092931123.pdf (accessed October 27, 2013).

Nuclear Energy Institute, 2010, “Conducting a Hostile Action-Based Emergency Response Drill,” 

NEI 06-04, Revision 2, April 2010. Available at: 

http://pbadupws.nrc.gov/docs/ML1011/ML101180293.pdf (accessed October 27, 2013).

Nuclear Energy Institute, 2012, “Perspective on Public Opinion,” November 2012. Available at: 

http://www.nei.org/CorporateSite/media/filefolder/POPO_Nov2012_FINAL.pdf?ext=.pdf

(accessed October 27, 2013). 
87

Ohio Office of Budget and Management, “Budget Recommendations: The State of Ohio 

Executive Budget Fiscal Years 2014-2015,” February 4, 2013. Available at: 

http://media.obm.ohio.gov/OBM/Budget/Documents/operating/fy-14-

15/bluebook/budget/Section-D_DPS.pdf (accessed October 27, 2013). 

Pennsylvania Code, “Radiation Emergency Response Fund,” Chapter 116. Available at: 

http://www.pacode.com/secure/data/004/chapter116/chap116toc.html (accessed October 27, 

2013).

Pennsylvania Department of Environmental Protection, “Report to the General Assembly 

Pursuant to Act 31 of 2007,” August 2010. Available at: 

http://www.elibrary.dep.state.pa.us/dsweb/Get/Document-81332/2900-BK-DEP4290.pdf

(accessed October 27, 2013). 

Pennsylvania Office of the Budget, “2013-2014 Governor’s Executive Budget,” February 5, 

2013. Available at: http://www.portal.state.pa.us/portal/server.pt/document/1320332/2013-

14_governors_executive_budget_cd_pdf (accessed October 27, 2013).

Perkins-Grew, S., Nuclear Energy Institute, “Comments on Scope of Future Revisions to 

‘Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and 

Preparedness in Support of Nuclear Power Plants’ (NUREG-0654/FEMA-REP-1, Rev. 1),” 

Attachment 2, January 23, 2013. Available at: 

http://www.regulations.gov/#!documentDetail;D=FEMA-2012-0026-0016 (accessed October 27, 

2013).

Sorenson, J. and Barbara Vogt, “Interactive Emergency Evacuation Guidebook,” February 

2006. Available at: 

https://web.archive.org/web/20130525085736/http://orise.orau.gov/csepp/documents/planning/e

vacuation-guidebook/index.htm (accessed October 27, 2013).

South Carolina, “2013 Multi-Year Training and Exercise Plan,” December 31, 2012. Available at: 

http://www.scemd.org/files/Training/2013%20%20SC%20MultiYear%20Training%20and%20Exercise%20Plan%20Final.pdf (accessed October 27, 2013).

Spence, R., “REP Integration with HSEEP and PPD-8 Compliance,” Presentation to the 2013 

National REP Conference, April 11, 2013. Available at: 

http://www.nationalrep.org/2013Presentations/Open%20Forum_SLIDES_FEMA%20RIV%20RE

P%20Integration%20with%20HSEEP%20and%20PPD%208%20Compliance_Spence.pdf

(accessed October 27, 2013).

Texas A&M Transportation Institute, “2012 Annual Urban Mobility Report,” February 2013.

Available at: http://mobility.tamu.edu/ums/ (accessed October 27, 2013).

Vermont Homeland Security Unit, 2010, “Multiyear Training and Exercise Plan,” February 11, 

2010. Available at: http://hsu.vermont.gov/sites/vhs/files/Multiyear_Plan_FINAL_021110.pdf

(accessed October 27, 2013). 

Vermont, 2012, “Multi-Year Training and Exercise Plan,” December 20, 2012. Available at: 

http://vem.vermont.gov/sites/vem/files/Training_and_Exercise_Plan_FINAL%20011613.pdf

(accessed October 27, 2013).
88

Vermont Department of Finance and Management, “Fiscal Year 2014 Executive Budget 

Recommendations,” January 24, 2013. Available at: 

http://finance.vermont.gov/sites/finance/files/pdf/Vantage/FY%202014%20Executive%20Budget

%20Recommendations.pdf (accessed October 27, 2013).

Washington Military Department, Emergency Management Division, “State of Washington 2013-

2015 Training and Exercise Plan,” August 30, 2012. Available at: 

http://www.emd.wa.gov/plans/documents/TEPW2013_2015_Final.pdf (accessed October 27, 

2013).

Wisconsin Emergency Management, “Multi-Year Training and Exercise Plan 2013-2015,” March 

5, 2013. Available at: 

http://emergencymanagement.wi.gov/training/docs/3_Year_TE_Plan_Final_2013-2015.pdf

(accessed October 27, 2013).

Yale University School of Forestry and Environmental Studies, “Nuclear Power in the American 

Mind,” March 11, 2012. Available at: http://environment.yale.edu/climate/the-climatenote/nuclear-power-in-the-american-mind/ (accessed October 27, 2013). 
A-1

APPENDIX A: SUBTASK CHECKLISTS

Each subtask on the following pages is organized in a similar manner to NEI 99-02 (Nuclear 

Energy Institute 2009), which discusses onsite facility preparation and response. For each 

indicator, an attempt is made to provide an indicator name, a definition of the indicator, 

measurement criteria for each subtask (including data reporting elements and clarifying notes), 

and an initial effort at calculating the subtask. 

Also included is a proposed method and frequency of demonstration. 

Unlike NEI 99-02, each measure (except for binary process measures) has values linked to 

evaluation categories. For example, in a time-based indicator, the ideal or target time would be 

labeled “green,” an unacceptable time that is expected to have small impact on public health 

and safety would be labeled “white,” an unacceptable time that is expected to have a moderate 

to major impact on public health and safety would be labeled “yellow.” 

Determining thresholds for assessments for each subtask requires more literature review, 

expert consultation, and analysis. Sample threshold levels are provided for some indicators. For 

some time-based metrics, the target value is given only as a letter value (e.g., “x”) indicating the 

need for additional analysis to determine a threshold value. However, the level of detail in this 

framework should serve to guide fuller development. 
A-2

Task 1: Receive Licensee Notification of Emergency

Subtask: Primary communications

Description: The primary means of communication from licensee to ORO is functional. 

Measurement: Primary communications system should be tested multiple times throughout 

the period of performance, to test that the system is functional. “Functional” means that a 

connection is made. Must demonstrate positive performance at least 90% (tentative) of the 

time to yield a positive finding. Tests should be conducted at least monthly to yield accurate 

findings. 

Calculation: 

Green: (# of successful tests)/(total # of tests) >/= 0.90

White: (# of successful tests)/(total # of tests) >/= w

Yellow: (# of successful tests)/(total # of tests) >/= y

Orange: (# of successful tests)/(total # of tests) < y

Demonstration Type: Communications test

Demonstration Frequency: Monthly

Subtask: Secondary communications

Description: Licensee and ORO have a functional, secondary means of communication.

Measurement: Secondary communications system should be tested multiple times 

throughout the period of performance, to test that system is functional. “Functional” means 

that a connection is made. Must demonstrate positive performance at least 90% (tentative) of 

the time to yield a positive assessment. Tests should be conducted at least monthly to yield 

accurate assessments. 

Calculation:

Green: (# of successful tests)/(total # of tests) >/= 0.90

White: (# of successful tests)/(total # of tests) >/= w

Yellow: (# of successful tests)/(total # of tests) >/= y

Orange: (# of successful tests)/(total # of tests) < y

Demonstration Type: Communications test

Demonstration Frequency: Monthly

Subtask: 24-Hour Warning Point

Description: ORO’s primary communications center is staffed at all times to receive a 

message.
A-3

Measurement: Call or otherwise contact ORO’s primary communications center at random 

times throughout the period of performance, to test that someone is available to answer the 

phone or secondary means of communication. Call must include notification that this is a test 

or drill. Must answer phone/device at least 90% (tentative) of the time to yield a positive

finding. Calls should be conducted on a monthly (or more frequent) basis, at all times of day 

and night. Test may be combined with tests of primary and secondary means of 

communication.

Calculation: 

Green: (# successfully answered calls)/(total # of calls) >/= 0.90

White: (# successfully answered calls)/(total # of calls) >/= w

Yellow: (# successfully answered calls)/(total # of calls) >/= y

Orange: (# successfully answered calls)/(total # of calls) < y

Demonstration Type: Communications test

Demonstration Frequency: Monthly

Subtask: Alternate Communications Center

Description: ORO can continue to receive notifications in the event of a primary facility being 

rendered unusable.

Measurement: At least quadrennially, as a condition of play, render ORO primary 

communications/watch center unusable. ORO must be able to identify and set up an alternate 

location for continuity of operations, and maintain required communications from this 

secondary facility. 

Calculation: This indicator requires two capability demonstrations. 

Green: Is ORO able to identify an alternate facility? Y or N. Is ORO able to establish 

operations at alternate facility? Y or N. Must achieve yes to both.

White: Is ORO able to identify an alternate facility? Y or N. Is ORO able to establish 

operations at alternate facility? Y or N. Must achieve yes to both, but may have trouble with 

one.

Yellow: Is ORO able to identify an alternate facility? Y or N. Is ORO able to establish 

operations at alternate facility? Y or N. Only achieves 1 demonstration of capability.

Orange: ORO cannot demonstrate either capability.

Demonstration Type: Exercise

Demonstration Frequency: Quadrennially 
A-4

Task 2: Understand the Notification (Emergency Classification Levels)

Subtask: Message comprehension

Description: Staff members at communications/watch center who receive a message are 

able to comprehend emergency classification levels/severity of emergency. 

Measurement: Done in conjunction with ability to receive a notification. Various levels of 

emergency are presented to the respondent. Respondent is then queried about the meaning 

of the classification level, and the actions that would be necessary on his or her part. Should 

demonstrate understanding of emergency response levels at least 90% (tentative) of the time. 

Calculation:

Green: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= 0.90

White: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= w

Yellow: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= y

Orange: (# of times able to explain response levels)/(total # of queries about response 

levels)< y

Demonstration Type: Drill (may be combined with communications test)

Demonstration Frequency: Monthly

Subtask: Secondary official message comprehension

Description: Other appropriate officials who receive the message from original offsite 

message recipient are able to comprehend emergency classification levels/severity of 

emergency.

Measurement: On at least a quarterly basis, secondary staff members at 

communications/watch center are questioned about the meaning of the classification levels, 

and the response actions that would be necessary on their part. Should demonstrate 

understanding of emergency response levels at least 75% (tentative) of the time. 

Calculation:

Green: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= 0.75

White: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= w

Yellow: (# of times able to explain response levels)/(total # of queries about response levels) 

>/= y

Orange: (# of times able to explain response levels)/(total # of queries about response levels) 

< y
A-5

Demonstration Type: Drill (may be combined with communications test)

Demonstration Frequency: Monthly
A-6

Task 3: Notify the Appropriate Officials 

Subtask: Timely notification 

Description: Offsite message recipient at communications center is able to contact 

appropriate officials/decision makers within reasonable amount of time following receipt of 

notification of emergency.

Measurement: During exercise play, communications center is able to contact appropriate 

officials in the chain of command within 15 minutes (tentative) of notification of an emergency. 

Calculation:

Green: (recorded time that message recipient successfully contacts officials)/(recorded time 

that message was received) </= 15 minutes

White: (recorded time that message recipient successfully contacts officials)/(recorded time 

that message was received) </= w

Yellow: (recorded time that message recipient successfully contacts officials)/(recorded time 

that message was received) </= y

Orange: (recorded time that message recipient successfully contacts officials)/(recorded time 

that message was received) > y

Demonstration Type: Exercise

Demonstration Frequency: Monthly

Subtask: Functional communications

Description: Functional primary and backup systems are in place to disseminate message to 

ORO primary personnel and other response organizations.

Measurement: On at least a quarterly basis, both primary and functional communications 

systems must be tested for ability to connect with officials/decision makers at least 75% 

(tentative) of the time. This could serve as both a test of systems, as well as a test of ability to 

actually reach decision-makers, or their backups in chains of command.

Calculation: 

Green: (# of successfully answered communications transmissions)/(total number of 

attempted communications) >/= 0.75

White: (# of successfully answered communications transmissions)/(total number of 

attempted communications) >/= w

Yellow: (# of successfully answered communications transmissions)/(total number of 

attempted communications) >/= y

Orange: (# of successfully answered communications transmissions)/(total number of 

attempted communications) < y
A-7

Demonstration Type: Communications test

Demonstration Frequency: Monthly
A-8

Task 4: Assess the Situation

Subtask: Radiological expertise

Description: A team of decision-makers and radiological technical advisors successfully 

interprets information related to a radiological event or release.

Measurement: State or local EOC personnel responsible for producing assessments 

accurately interpret information provided regarding the emergency at least 90% of the time 

(tentative). Information may include figures about the severity of an emergency, radioactive 

dosage and PAGs in the event of a release, plume projections given weather conditions, and 

other information related to the preservation of public safety in a radiological emergency. To 

ensure that the interpretation is accurate, playing a pre-developed scenario will ensure that 

“correct” answers are available to grade the technical advisor’s performance against. 

Calculation: 

Green: (# of times radiological technical advisor/decision-maker is able to correctly interpret 

information)/(# of times radiological technical advisor receives information) >/= 0.90

White: (# of times radiological technical advisor/decision-maker is able to correctly interpret 

information)/(# of times radiological technical advisor receives information) >/= w

Yellow: (# of times radiological technical advisor/decision-maker is able to correctly interpret 

information)/(# of times radiological technical advisor receives information) >/= y

Orange: (# of times radiological technical advisor/decision-maker is able to correctly interpret 

information)/(# of times radiological technical advisor receives information) < y

Demonstration Type: Exercise

Demonstration Frequency: Biannually (but multiple demonstration points within each 

exercise)

Subtask: Contact licensee

Description: ORO can demonstrate ability to contact nuclear plant for questions pertaining to 

a developing situation.

Measurement: ORO can demonstrate use of communication systems to communicate with 

nuclear plant in real time to gain information within 15 minutes of requiring information about 

a situation x% of the time.

Calculation:

Green: (# of times ORO is able to successfully communicate with nuclear plant within 15 

minutes)/(# of times ORO attempts to communicate with nuclear plant) >/= g

White: (# of times ORO is able to successfully communicate with nuclear plant within 15 

minutes)/(# of times ORO attempts to communicate with nuclear plant) >/= w

Yellow: (# of times ORO is able to successfully communicate with nuclear plant within 15 

minutes)/(# of times ORO attempts to communicate with nuclear plant) >/= y
A-9

Orange: (# of times ORO is able to successfully communicate with nuclear plant within 15 

minutes)/(# of times ORO attempts to communicate with nuclear plant < y

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Monitor radioactivity

Description: ORO can demonstrate capability to monitor the environment outside the facility 

boundary for increased levels of radioactivity.

Measurement: ORO must be able to activate and provide field monitoring teams at random 

times over a given period of time. To measure this, a series of unannounced tests must be 

performed, in which the ORO is required to provide at least 2 offsite monitoring teams with 

capability, equipment, and knowledge to accomplish a reading. During exercise, the ORO 

should also be required to detect elevated levels of radioiodine concentrations in the air to an 

amount as low as 10^-7 μCi/cc within x hours of notification, even with interference from noble 

gases and background radiation.

Calculation: Two calculations must be performed for this measure. 

Green: First: (# of unannounced tests in which ORO is able to successfully perform an offsite 

monitoring test within g hour of notification)/(total # of unannounced offsite monitoring tests) 

>/= 0.90. Second: ORO is able to detect elevated levels of radioiodine concentration as low 

as 10^-7 μCi/cc in a test within g hour of notification. ORO meets both elements.

White: First: (# of unannounced tests in which ORO is able to successfully perform an offsite 

monitoring test within g hours of notification)/(total # of unannounced offsite monitoring tests) 

>/= 0.90. Second: ORO is able to detect elevated levels of radioiodine concentration as low 

as 10^-7 μCi/cc in a test within g hour of notification. ORO meets both elements, but may 

have difficulties.

Yellow: First: (# of unannounced tests in which ORO is able to successfully perform an offsite 

monitoring test within g hours of notification)/(total # of unannounced offsite monitoring tests) 

>/= 0.90. Second: ORO is able to detect elevated levels of radioiodine concentration as low 

as 10^-7 μCi/cc in a test within g hour of notification. ORO meets only one element.

Orange: ORO is unable to meet either element.

Demonstration Type: Drill

Demonstration Frequency: Quarterly 

Subtask: Weather evaluation

Description: ORO can demonstrate ability to evaluate current and future real-world weather 

conditions that may have a direct impact on the protective actions necessary to protect the 

public.

Measurement: ORO has direct access to a weather authority (such as the National Weather 

Service) or meteorologist who is able to provide guidance on real-world weather and its effect 

on a nuclear emergency. Random tests should be performed over a given period of time, and 

should assess whether the ORO can contact the weather service in a timely manner, and 
A-10

whether the meteorologist can produce accurate implications of forecast weather in a timely 

manner.

Calculation: 2 calculations must be made to capture this subtask. 

Green: First: (# of times ORO successfully contacts weather evaluator within x minutes)/(total 

# of times ORO attempts to contact weather authority) >/= g. Second: Time it takes for 

weather service to produce accurate implications of forecast weather </= G minutes. 

White: First: (# of times ORO successfully contacts weather evaluator within x minutes)/(total 

# of times ORO attempts to contact weather authority) >/= w. Second: Time it takes for 

weather service to produce accurate implications of forecast weather </= W minutes.

Yellow: First: (# of times ORO successfully contacts weather evaluator within x minutes)/(total 

# of times ORO attempts to contact weather authority) >/= y. Second: Time it takes for 

weather service to produce accurate implications of forecast weather </= Y minutes.

Orange: First: (# of times ORO successfully contacts weather evaluator within x 

minutes)/(total # of times ORO attempts to contact weather authority) < y. Second: Time it 

takes for weather service to produce accurate implications of forecast weather > Y minutes.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Plume mapping

Description: ORO can demonstrate the ability to develop detailed plume maps of the area of 

likely impact.

Measurement: ORO uses available resources, samples, and information to produce an 

accurate map of projected plume, given current or provided release and wind conditions, x% 

of the time. 

Calculation:

Green: (# of times ORO produces accurate plume map from current or provided weather 

conditions)/(# of times ORO is asked to produce plume map) >/= g

White: (# of times ORO produces accurate plume map from current or provided weather 

conditions)/(# of times ORO is asked to produce plume map) >/= w

Yellow: (# of times ORO produces accurate plume map from current or provided weather 

conditions)/(# of times ORO is asked to produce plume map) >/= y

Orange: (# of times ORO produces accurate plume map from current or provided weather 

conditions)/(# of times ORO is asked to produce plume map) < y

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Risk mapping (dose projection)
A-11

Description: ORO can demonstrate the ability to identify people at risk in a radiological 

event.

Measurement: ORO develops accurate projections of integrated dose, including gross 

radioactivity measurements from contamination data (water and air) and produces 

assessments for population in 10-mile EPZ within x hours of notification of protective action 

recommendation, y% of the time.

Calculation:

Green: (# of times ORO accurately produces an integrated dose assessment within x hours of 

protective action recommendation from plant)/(# of times ORO attempts to produce an 

integrated dose assessment) >/= g

White: (# of times ORO accurately produces an integrated dose assessment within x hours of 

protective action recommendation from plant)/(# of times ORO attempts to produce an 

integrated dose assessment) >/= w

Yellow: (# of times ORO accurately produces an integrated dose assessment within x hours 

of protective action recommendation from plant)/(# of times ORO attempts to produce an 

integrated dose assessment) >/= y

Orange: (# of times ORO accurately produces an integrated dose assessment within x hours 

of protective action recommendation from plant)/(# of times ORO attempts to produce an 

integrated dose assessment) < y

Demonstration Type: Exercise

Demonstration Frequency: Biannually
A-12

Task 5: Make Protective Action Decision

Subtask: Consult with experts

Description: ORO can demonstrate ability to consult with technical advisors for advice on 

maximum protection of public health and safety.

Measurement: ORO consults with technical advisor to review protective action 

recommendations provided by licensee, and determine if recommendations are adequate. 

When deemed necessary, the technical advisor provides additional recommendations for 

protective action. 

Calculation:

Green: Does ORO consult with technical advisor? Y or N. Does technical advisor review 

recommendations? Y or N. Does technical advisor provide additional recommendation for 

protective action if he/she deems necessary? Y or N. Must answer Y to 3/3 questions.

White: Does ORO consult with technical advisor? Y or N. Does technical advisor review 

recommendations? Y or N. Does technical advisor provide additional recommendation for 

protective action if he/she deems necessary? Y or N. Must answer Y to 2/3 questions.

Yellow: Does ORO consult with technical advisor? Y or N. Does technical advisor review 

recommendations? Y or N. Does technical advisor provide additional recommendation for 

protective action if he/she deems necessary? Y or N. Answers Y to 1/3 or 0/3 questions.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Make protective action decision

Description: ORO leaders review recommendations from technical advisor and licensee and 

make a PAD for public health and safety.

Measurement: Do decision-makers in the ORO reach a PAD in exercise play within x 

minutes of receiving protective action recommendation from nuclear licensee?

Calculation: 

Green: Decision-makers in the ORO reach a PAD within g minutes of receiving protective 

action recommendation from nuclear licensee? 

White: Decision-makers in the ORO reach a PAD within w minutes of receiving protective 

action recommendation from nuclear licensee?

Yellow: Decision-makers in the ORO reach a PAD within y minutes of receiving protective 

action recommendation from nuclear licensee?

Orange: ORO is unable to provide within y minutes.

Demonstration Type: Exercise
A-13

Demonstration Frequency: Biannually

Subtask: Evacuation mapping

Description: ORO can identify on a map or mapping system zones for evacuation.

Measurement: ORO must develop or provide a map identifying evacuation zones in the 10-

mile EPZ.

Calculation: 

Green: Can ORO provide an accurate map of evacuation zones in the 10-mile EPZ within g 

hours of receiving protective action recommendations from licensee?

White: Can ORO provide an accurate map of evacuation zones in the 10-mile EPZ within w 

hours of receiving protective action recommendations from licensee?

Yellow: Can ORO provide an accurate map of evacuation zones in the 10-mile EPZ within y 

hours of receiving protective action recommendations from licensee?

Orange: ORO is unable to provide within y hours.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Shelter-in-place mapping

Description: ORO is able to identify on a map or mapping system zones for shelter-in-place.

Measurement: ORO must develop or provide a map identifying zones for shelter-in-place in 

the 10-mile EPZ.

Calculation: 

Green: Can ORO provide an accurate map of zones for shelter-in-place in the 10-mile EPZ 

within g hours of receiving protective action recommendations from licensee?

White: Can ORO provide an accurate map of zones for shelter-in-place in the 10-mile EPZ 

within w hours of receiving protective action recommendations from licensee?

Yellow: Can ORO provide an accurate map of zones for shelter-in-place in the 10-mile EPZ 

within y hours of receiving protective action recommendations from licensee?

Orange: ORO is unable to provide within y hours.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: KI mapping

Description: ORO is able to identify on a map or mapping system zones for administration of 

KI.
A-14

Measurement: ORO must develop or provide a map identifying zones for administration of KI 

within the 10-mile EPZ, and within the 50-mile EPZ as needed.

Calculation: 

Green: Can ORO provide an accurate map of zones for administration of KI in the 10-mile 

EPZ/50-mile EPZ within g hours of receiving protective action recommendations from 

licensee?

White: Can ORO provide an accurate map of zones for administration of KI in the 10-mile 

EPZ/50-mile EPZ within w hours of receiving protective action recommendations from 

licensee?

Yellow: Can ORO provide an accurate map of zones for administration of KI in the 10-mile 

EPZ/50-mile EPZ within y hours of receiving protective action recommendations from 

licensee?

Orange: ORO is unable to provide within y hours.

Demonstration Type: Exercise

Demonstration Frequency: Biannually
A-15

Task 6: Mobilize Response Components

Subtask: Contact emergency responders

Description: ORO is able to establish contact with emergency responders working in or near 

the 10-mile EPZ (police, EMTs, firefighters) in a timely manner.

Measurement: ORO is able to contact police, EMTs, firefighters, and other first responders 

working in or near the 10-mile EPZ within 15 minutes (tentative) of making a PAD.

Calculation: Requires three demonstrations of capability.

Green: ORO able to contact police within 15 minutes of making a PAD? Y or N. ORO able to 

contact EMTs within 15 minutes of making a PAD? Y or N. ORO able to contact firefighters 

within 15 minutes of making a PAD? Y or N. ORO able to contact other (if needed) within 15 

minutes of making a PAD? Y or N. Must demonstrate all.

White: ORO able to contact police within 15 minutes of making a PAD? Y or N. ORO able to 

contact EMTs within 15 minutes of making a PAD? Y or N. ORO able to contact firefighters 

within 15 minutes of making a PAD? Y or N. ORO able to contact other (if needed) within 15 

minutes of making a PAD? Y or N. Must demonstrate at least 2/3. 

Yellow: ORO able to contact police within 15 minutes of making a PAD? Y or N. ORO able to 

contact EMTs within 15 minutes of making a PAD? Y or N. ORO able to contact firefighters 

within 15 minutes of making a PAD? Y or N. ORO able to contact other (if needed) within 15 

minutes of making a PAD? Y or N. Must demonstrate 1/3.

Orange: ORO able to contact police within 15 minutes of making a PAD? Y or N. ORO able to 

contact EMTs within 15 minutes of making a PAD? Y or N. ORO able to contact firefighters 

within 15 minutes of making a PAD? Y or N. ORO able to contact other (if needed) within 15 

minutes of making a PAD? Y or N. ORO unable to demonstrate any element.

Demonstration Type: Communications/notification test

Demonstration Frequency: Quarterly

Subtask: Activate responders

Description: ORO is able to demonstrate the ability to notify and activate necessary off-duty 

personnel.

Measurement: ORO can contact and activate off-duty personnel within x hour of notification 

of event. Off-duty personnel should be contacted via telephone or other method. In an ideal 

test, off-duty personnel would be required to report for duty in a designated location (e.g., for 

training), but due to overtime staffing concerns this may not be possible for many 

jurisdictions. Instead, ORO can use the ability to contact responders as a proxy measure, OR 

can attempt to use demonstration of successful mobilization of response components either 

in a series of drills, or in a massive mobilization exercise, tied to a time requirement. This test 

may occur in an unannounced environment in order to be a true demonstration of capability. 

Calculation: Two calculations are possible to test responder activation. Both are not 

necessary, though in a mobilization drill, a response activation call drill should also occur. 
A-16

Green: First: (# of off-duty responders who respond to activation call drill)/(total # off duty 

responders contacted) >/= G. Second: (# of off-duty responders who report to a designated 

location following an activation call within x hours)/(total # off duty responders who receive 

activation call) >/= g

White: First: (# of off-duty responders who respond to activation call drill)/(total # off duty 

responders contacted) >/= W. Second: (# of off-duty responders who report to a designated 

location following an activation call within x hours)/(total # off duty responders who receive 

activation call) >/= w

Yellow: First: (# of off-duty responders who respond to activation call drill)/(total # off duty 

responders contacted) >/= Y. Second: (# of off-duty responders who report to a designated 

location following an activation call within x hours)/(total # off duty responders who receive 

activation call) >/= y

Orange: First: (# of off-duty responders who respond to activation call drill)/(total # off duty 

responders contacted) </= Z. Second: (# of off-duty responders who report to a designated 

location following an activation call within x hours)/(total # off duty responders who receive 

activation call) < y

Demonstration Type: Drill

Demonstration Frequency: Quadrennially (if massive mobilization drill)

Subtask: Response support

Description: ORO is able to demonstrate the ability to bring in additional emergency service 

personnel through existing mutual aid agreements, contract services, or contact with other 

agencies/levels of government.

Measurement: Some portion of needs could likely be taken care of through mobilizing 

response components. However, to the extent that needs remain, measurements must 

capture whether the ORO is able to successfully contact resource support providers listed in 

existing plans or agreements, whether ORO can obtain a [notional] commitment for asset(s) 

relied upon in the plan, and whether ORO can obtain an estimated delivery time for asset(s) 

relied upon in plan (with a recording of the time). This would not need to be demonstrated 

frequently, but since it supports a task with high risk significance it should be tested at least 

quadrennially. 

Calculation: Measuring this subtask requires three demonstrations of capability. 

Green: First: (# of resource support providers ORO is able to contact)/(total # resource 

support providers) >/= g. Second: (# of resources ORO obtains notional commitment 

for)/(total # of resources ORO requests) >/= G. Third: Average estimated delivery time for 

committed assets </= 24 hours. Must demonstrate 3/3.

White: First: (# of resource support providers ORO is able to contact)/(total # resource 

support providers) >/= g. Second: (# of resources ORO obtains notional commitment 

for)/(total # of resources ORO requests) >/= G. Third: Average estimated delivery time for 

committed assets </= 24 hours. Must demonstrate at least 2/3.
A-17

Yellow: First: (# of resource support providers ORO is able to contact)/(total # resource 

support providers) >/= g. Second: (# of resources ORO obtains notional commitment 

for)/(total # of resources ORO requests) >/= G. Third: Average estimated delivery time for 

committed assets </= 24 hours. Must demonstrate 1/3.

Orange: ORO cannot demonstrate any of the three elements.

Demonstration Type: Exercise/Drill

Demonstration Frequency: Quadrennially
A-18

Task 7: Notify the Public

Subtask: Delivery of mass notification

Description: ORO is able to use a range of communications devices to reach x% of the EPZ 

population within 15 minutes.

Measurement: Primary alert and notification methods cover x% of the 10-mile EPZ 

population, and can reach them within 15 minutes. The range of communications devices 

may include emergency messaging over television or radio, cellular EAS, sirens, or other 

methods as deemed appropriate. It is beyond the scope of an exercise to monitor the entire 

population during an exercise. Instead, this should serve as a proxy measure that the 

communications systems extend over a wide enough range and in a diverse enough manner 

to reach an estimated 100% of the population. For example, a combination of cellular EAS

and sirens might be projected to reach x% of the population through the vast majority of the 

population having cellular phones (and a secondary method operating to reach those who do 

not). 

Notification tests can serve to meet requirements within a number of tasks, as public 

information and notification are critical throughout all phases of a radiological emergency. For 

example, though not explicitly listed as subtasks here, it is important for the ORO to notify the 

public of the protective action plan, including any evacuation routes, registration stations, 

decontamination stations, shelters, special needs facilities, or other relevant information to 

assist in the implementation of the protective action plan. 

Calculation: Test of EAS(s) able to disseminate emergency notification to an estimated x% 

of population. Estimation may come from executing a drill utilizing available communications 

systems, and following this with a random survey of the population in the 10-mile EPZ to 

determine if they received a test alert. This sample may be extrapolated to determine how 

much of the population could be reached in an emergency.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Develop follow-on emergency instruction message

Description: ORO develops an accurate and correct instructional message to the public for 

implementing the PAD.

Measurement: Within 15 minutes (tentative) following initial PAD, ORO develops an 

appropriate warning/instructional message that includes protective actions to be taken by the 

general public, evacuation routes by affected areas, methods to maximize shelter-in-place 

protection, a public inquiry number, and what evacuees should or should not take with them. 

Calculation:

Green: ORO is able to develop or provide instructional messaging within 15 minutes following 

initial PAD.

White: ORO is able to develop or provide instructional messaging within w minutes following 

initial PAD.
A-19

Yellow: ORO is able to develop or provide instructional messaging within y minutes following 

initial PAD.

Orange: ORO is unable to develop or provide instructional messaging within y minutes 

following initial PAD.

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Notification of non-English-speaking population

Description: ORO develops messages in languages other than English for delivery to nonEnglish speakers comprising a significant percentage of the population.

Measurement: Within 15 minutes following delivery of follow-on message, ORO develops an 

appropriate warning/instructional message equivalent to its English-language message in 

each language other than English spoken by more than 10,000 people within a given county 

or 5% of the voting-age population within a given county. 

Calculation:

Green: ORO is able to develop or provide instructional messaging in non-English languages 

spoken by 10,000 people or more than 5% of the population in counties within the EPZ within 

15 minutes following the initial follow-on message. 

White: ORO is able to develop or provide instructional messaging in non-English languages 

spoken by 10,000 people or more than 5% of the population in counties within the EPZ within 

w minutes following the initial follow-on message. 

Yellow: ORO is able to develop or provide instructional messaging in non-English languages 

spoken by 10,000 people or more than 5% of the population in counties within the EPZ within 

y minutes following the initial follow-on message. 

Orange: ORO is unable to develop or provide instructional messaging in non-English 

languages spoken by 10,000 people or more than 5% of the population in counties within the 

EPZ within y minutes following the initial follow-on message. 

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Establish Joint Information Center (JIC)

Description: The ORO must establish a JIC with the licensee in order to yield unified 

responses to questions from public and media.

Measurement: The ORO and licensee must work together to establish a JIC functional within 

x minutes of a Site Area Emergency.

Calculation:

Green: (time JIC is established) – (time Site Area Emergency is declared) </= g minutes
A-20

White: (time JIC is established) – (time Site Area Emergency is declared) </= w minutes

Yellow: (time JIC is established) – (time Site Area Emergency is declared) </= y minutes

Orange: (time JIC is established) – (time Site Area Emergency is declared) > y minutes

Demonstration Type: Exercise

Demonstration Frequency: Biannually

Subtask: Special needs populations

Description: ORO is able to alert and notify schools, licensed care facilities, correctional 

facilities, and persons with specific evacuation support requirements within the 10-mile EPZ.

Measurement: Demonstrate the capability to alert and notify all public school districts, 

correctional facilities and special care facilities that are expected or may necessitate 

protective actions for students. Demonstration requires that OROs actually contact public 

school systems/etc. during exercise.

Calculation:

Green: (# of facilities successfully alerted and notified within 15 minutes)/(total # of facilities) 

>/= g

White: (# of facilities successfully alerted and notified within 15 minutes)/(total # of facilities) 

>/= w

Yellow: (# of facilities successfully alerted and notified within 15 minutes)/(total # of facilities) 

>/= y

Orange: (# of facilities successfully alerted and notified within 15 minutes)/(total # of facilities) 

< y

Demonstration Type: Exercise

Demonstration Frequency: Biannually
A-21

Task 8: Implement Protective Action Decision

Subtask: Evacuation

Description: ORO is able to evacuate population in evacuation zones within required time, 

whether through lateral evacuation, radial evacuation, staged evacuation, or shelter-beforeevacuation.

Measurement: N/A (impractical). May involve real-world demonstration, constructed index, 

etc.

Calculation: N/A

Demonstration Type: N/A

Demonstration Frequency: N/A

Subtask: Shelter-in-place

Description: ORO is able to have population appropriate zones seek shelter within required 

time, whether through shelter-in-place, shelter in enhanced facilities, or shelter-beforeevacuation.

Measurement: N/A (impractical). May rely on survey data.

Calculation: N/A

Demonstration Type: N/A

Demonstration Frequency: N/A

Subtask: KI distribution

Description: ORO has distributed or has the ability to distribute KI to the public, responders, 

special needs populations, and transients.

Measurement: N/A. May use inventory as proxy, rather than a performance measure. 

Calculation: This is a proposed proxy measure. 

Green: (total # of doses disseminated to emergency workers) + (total # of doses available to 

emergency workers) = total emergency worker population? Y or N. (total # of doses 

disseminated to correctional facilities) + (total # of doses available to institutionalized 

individuals) = total incarcerated population? Y or N. (total # of doses disseminated to the 

public) + (total # of doses available to the public) = total population in 10-mile EPZ, including 

incarcerated population. Y or N. Demonstrate 3/3

White: (total # of doses disseminated to emergency workers) + (total # of doses available to 

emergency workers) = total emergency worker population? Y or N. (total # of doses 

disseminated to correctional facilities) + (total # of doses available to institutionalized 

individuals) = total incarcerated population? Y or N. (total # of doses disseminated to the 

public) + (total # of doses available to the public) = total population in 10-mile EPZ. Y or N. 

Demonstrate 2/3
A-22

Yellow: (total # of doses disseminated to emergency workers) + (total # of doses available to 

emergency workers) = total emergency worker population? Y or N. (total # of doses 

disseminated to correctional facilities) + (total # of doses available to institutionalized 

individuals) = total incarcerated population? Y or N. (total # of doses disseminated to the 

public) + (total # of doses available to the public) = total population in 10-mile EPZ, including 

incarcerated population. Y or N. Demonstrate 1/3

Orange: ORO is unable to perform any of the required subtasks

Demonstration Type: N/A

Demonstration Frequency: N/A

Subtask: Emergency Worker Protection

Description: ORO is able to develop protective actions for responders in the 10-mile EPZ 

and effectively implement them for affected responders. 

Measurement: Demonstrate the capability to alert and notify responders of PADs affecting 

them (including administration of KI or shelter-in-place). In place of actually requiring 

demonstration of KI ingestion, can determine that have stockpile of KI available, and are able 

to comprehend the order to ingest when directed to do so. 

Calculation: Requires two calculations

Green: First: (# of responders alerted to PADs for responders)/(total # responders) >/= g. 

Second: (# of responders who have KI available)/(total # of responders) >/= G

White: First: (# of responders alerted to PADs for responders)/(total # responders) >/= w. 

Second: (# of responders who have KI available)/(total # of responders) >/= W

Yellow: First: (# of responders alerted to PADs for responders)/(total # responders) >/= y. 

Second: (# of responders who have KI available)/(total # of responders) >/= Y

Orange: First: (# of responders alerted to PADs for responders)/(total # responders) < y. 

Second: (# of responders who have KI available)/(total # of responders) < Y

Demonstration Type: Exercise

Demonstration Frequency: Biannually
A-23

Task 9: Receive, Screen, and Support Evacuees

Subtask: Reception center operation

Description: ORO must be able to demonstrate the ability to establish and operate an 

appropriate number of reception centers for citizens evacuating the 10-mile EPZ.

Measurement: Demonstration will process simulated evacuees, to process evacuees at the 

reception center. Ideally, these “evacuees” will be actual residents of the 10-mile EPZ. 

Registration centers should be able to handle at least 20% of the total population of the EPZ 

within 12 hours. However, this figure is infeasible to conduct in a drill setting. Instead, a onehour drill should be conducted, if possible including “evacuees” totaling a threshold amount of 

the population for a one-hour time period (possibly 1.66%, 1/12 of 20%). “Evacuees” should 

include special needs populations, and non-English speakers. 

Calculation:

Green: (# of evacuees processed in 1 hour drill)/(total population of 10-mile EPZ) >/= g

White: (# of evacuees processed in 1 hour drill)/(total population of 10-mile EPZ) >/= w

Yellow: (# of evacuees processed in 1 hour drill)/(total population of 10-mile EPZ) < w

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Radiological monitoring

Description: ORO must demonstrate the ability to monitor residents of EPZ for 

contamination.

Measurement: ORO may use hand-held instruments, portal monitors, or other equipment as 

necessary, and should have sufficient number of trained staff to operate monitoring 

equipment. ORO should demonstrate the ability to monitor 20% of the EPZ in a 12-hour 

period. In a drill setting, this may involve calculating the hourly rate of monitoring necessary to 

meet the 20% threshold (possibly 1.66% per hour), and conducting a one-hour drill, using a 

pool of volunteer “evacuees,” who may be recycled through to avoid undue burden on the 

local population.

Calculation: 

Green: (# of evacuees monitored per hour)/(total number of residents in 10-mile EPZ) >g

White: (# of evacuees monitored per hour)/(total number of residents in 10-mile EPZ) >/= w

Yellow: (# of evacuees monitored per hour)/(total number of residents in 10-mile EPZ) < w

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Register evacuees
A-24

Description: ORO must be able to register and address the needs of evacuees—either 

online or through registration stations.

Measurement: Conduct a drill/reception center demonstration as part of a full-scale exercise. 

Could require demonstration for each affected county. Registration stations as a whole must 

be able to account for x% of total population in EPZ. This should be a throughput measure 

that can extrapolate from a small sample of volunteer “evacuees” to give an estimate of an 

ORO’s ability to deal with the entire population. To better approximate the need to rapidly 

process registration of evacuees, evaluators could monitor ability to achieve a certain rate of 

registration (# of registrants/hr). Ideally, bring in “evacuees” from local 10-mile EPZ so that 

this can serve as a learning experience for them, as well.

Calculation:

Green: (# evacuees registered)/(allotted time) >/= (required # evacuees registered)/(allotted 

time)

White: (# evacuees registered)/(allotted time) >/= w

Yellow: (# evacuees registered)/(allotted time) < w

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Congregate care

Description: ORO must be able to provide sufficient public shelter for evacuees from the 10-

mile EPZ.

Measurement: Demonstrate the ability to set up shelters adequate to accommodate 20% 

(tentative) of the EPZ population, including addressing access and functional needs. Because 

of the volume of this test, does not need “evacuees” to participate, but should test availability 

of beds, staff, and supplies. Because this is a massive sheltering exercise, congregate care 

should only be tested once every eight years. However, it is also important to note that this 

does have all-hazards value outside of radiological emergencies, and a congregate care test 

may be supplemented by demonstration of capacity during an actual emergency requiring a 

significant evacuation (e.g., hurricane or earthquake). 

Calculation: Requires two calculations

Green: First: (# of persons congregate care facilities in host or support jurisdictions can 

accommodate)/(total population in 10-mile EPZ) >/= 0.20. Second: (# congregate care 

facilities meeting ADA requirements without modification)/(# congregate care facilities) >/= G

White: First: (# of persons congregate care facilities in host or support jurisdictions can 

accommodate)/(total population in 10-mile EPZ) >/= w. Second: (# congregate care facilities 

meeting ADA requirements without modification)/(# congregate care facilities) >/= W

Yellow: First: (# of persons congregate care facilities in host or support jurisdictions can 

accommodate)/(total population in 10-mile EPZ) < w. Second: (# congregate care facilities 

meeting ADA requirements without modification)/(# congregate care facilities) < W
A-25

Demonstration Type: Drill

Demonstration Frequency: Once per cycle
A-26

Task 10: Manage Exposure and Contamination Risks

Subtask: General decontamination

Description: ORO must demonstrate the ability to decontaminate people and vehicles.

Measurement: ORO must be able to decontaminate victims, vehicles attempting to evacuate 

the EPZ, and pets or animals. Effective decontamination involves contamination control 

measures, such as safety requirements and decontamination protocol to be in place. 

Decontamination should occur in a timely and efficient manner. While the ORO should have 

the capacity to decontaminate 20% (tentative) of the EPZ population in a 12-hour period, 

conducting such a drill would be massive and infeasible. Instead, the ORO should be required 

to decontaminate a certain number of victims and vehicles, with a maximum time allowed for 

each decontamination. The mean time for all decontamination demonstrations should not 

exceed the maximum allowed time. 

Calculation: Calculation of this subtask requires two demonstrations.

Green: First: (total time for all demonstrations of human decontamination)/(total # of human 

decontaminations) </= g minutes. Second: (total time for all demonstrations of vehicle 

decontamination)/(total # of vehicle decontaminations) </= G

White: First: (total time for all demonstrations of human decontamination)/(total # of human 

decontaminations) </= w minutes. Second: (total time for all demonstrations of vehicle 

decontamination)/(total # of vehicle decontaminations) </= W

Yellow: First: (total time for all demonstrations of human decontamination)/(total # of human 

decontaminations) > w minutes. Second: (total time for all demonstrations of vehicle 

decontamination)/(total # of vehicle decontaminations) > W

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Emergency worker decontamination

Description: ORO has the ability to monitor and decontaminate emergency workers and 

emergency service vehicles.

Measurement: The ORO should have the ability to decontaminate all emergency workers 

and emergency service vehicles operating in the 10-mile EPZ. Again, because such a drill 

would be massive, it would be infeasible. A decontamination drill for emergency workers and 

vehicles could be conducted in coordination with a general decontamination drill, using similar 

time maximum time allowed.

Calculation: Calculation of this subtask requires two demonstrations. 

Green: First: (total time for all demonstrations of responder decontamination)/(total # of 

responder decontaminations) </= g minutes. Second: (total time for all demonstrations of 

emergency vehicle decontamination)/(total # of emergency vehicle decontaminations) </=G

White: First: (total time for all demonstrations of responder decontamination)/(total # of 

responder decontaminations) </= w minutes. Second: (total time for all demonstrations of 

emergency vehicle decontamination)/(total # of emergency vehicle decontaminations) </= W
A-27

Yellow: First: (total time for all demonstrations of responder decontamination)/(total # of 

responder decontaminations) >/= > w minutes. Second: (total time for all demonstrations of 

emergency vehicle decontamination)/(total # of emergency vehicle decontaminations) > W

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Evacuation Zone Access Control

Description: ORO is able to establish access control for evacuation zones and other nonplant restricted zones.

Measurement: Required demonstration of implementing access control for evacuation zone 

through a drill. May involve limited number of roads or complete functional test establishing 

access control to all roads in 10-mile EPZ. Response time needed to establish access control 

into evacuation zone should not exceed x hours from the notification of PAD. 

Calculation: 

Green: (X time recorded) – (Z maximum time allowed) </= g

White: (X time recorded) – (Z maximum time allowed) </= w

Yellow: (X time recorded) – (Z maximum time allowed) > w

Demonstration Type: Drill

Demonstration Frequency: Annually

Subtask: Secure contaminated/restricted zones

Description: ORO has the ability to secure potentially contaminated areas.

Measurement: This subtask may be a shared responsibility of the facility operator, the ORO, 

NRC, and FEMA, especially during the intermediate to late phases of a radiological event. 

During the early phase (as a release is occurring or just after occurring), the ORO would likely 

be responsible for providing physical security to contaminated areas, which is the primary 

concern for this subtask. Ability to rapidly establish physical security perimeters around a 

designated area during a drill setting could serve as an effective proxy for securing multiple 

zones in the EPZ. The time allowed to establish a physical security perimeter will vary by 

plant depending upon the size, population, road access, and geographic features in the 10-

mile EPZ. 

Calculation: (Time required to establish physical security perimeter in drill)/(time allowed to 

establish physical security perimeter in drill) </= x

Green: (Time required to establish physical security perimeter in drill)/(time allowed to 

establish physical security perimeter in drill) </= g

White: (Time required to establish physical security perimeter in drill)/(time allowed to 

establish physical security perimeter in drill) </= w
A-28

Yellow: (Time required to establish physical security perimeter in drill)/(time allowed to 

establish physical security perimeter in drill) > w

Demonstration Type: Drills

Demonstration Frequency: Once per cycle

Subtask: Nuclear facility access control support

Description: ORO is able to effectively assist nuclear facility with controlling and restricting 

access to the facility when needed. 

Measurement: Required demonstration of implementing access control for nuclear facility 

through a drill. May involve limited restriction of access, or test of complete access control. 

Response time needed to establish access control into evacuation zone should not exceed x 

hours from the notification of the PAD. This may not need to be tested every performance 

period, but could be tested in conjunction with hostile action drills.

Calculation: 

Green: (X time recorded) – (Z maximum time allowed) </= g

White: (X time recorded) – (Z maximum time allowed) </= w

Yellow: (X time recorded) – (Z maximum time allowed) > w

Demonstration Type: Drill

Demonstration Frequency: Once per cycle

Subtask: Identify agricultural contamination

Description: ORO is able to identify farms/livestock/agriculture that have potential to become 

contaminated in the event of a release.

Measurement: ORO must develop or provide a map identifying locations of farms 

possessing livestock or agriculture with potential to become contaminated in the event of a 

release. Map must account for 50-mile ingestion pathway, or beyond if plume projections 

exceed 50 miles. 

Calculation: 

Green: ORO provides an accurate map of potentially affected farms in the 50-mile ingestion 

pathway within g minutes.

White: ORO provides an accurate map of potentially affected farms in the 50-mile ingestion 

pathway within w minutes.

Yellow: ORO cannot provide an accurate map of potentially affected farms in the 50-mile 

ingestion pathway within w minutes.

Demonstration Type: Exercise

Demonstration Frequency: Annually
A-29

Subtask: Stored feed advisories

Description: ORO is able to advise farmers to shelter and place at-risk livestock on stored 

feed.

Measurement: ORO is able to use a variety of communications methods to advise x% of 

farmers in plume pathway within 50 miles to place livestock on stored feed, or take other 

preemptive actions. Should be tested in a drill setting, with OROs having to contact farmers, 

notifying them that this is just a drill.

Calculation:

Green: (# of farmers contacted)/(population of farmers in ingestion pathway zone) >/= g

White: (# of farmers contacted)/(population of farmers in ingestion pathway zone) >/= w

Yellow: (# of farmers contacted)/(population of farmers in ingestion pathway zone) < w

Demonstration Type: Exercise

Demonstration Frequency: Annually

B-1

APPENDIX B: INITIAL REVIEW OF EXISTING CRITERIA

Extract from Deliverable 4.2a:

For some RERP elements, performance may be difficult to demonstrate directly. In those 

instances, indicators of capability may be useful to suggest that adequate performance is 

possible or likely. 

To support initial consideration of which RERP elements are most suitable for performancebased oversight, the project team has reviewed existing documentation—principally the FEMA 

REP Program Manual—to find objective performance criteria or capability indicators that have 

already been developed. All-hazards National Preparedness Goal core capability metrics also 

have been considered.

Table B-1 below and on the following pages lists, for each standard, performance measures 

(PM) and capability indicators (CI). Those not drawn from existing REP Program documentation 

are indicated with an asterisk (*). For existing REP Program criteria, citations are given for either 

the demonstration criteria or the planning criteria. Some of these are followed by italicized 

suggestions for a more quantitative and objective measure that could be adapted from the 

existing REP Program criteria. Existing RERP elements most suitable for performance-based 

oversight will be those with existing quantitative PMs.

Table B-1. Review of Existing Criteria for Actual or Implied Quantitative Measures

 and Indicators

NRC/FEMA Planning 

Standard

Performance Measure (PM)/ Capability 

Indicator (CI)

Existing 

PM? Comments

A – Assignment of 

Responsibility 

• *CI: Number of activations of ORO 

(State and each locality) EOC over 

previous year for real-world incidents 

and exercises. Portion of such 

activations involving 24-hour (multi-shift) 

operations.

NO

Generally involves 

planning input for 

performance, not 

performance itself. 

Suggested CI may also 

be indicator for “training” 

(job proficiency) for 

general emergency 

response.

C – Emergency 

Response Support and 

Resources

NO

Generally involves 

identification of outside 

resources available to 

assist. See comment 

under H regarding 

resource typing.

D – Emergency 

Classification System NO

Involves yes/no 

determination of whether 

ORO is using same 

classification and 

emergency action levels 

as licensee.
B-2

E – Notification 

Methods and 

Procedures

• PM: Activities associated with primary 

alerting and notification of the public are 

completed in a timely manner following 

the initial decision by authorized offsite 

emergency officials to notify the public 

of an emergency situation. The initial 

instructional message to the public must 

include as a minimum the elements 

required by current REP guidance. 

(Criterion 5.a.1) Initial instructional 

message to population is ready for 

transmission in x minutes.

• PM: Backup alert and notification of the 

public is completed within a reasonable 

time following the detection by the ORO 

of a failure of the primary alert and 

notification system. (Criterion 5.a.3) 

Message is ready for delivery by backup 

system within the recommended 45 

minutes following identification of need 

for backup notification.

• CI: Primary alert and notification method 

covers 100% of the 10-mile EPZ 

population. 

• CI: Computation of siren operability 

(percentage of sirens operable) for the 

immediately preceding calendar year is 

at least 90%, based on simple average 

of all regularly conducted tests 

employed as part of testing program. 

(FEMA-REP-10)

YES

Existing criteria may be 

made more explicit and 

quantitative.

F – Emergency 

Communications

• PM: Periodic test results of primary and 

backup communications systems (to 

demonstrate 24/7 availability of a 

primary and at least one backup 

system). (Criterion 1.d.1) Percentage 

successful test results.

YES

G – Public Education 

and Information

• *CI: Results of survey indicating 

whether 10-mile EPZ population 

understands alert signals, knows by 

what means to receive information, 

knows evacuation concept, knows 

evacuation routes and host site, and 

has prepared an evacuation or shelterin-place kit.

NO
B-3

H – Emergency

Facilities and 

Equipment

• CI: Quantities of monitoring instruments 

required, based on the number of field 

monitoring teams and reception center 

requirements, and quantities available 

by model. (H.10) Percentage of 

requirement available. Portable 

monitoring units per emergency worker. 

Portal monitoring units per EPZ 

population. [Availability]

• CI: Periodic operational checks and 

calibration of monitoring instruments. 

Percentage successful test results. 

[Reliability]

• CI: Number and contents of emergency 

kits, and number of items in each 

emergency kit. (H.11) 

NO

Generally involves input 

for performance, not 

performance itself. 

Determinations on 

adequate equipment 

may require a resource 

typing effort for RERP 

under the auspices of 

NIMS and Emergency 

Management Assistance 

Compact. This would 

define, for example, field 

monitoring teams of 

different capacities, and 

the equipment 

requirements for each. 

Such definitions would 

enable: (1) monitoring of 

team status (number of 

teams operationally 

ready vs. reconstituting

vs. assigned), (2) better 

gauging outside 

resources available 

under mutual aid. 

I – Accident 

Assessment

• PM: Two or more FMTs must 

demonstrate the capability to make and 

report measurements of ambient 

radiation to the field team coordinator, 

dose assessment team, or other 

appropriate authority. (Criterion 4.a.3) 

Percentage of accurate readings (from 

simulation source) delivered within x 

time.

• *PM: Number of timely and accurate 

(given available source term and 

meteorological data) independent plume 

projections developed during drills and 

exercises over previous eight quarters, 

given “opportunities” to do so.

YES

Criteria may need to be 

made more explicit and 

quantitative. Suggested 

PM criterion is based on 

a licensee performance 

indicator.
B-4

J – Protective 

Response

• PM: Demonstrate the capability to alert 

and notify all public school 

systems/districts of

emergency conditions that are expected 

to or may necessitate protective actions 

for students. Demonstration requires 

that the OROs actually contact public 

school systems/districts during the 

exercise. Percentage of districts 

successfully contacted within x time. 

• PM: Demonstrate capability to mobilize 

one third of the resources necessary to 

monitor 20% of the 10-mile EPZ 

population within a 12-hour period. 

(Criterion 6.a.1)

• PM: Staff responsible for the 

radiological monitoring of evacuees 

must demonstrate the capability to 

attain and sustain, within 12 hours, a 

monitoring productivity rate per hour 

needed to monitor the 20% EPZ 

population planning base. The 

monitoring productivity rate per hour is 

the number of evacuees that can be 

monitored, per hour, by the total 

complement of monitors using an 

appropriate procedure. (Criterion 6.a.1)

• CI: Availability of [blankets, cots, food 

supplies for congregate care centers] 

must be verified by providing the 

evaluator a list of sources with locations 

and estimates of quantities. (Criterion 

6.c.1)

• CI: OROs must plan for a sufficient 

number of congregate care centers in 

host/support jurisdictions to 

accommodate a minimum of 20% of the 

EPZ population. (Criterion 6.c.1)

• CI: Inventories of KI sufficient for use 

by: (1) emergency workers; (2) 

institutionalized individuals, as indicated 

in capacity lists for facilities; and (3) 

where stipulated by the 

plans/procedures, members of the 

general public (including transients) 

within the plume pathway EPZ. 

(Criterion 1.e.1) Percentage of 

estimated requirement available in 

inventory.

• CI: Number of persons without private 

transportation [personally owned 

vehicles] in 10-mile EPZ. (J.10.g) 

Number of such persons as percentage 

of EPZ population; number of such 

persons as percentage of identified nonprivate transportation capacity.

YES
B-5

K – Radiological 

Exposure Control

• CI: Sufficient quantities of appropriate 

direct-reading and permanent record 

dosimetry and dosimeter chargers must 

be available for issuance to all 

emergency workers who will be 

dispatched to perform an ORO mission. 

(Criterion 1.e.1) Dosimeters divided by 

emergency personnel. [Availability]

• CI: Dosimeters must be inspected 

periodically for electrical leakage 

(Criterion 1.e.1) Percentage of 

dosimeters tested, and percentage of 

successful test results. [Reliability]

NO

L – Medical and Public 

Health Support

• PM: Approximate response time 

needed to establish controlled areas 

and fully prepare necessary 

medical/radiological staff. (L.1) Specify 

a maximum time from notification; 

specify ability to demonstrate 

throughput of some percentage of 

requirement, similar to demonstrating 

monitoring for congregate care.

• CI: There is at least one primary and 

one backup medical facility for 

treatment of contaminated injured, and 

each has at least one trained physician 

and one trained nurse to perform and 

supervise treatment of contaminated 

injured individuals. (L.1; REP Program

Manual, p. III-63)

CI: Maximum number of contaminated 

injured or exposed patients who could 

be treated at one time. (L.1) This 

maximum divided by 10-mile EPZ 

population.

YES

Criteria may need to be 

made more explicit and 

quantitative.

M – Recovery and 

Reentry Planning and 

Post-Accident 

Operations

NO

N – Exercises and 

Drills

• CI: Number and frequency of drills and 

exercises conducted for different 

purposes (e.g., ingestion pathway, 

medical drills).

NO

Generally involves 

planning input for 

performance, not 

performance itself.

O – Radiological 

Emergency Response 

Training

• CI: Percentage of ORO members who 

have participated in a radiological 

emergency preparedness exercise over 

previous eight quarters.

NO

Generally involves 

planning input for 

performance, not 

performance itself. CI 

given is based on a 

licensee indicator. It is 

listed for training rather 

than exercises as a 

matter of maintaining 

proficiency.
B-6

P – Responsibility for 

the Planning Effort NO

Generally involves 

planning input for 

performance, not 

performance itself.
C-1

APPENDIX C: ONE SOFTWARE TOOL FOR EVALUATION

The All-hazards Response and Preparedness Assessment Tool (ARPAT), developed by 

Science Applications International Corporation (SAIC), aids in organizing information for 

assessments and presenting it in a more visually appealing manner. This functionality could be 

especially useful under the current RERP oversight system, given the large amount of 

documentation it collects for determinations on the adequacy of plans and supporting 

documentation. It is not clear whether it would be useful for the performance-based RERP 

oversight system proposed in this paper.

ARPAT is designed to support assessments for FEMA REP Program, among other constructs. 

ARPAT presents a visual map of the 16 planning standards, linked to a central “Overall 

Evaluation” category, as shown in Figure C-1, below.

Figure C-1. ARPAT Summary Screen

Source: FEMA 2011
C-2

Each of the 16 current planning standards exists as a separate folder, within which each 

assessment criterion affiliated with a task is listed in a separate line, to be coded according to 

whether performance was “Adequate,” “Adequate with Consequences,” “Inadequate,” or “Not 

Applicable” (as shown in Figure C-2, below). These evaluations are color coded green-yellowred, and when entered for all of the criteria, produce a visual depiction of the planning standard, 

shaded green, yellow, red, or some mix thereof depending on how well the ORO met the mix of 

criteria for the planning standard. When all planning standards have been entered, a complete 

evaluation map is produced, with visualization of the areas of greatest strength and weakness.

Figure C-2. ARPAT Data Entry

Source: FEMA 2011

ARPAT also allows explanations to be attached to each score, allowing an easy means of 

organizing a plan review or an AAR. Additionally, users may attach documents to the relevant 

planning standard folder, which can serve as a repository for additional information about 

mutual aid agreements, capability improvement projects, or other important documents. 

ARPAT could have some utility for concisely presenting performance-based oversight results. 

However, ARPAT would require several changes to fit the framework proposed in this paper. 

First, the planning standards would need to be changed to reflect the new critical tasks. The 

folders attached to each critical task would also need to be changed to reflect the subtasks for 

each. When it was developed, ARPAT also allowed for assessments against the target 

capabilities (now core capabilities) of the all-hazards National Preparedness Goal, so ARPAT is 

able to accommodate different criteria.

Scoring within ARPAT also would need to change. For subtasks, the green-yellow-red scoring 

would need to be altered to include “white” categories, and to replace the “red” assessment 

level with an “orange” assessment level. 
C-3

Additionally, ARPAT would need to incorporate the overall task scoring rules described in 

Section 6.6 of this paper, to generate rollup scores. ARPAT currently allows for weighting 

criteria within the calculation of a mathematical score; the revised approach would require 

if/then rules for rollup scores. 

ARPAT’s document repository features—the ability to associate maps, mutual aid agreements, 

and capability improvement plans with a given planning standard or criterion—seem less 

important for a performance-based system, except when less-than-target performance occurs. 

ARPAT could allow for compiling information relevant to capabilities that underlie performance, 

and for tracking corrective actions and their resolution. The folder organization for compiling 

associated documents would need to be reworked to serve this purpose. 

In summary, ARPAT software does not directly support performance-based evaluation in its 

current form. Yet a modified version of ARPAT could support compilation of well-organized and 

visually appealing summaries of findings and follow-on actions from single exercises and/or an 

exercise cycle for the ORO associated with a particular NPP site. 





NUREG/CR-7195 Risk-Informed and Performance-Based Oversight of Radiological 

Emergency Response Programs