NUREG-1764 

 

 

Guidance for the

Review of Changes to

Human Actions

Draft Report for Comment

U.S. Nuclear Regulatory Commission

Office of Nuclear Reactor Regulation

Washington, DC 20555-0001
NUREG-1764

Guidance for the

Review of Changes to

Human Actions

Draft Report for Comment

Manuscript Completed: December 2002

Date Published: December 2002

 

Division of Inspection Program Management

Office of Nuclear Reactor Regulation

U.S. Nuclear Regulatory Commission

Washington, DC 20555-0001

iii

ABSTRACT

This document is incorporated by reference into NRC’s “Standard Review Plan” (NUREG-0800),

Chapter 18, “Human Factors Engineering.” 

The U.S. Nuclear Regulatory Commission (NRC) is addressing the human performance aspects of

changes to operator actions that are credited for safety, especially those involving changes in the

licensing basis of the plant; e.g., use of manual action in place of an automatic action for safety system

operations. In this document the term human action and operator action are used synonymously because

most of the type of actions addressed are performed by operations staff. Risk-informed guidance and

acceptance criteria for the review of licensee proposals are presented. The review method uses a graded,

risk-informed approach and provides guidance for reviewing the human performance aspects of changes

to plant systems and operations. The evaluation method uses a two-phase approach. The first phase is a

screening analysis of the plant modification and the affected human actions (HAs) to determine their risk

importance. This phase is comprised of three steps: use of Regulatory Guide 1.174 to determine the risk

importance of the entire plant change or modification which involves the operator action; a quantitative

determination of the risk importance of the operator action itself, and a qualitative evaluation. Three risk

regions are defined for placement of the proposed operator action: high, medium, and lower risk regions. 

In the second phase, HAs are reviewed using human factors engineering criteria to ensure the proposed

HA can be reliably performed when required. HAs in the high-risk region receive a detailed review and

those in the medium-risk region receive a less detailed review that is commensurate with their risk. For

HAs placed in the lower-risk region, a minimal or no human factors review is performed.

v

TABLE OF CONTENTS

Page

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

1 INTRODUCTION ............................................................... 1

2 RISK SCREENING PROCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1 Changes to Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Overview of Screening Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Step 1 - Change in Risk from Overall Modification (Permanent Change) . . . . . . . . . . . 9

2.4 Step 1 - Change in Risk from Overall Modification (Temporary Change) . . . . . . . . . . 13

2.5 Step 2 - Risk Evaluation of the Changed Human Action . . . . . . . . . . . . . . . . . . . . . . . 16

2.6 Step 3 - Qualitative Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.7 Generic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.8 Comparison of PRA Results to Acceptance Guidelines . . . . . . . . . . . . . . . . . . . . . . . . 26

2.9 Level of HFE Review of the Changed Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 REGION I REVIEW GUIDANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.1 General Deterministic Review Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.2 Operating Experience Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 Functional Requirements Analysis and Functional Allocation . . . . . . . . . . . . . . . . . . . 31

3.4 Task Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5 Staffing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.6 Probabilistic Risk and Human Reliability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.7 Human-System Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.8 Procedure Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.9 Training Program Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.10 Human Factors Verification and Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.11 Human Performance Monitoring Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4 REGION II REVIEW GUIDANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.1 General Deterministic Review Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.3 Design of HSIs, Procedures, and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.4 Human Action Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
vi

TABLE OF CONTENTS

(continued)

Page

5 FINAL DECISION ON ACCEPTANCE OF HUMAN ACTIONS . . . . . . . . . . . . . . . . . . . . . . 49

6 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

GLOSSARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

APPENDIX Generic Risk-Important Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1

FIGURES

2.0 Risk-Informed Screening and HFE Review Flow Chart ................................... 7

2.1 Acceptance Guidelines for Core Damage Frequency (CDF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Acceptance Guidelines for Large Early Release Frequency (LERF) . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 Guidelines for Integrated Risk Increase - ICCDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4 Guidelines for Integrated Risk Increase - ICLERP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.5 RAW vs. New Baseline CDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.6 FV vs. New Baseline CDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.7 RAW(L) vs. New Baseline LERF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.8 FV(L) vs. New Baseline LERF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

TABLES

2.1 Action on Completion of Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2 Preliminary Placement of HAs in Risk Regions for Submittals without Risk Information . . . . . . . 25

2.3 Levels of Review for Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

A.1 Generic BWR Risk-Important Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1

A.2 Generic PWR Risk-Important Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2
vii

EXECUTIVE SUMMARY

The U. S. Nuclear Regulatory Commission (NRC) reviews changes in operator actions that are credited

in plant safety analyses. Changes in credited action may result from a variety of plant activities such as: 

plant modifications, procedure changes, equipment failures, justifications for continued operations

(JCOs), and identified discrepancies in equipment performance or safety analyses. Relevant review

considerations are described in NRC information notices and generic issues. Information Notice (IN) 

91-18 (NRC, 1991) discusses the conditions under which manual actions may be used in place of

automatic actions for safety system operations. IN 97-78 (NRC, 1997) alerts licensees to the importance

of considering the effects on human performance of such changes made to plant safety systems. 

This document contains guidance to address the review of risk-important operator actions, including

emergency core cooling system (ECCS) switchover, and other types of required operator actions. A

graded, risk-informed approach is used in conformance with Regulatory Guide (RG) 1.174 (NRC, 1998)

and guidance is provided for reviewing the human performance aspects of changes to plant systems and

operations. Risk insights are used to determine the level of regulatory review the staff should perform. 

Human actions (HAs) that are considered more risk significant receive a detailed review, while those of

less risk significance receive a less detailed review. In this document the term HA and operator action are

used synonymously because most of the type of actions addressed are performed by operations staff. In

keeping with RG 1.174, this guidance does not preclude other approaches for requesting changes to a

plant’s licensing basis or other approaches for requesting changes in HAs. Rather, this approach to the

review of HAs is intended to improve consistency in regulatory reviews and decisions. 

A two-phase evaluation method is used. The first phase is a risk screening and analysis of the licensee’s

identification of affected HAs and a determination of their risk importance. The second is a human

factors engineering (HFE) review of the affected HAs. Each is described below.

Risk Screening Process

A plant change may include equipment, as well as human action changes. This approach focuses on the

HA, while risk screening of equipment changes can be addressed directly using RG 1.174.

A three step screening analysis is used to locate the plant modification and its associated HAs in risk

space using guidance similar to that of RG 1.174. The first two steps of the process are quantitative

while the third step is qualitative. Essentially, plant modifications and their associated HAs can be

categorized into regions of high, medium, and lower risk. This categorization is used to determine the

level of graded HFE review needed. The important steps of this process are described below. 

The licensee reviews a proposed plant change to identify HAs that are new actions, modified actions, or

involve modified task demands. A 10 CFR (Code of Federal Regulation) 50.59 evaluation is conducted

by the licensee for any changes that affect the licensee’s Final Safety Analysis Report (FSAR). This

evaluation may result in the identification of activities associated with the change, which require NRC

review and approval prior to implementation.

For the risk-informed review, the licensee would make an initial screening risk calculation and submit

this to NRC with the request for approval of the change. The first step of the risk screening evaluates the

full modification including both equipment and HAs and is performed using NRC’s RG 1.174 directly. 
EXECUTIVE SUMMARY

viii

Risk calculations include the change in risk or core damage frequency (CDF) due to the full modification

(CDFmod) that includes the HA. This may determine that the full modification, including the HA, is in

Region III (the lower risk region). If so, then in most cases the only NRC review that would be

necessary, would be one to ensure that there exists a valid technical basis for the low risk.

The second step of the NRC’s risk screening focuses on the risk due to the HA. This step uses both the

Risk Achievement Worth (RAW) and the Fussel-Vesely (FV) risk importance measures to determine the

risk significance of the HA in question. This step identifies the effect on risk if the HA were to fail

(using the RAW) and the relative contribution to risk of the action (using the FV). Uncertainty with

respect to human actions is treated by setting the HA failure probability to 1.0 for the action under

review. This second step of the risk screening tentatively places the HA in one of three risk regions

(high, medium, or lower) for determining the level of HFE review to be performed by NRC. The

importance of the HA with respect to both CDF and large early release frequency (LERF) is then

assessed. For those HAs determined to be risk significant (i.e., in the high or medium risk regions), the

intent of the detailed HFE review is to ensure that they can be successfully performed, when required, in

order to limit the risk associated with failure of the HAs.

The third and final step of the risk screening is qualitative and allows NRC to adjust the quantitative

evaluation that is produced in Step 2, if necessary. It includes several factors, such as: defense in depth,

quality of the (human reliability analysis/probabilistic safety analysis) HRA/PSA analyses, and changes

in automation involved.

If the action is verified to be in the lower risk region, then the licensee’s change likely would be

permitted with either minimal or no further NRC human factors review. If the action is in the medium

risk region then a moderate, top level human factors review is performed by NRC. If the action is in the

high risk region, then a more detailed review is in order, which would include human factors,

deterministic, and risk analysis.

The risk screening calculations also consider whether the proposed change is a permanent or a temporary

change. If temporary, the screening includes consideration of the length of time that it will be in place. 

In this case the method considers the integrated risk due to the modification over the time that the change

or modification is to be in place (or the integrated conditional core damage probability - ICCDP). 

Similar calculations would be performed for (LERF) where appropriate.

Human Factors Engineering Review

In this phase, the proposed HAs are reviewed to ensure that they can be reliably performed when needed. 

Again, the details of the review are commensurate with the risk. Three levels of risk and NRC review are

presented. The review criteria are based on an adaptation of existing NRC review guidance for human

factors, as found in: NUREG-0800 (NRC, 1996a), NUREG-0711 (NRC, 1994), NUREG-0700, Rev. 1,

(NRC, 1996b), and IN 97-78 (NRC, 1998). 

A Region I review is used for HAs in the high risk category. Changes in Region I require the most

stringent review and include most of the review elements from NUREG-0711.
EXECUTIVE SUMMARY

ix

HAs in the medium risk category receive a Region II review by the NRC. While the guidance addresses

similar topical areas as the Region I review, the extent of the staff review is notably less. The evaluation

processes for this region are less prescriptive and provide greater latitude to both the licensee and the

NRC reviewers for the collection and analysis of information. The Region II evaluation process includes

the following elements: General Deterministic Review Criteria; Analysis; Design of Human-System

Interface (HSI), Procedures, Training; and Human Action Verification.

HAs in the lower risk category receive only a limited Region III review by the NRC. The staff review is

generally limited to verification that the action is in fact in Region III. Typically, no human factors

review is necessary. However, NRC may specify a few review areas based on the results of Step 3 of the

risk screening process.

Licensees may choose to use the Region II guidance to address human factors considerations for HAs

that fall into Region III.

Final Decision on Acceptance of Human Actions

Once the NRC completes its review of a proposed change in HAs, a final decision on the acceptability of

the change is made based on the information that has been gathered, reviewed, and evaluated. The

results of the different analyses are considered in an integrated manner (i.e., the decision is not driven

solely by the numerical results of the risk assessment). This approach complements the NRC's

deterministic approach, supports the NRC's traditional defense-in-depth philosophy, and takes into

consideration both traditional engineering and risk information. Both qualitative and quantitative

analyses and information are used. The main factors considered in the decision process include the

following:

 Change in CDF - The increase in CDF due to the modification (CDFmod).

 Change in LERF - The increase in LERF due to the modification (LERFmod).

 Risk Importance Measures for the Human Action - The values of the Risk Achievement Worth

(RAW) and Fussell-Vesely (FV) risk importance measures.

 Time and Integrated Risk - Risks integrated over the length of time that a temporary change will

be in place.

 Human Factors - The basis that operators can perform the actions required for the modification in

question, as determined by the HFE review criteria used by NRC for the review.

 Deterministic Criteria - Satisfaction of the deterministic review guidance provided in Section 3.1

of the Region I review guidance or Section 4.1 of the Region II review guidance.

Additional or supplemental factors that could influence the acceptability of a change are delineated.

xi

FOREWORD

This document is consistent with the NRC strategic performance goals as stated in “Strategic Plan: Fiscal

Year 2000 - Fiscal Year 2005" (NUREG-1614, Vol. 2, Part 1) as follows:

Maintain safety -- This document will ensure that changes to operating licenses will maintain safety

and meet regulatory requirements. This NUREG provides unified, standardized, risk-informed guidance

for NRC staff when they review changes to human actions that are credited for safety in nuclear power

plants. It is incorporated by reference into NRC’s “Standard Review Plan” (NUREG-0800), which is

NRC’s fundamental document for NRC reviews related to nuclear power plants. 

In particular, this document will aid in the resolution of issues related to age-degradation of equipment. 

As equipment wears out, licensees are increasingly proposing that humans perform functions that

previously were performed by safety-related equipment. Because the scope of this NUREG includes

reviews of this type, and because reviews of this type are increasing, publication of this document at this

time is particularly relevant to maintaining safety. 

Make NRC activities and decisions more effective and efficient -- This document promotes effective

and efficient decision making by collecting into one document review guidance that previously was

contained in multiple documents. This document therefore unifies and standardizes the review guidance. 

Furthermore, this document promotes efficiency by being risk-informed: more regulatory attention is

focused on the more risk-important changes; less regulatory attention is paid to less risk-important

changes. This NUREG is the first NRC document to provide official guidance to NRC staff on how to

use risk information for reviews in the field of human factors engineering. 

Reduce unnecessary regulatory burden -- By being risk-informed, and by focusing less regulatory

attention on changes that are less risk-important, this document reduces unnecessary regulatory burden

on stakeholders. By issuing this document first as a Draft NUREG for pubic comment, NRC is actively

seeking stakeholder input to identify opportunities for reducing unnecessary regulatory burden.

Increase public confidence -- By gathering into one document guidance that was previously in multiple

documents, this document improves communication and enhances NRC’s credibility by being a wellmanaged agency. By issuing this document first as a Draft NUREG for public comment, NRC is making

public participation in the regulatory process more accessible.

Office of Nuclear Regulatory Research

U.S. Nuclear Regulatory Commission

xiii

ACKNOWLEDGMENTS

This document was developed through the efforts of a number of individuals. James C. Higgins and John

M. O’Hara, Brookhaven National Laboratory, were the principal contributors to the preparation of this

document. Paul M. Lewis and J. J. Persensky of the U. S. Nuclear Regulatory Commission, Office of

Nuclear Regulatory Research, provided overall management to preparing this document. James P.

Bongarra, Jr., U. S. Nuclear Regulatory Commission, Office of Nuclear Reactor Regulation, was the

technical coordinator for this project. Staff of the Office of Nuclear Reactor Regulation who provided

critical review of this document were, Richard J. Eckenrode; Clare P. Goodman; Gareth W. Parry;

Michael C. Cheok; and Michael Franovich. Members of the U. S. Nuclear Regulatory Commission,

Office of Nuclear Regulatory Research who provided critical review of this document were, 

Nathan O. Siu and Erasmia Lois. 

xv

ACRONYMS

ADS automatic depressurization system

AEOD analysis and evaluation of operational data

AFW auxiliary feedwater

ANS American Nuclear Society

ANSI American National Standards Institute

AOT allowed outage time

ATWS anticipated transient without scram

BNL Brookhaven National Laboratory

BWR boiling water reactor

CBP computer-based procedures

CCDF cumulative value of core damage frequency

CDF core damage frequency

CFR Code of Federal Regulations

CR control room

DBE design basis event

DEP depressurization

DHR decay heat removal

ECCS emergency core cooling system

EOP emergency operating procedures

FSAR final safety analysis report

FV Fussell-Vesely

FW feedwater 

GDC general design criteria

GTG generic technical guidelines

HA human actions

HEP human error probability

HFE human factors engineering

HRA human reliability analysis

HSI human-system interface

I&C instrumentation and control

IC isolation condenser

ICCDP incremental conditional core damage probability

ICLERP incremental conditional large early release probability

IN information notice 

IPE individual plant examination

ISLOCA interfacing systems LOCA

JCO justification for continued operations

LERF large early release frequency

LOCA loss-of-coolant accident

LOOP loss of offsite power

MSLB main steam line break

NEI Nuclear Energy Institute

NPP nuclear power plant

NRC Nuclear Regulatory Commission

OER operating experience review
ACRONYMS

xvi

PORV power operated relief valve

PRA probabilistic risk assessment

PSA probabilistic safety assessment

PSF performing shaping factor

PWR pressurized water reactor

RAI request for additional information

RAW risk achievement worth

RCP reactor coolant pump

RCS reactor coolant system

RG regulatory guide

RIS regulatory issue summary

SBO station blackout

SG steam generator

SGTR steam generator tube rupture

SLC standby liquid control

SROA safety-related operator action

SRV safety relief valves

TA task analysis

TMI Three-Mile Island

USQ unreviewed safety question

V&V verification and validation
1

1 INTRODUCTION

In Information Notice (IN) 91-18 (NRC, 1991), the U.S. Nuclear Regulatory Commission (NRC)

discussed the conditions under which manual actions may be used in place of automatic actions for safety

system operations. IN 97-78 (NRC, 1997) alerted licensees to the importance of considering the effects

on human performance of such changes made to plant safety systems:

The original design of nuclear power plant safety systems and their ability to respond to designbasis accidents are described in licensees' FSARs (final safety analysis report) and were reviewed

and approved by the NRC. Most safety systems are designed to rely on automatic system actuation

to ensure that the safety systems are capable of carrying out their intended functions. In a few

cases, limited operator actions, when appropriately justified, were approved. Proposed changes

that substitute manual action for automatic system actuation or that modify existing operator

actions, including operator response times, that were not reviewed and approved during the

original licensing review of the plant may raise the issue of an unreviewed safety question (USQ). 

Such changes must be evaluated under the criteria of 10 CFR 50.59 to determine whether a USQ is

involved and whether NRC’s review and approval are required before implementation... In the

NRC staff's experience, many of the changes involving operator actions proposed by licensees do

involve a USQ. (p. 3)

While it is recognized that to 10 CFR 50.59 has been updated to remove the USQ wording, the intent of

IN 97-78 is still pertinent. That is many changes to operator actions will still need to be submitted to

NRC for review and approval in accordance with the revised 10 CFR 50.59

A definition of the term “safety-related operator action” (SROA) is provided in (American National

Standard Institute/American Nuclear Society) ANSI/ANS-58.8-1994: 

A manual action required by plant emergency procedures that is necessary to cause a safety-related

system to perform its safety-related function during the course of any design basis accident (DBE). 

The successful performance of a safety-related operator action might require that discrete

manipulations be performed in a specific order. (p.4)

The guidance presented in this document can be used to address all SROAs, as well as other required

operator actions.

The present document proposes the use of a graded, risk-informed approach in conformance with

Regulatory Guide (RG) 1.174 (NRC, 1998) and provides guidance for reviewing the human performance

aspects of changes to plant systems and operations. The guidance uses risk insights to determine the

level of regulatory review the staff should perform. Human actions (HAs) that are considered more risk

significant receive a detailed review, while those of less risk significance receive a less detailed review

commensurate with their risk. In this document the term HA and operator action are used synonymously

because most of the type of actions addressed are performed by operations staff. 

The evaluation method uses a two-phase approach. The first phase is a three-step screening process to

locate the plant modification and its associated HAs in risk space using guidance similar to that of RG

1.174. The first two steps of the risk screening process are quantitative while the third step is qualitative. 

Essentially, plant modifications and their associated HAs are categorized into regions of high, medium,

and lower risk. This categorization is used to determine the level of graded human factors engineering

(HFE) review needed. 
1 INTRODUCTION

2

In the second phase, the HFE review, the HAs are reviewed. The intent of this phase is to ensure the

proposed HA can be reliably performed when needed. The details of the review are commensurate with

the risk. Three levels of NRC review are presented. A Region I review is used for HAs placed into the

high-risk category (see Section 3 of this document). It examines the licensee’s planning, analysis, design

activities, and verification and validation, as related to the change. The review criteria are based on an

adaptation of existing NRC review guidance for HFE, as found in: NUREG-0800 (NRC, 1996a),

NUREG-0711, Rev. 1 (NRC, 2002a), NUREG-0700, Rev. 2, (NRC, 2002b), and IN 97-78 (NRC, 1998).

The adaptation is based on a consideration of the types of cases for which this guidance will be used. 

This was accomplished by an analysis of past cases reviewed by NRC (Higgins, et al., 1999). While HAs

in the high-risk area of Region I are generally not desired, there are certainly examples of such actions in

plants today, such as the pressurized water reactor (PWR) emergency core cooling system (ECCS)

switchover. Also, there may be extenuating circumstances in which the licensee can adequately justify a

modification to add a Region I HA, e.g., if the change is temporary or if there are other changes that

lower the core damage frequency (CDF). Another important consideration is how well the licensee has

addressed the HFE aspects of the modification.

HAs in the medium risk category receive a Region II review by the NRC. While the guidance addresses

the same topical areas as the Region I review, the extent of the staff review is notably less (see Section 4

of this document). 

Finally, the third region is called lower risk to indicate that the modification involves less risk than those

in the high or medium regions. For HAs in the lower risk category (Region III), staff review, generally, 

would be limited to verification of the technical basis that the action is, in fact, in Region III. Such a

verification can be accomplished by reviewing the licensee’s analysis methods and risk results that show

the placement of the action in that risk region. Typically, no human factors review is necessary. 

However, NRC may specify a few review areas based on the results of Step 3 of the risk screening

process.

In keeping with RG 1.174, this guidance does not preclude the licensee from using other approaches for

justifying changes to a plant’s licensing basis or other approaches for requesting changes in HAs. 

Rather, this review approach is intended to improve consistency in regulatory decisions in areas where

the results of risk analyses are used to help justify regulatory action. RG 1.174 notes that risk-informed

principles, process, and approach provide useful guidance for the application of risk information to a

broader set of activities than plant-specific changes to a plant's licensing basis. This document was

developed within the spirit of such applications.

The RG notes that the use of probabilistic risk assessment (PRA) technology should be increased in all

regulatory matters to the extent supported by the state-of-the-art in PRA methods and data. Its application

should complement the NRC's deterministic approach and support the NRC's traditional defense-in-depth

philosophy. The NRC’s review of HAs also takes this concept into consideration.

RG 1.174 notes that decisions concerning proposed changes are expected to be reached in an integrated

fashion, considering traditional engineering and risk information. They may be based on qualitative

factors as well as quantitative analyses and information. Thus, the approach presented herein also
1 INTRODUCTION

3

considers such qualitative factors, both in Step 3 of the risk screening and in the final decision on

acceptance of human actions.

The Commission also noted on many occasions that the regulatory process should become “riskinformed” as opposed to “risk-based” (Thadani, 1998, p.1). Thus, the approaches described here retain

some deterministic aspects, for example dealing with defense-in-depth, meeting existing regulatory

requirements, and addressing the HFE aspects of the HAs.

This guidance is expected to contribute to satisfying the NRC’s goals of (1) maintaining safety, 

(2) increasing public confidence, (3) increasing regulatory efficiency and effectiveness, and (4) reducing

unnecessary regulatory burden. By implementing the guidance presented in this document, the NRC will

improve the regulatory process in three areas: foremost, through safety decision-making enhanced by the

use of PRA insights; through more efficient use of agency resources; and through a reduction in

unnecessary burdens on licensees. The use of risk insights by licensees, in submittals that request

changes to HAs, will assist the staff in the disposition of such licensee proposals.

5

2 RISK SCREENING PROCESS

2.1 Changes to Human Actions

Changes to HAs may result from a variety of plant activities such as: plant modifications, procedure

changes, equipment failures, justifications for continued operations (JCOs), and identified discrepancies

in equipment performance or safety analyses. The licensee should evaluate changes in these various

activities to determine their effect on HAs. The following types of changes to HAs may occur as a result

of these plant activities:

 New actions - an action that was not previously performed by personnel such as when an

action formerly performed by automation is allocated to the operators

 Modified actions - a change to the way actions were previously performed, such as

through the introduction of new task steps (e.g., due to new system components, a

modification to a component, or failed components), or the introduction of new control

and display devices for performing the action

 Modified task demands - rather than affecting the task steps themselves, a change in the

plant may affect the task demands, such as the amount of time available.

2.2 Overview of Screening Process

Any changes that affect the licensee’s (FSAR) will require the licensee to perform a safety analysis per

10 CFR 50.59. This evaluation may result in the identification of changes that require NRC review and

approval because they result in more than a minimal increase in risk, as defined by one of the eight

criteria in section (c) (2) of the latest version of 10 CFR 50.59, “Changes, tests and experiments.” The

present document provides guidance for the NRC review of changes to HAs that are part of modifications

that exceed the threshold criteria of 50.59 (c) (2) and are submitted to the NRC for approval. The

modification in question may involve only an HA change or may involve equipment as well as HA

changes.

The intent of the 50.59 process is to permit licensees to make changes to their facilities as described in

the FSAR, provided the changes maintain the level of safety documented in the original licensing basis,

such as the FSAR, as updated. Historically the process has been structured around the licensing

approach to design-basis events. The staff has recognized that the 50.59 process needed improvement to

become consistent with the Commission policy of risk-informed regulation (Thadani, 1998). Thus, the

NRC has formally modified the 50.59 process to incorporate risk insights. 

The rulemaking to revise the 50.59 requirements was published as a final rule, October 4, 1999 (64 FR

53582). Implementation guidance has also been provided in RG 1.187, “Guidance for Implementation of

10 CFR 50.59, Changes, Tests, and Experiments,” which endorses the Nuclear Energy Institute (NEI)

document, NEI 96-07, Rev. 1, “Guidelines for 10 CFR 50.59 Implementation” dated November 2000. 

The methods provided in this document are consistent with the intent of the revised 10 CFR 50.59 and

RG 1.187 and they combine risk-informed approaches with both qualitative and quantitative HFE review

methods. 
2 RISK SCREENING PROCESS

6

The risk screening of this section is a general risk-informed evaluation, which is performed first and then

may be followed, as appropriate, by the human factors evaluations of Section 3 and 4. RG 1.174 was

used to develop the risk-informed approach herein. A three step screening analysis is used to locate the

plant modification and its associated HAs in risk space using guidance similar to that of RG 1.174. The

first two steps of the process are quantitative while the third step is qualitative. Sections 2.3 and 2.4

below describe Step 1 of the screening process for the overall modification using RG 1.174. Section 2.5

describes Step 2 of the screening process for the human actions involved in the modification. Section 2.6

describes the qualitative Step 3 of the screening process. Figure 2.0 illustrates the overall risk screening

and HFE review process outlined in this document. 

Changes proposed by licensees may be permanent or temporary. This guidance addresses both cases. 

Temporary changes are addressed in Section 2.4. There are two ways to determine the risk importance of

HAs: through the use of the plant-specific PRA and through the use of generic information. Trial

applications of these methods have shown that plant-specific approaches are necessary to accurately

place the affected HAs in the risk regions of this approach. However, a conservative method of using

generic information is also discussed for use with non-risk-informed submittals by licensees.

The licensee should determine the overall risk importance of the entire proposed change in order to place

it on Figures 3 and 4 of RG 1.174 and to preliminarily determine the appropriate level of review. These

initially may be simplified or scoping risk calculations. Any scoping type analyses should be appropriate

to the modification or change in the HA involved to ensure that actual changes in risk are reflected in the

calculations. If the change is in Region II or Region III, no further detailed risk calculations may be

necessary. However, if the change is in Region I, then the PRA and (HRA) should be requantified per

Section 3.7 of the Region I review guidance to address the change. This requantification should

eventually account for all aspects of the change, including those that result from the Region I review. 

The general intent of the lowest risk region, Region III, is that it encompass HAs that involve no increase

in risk or only a minimal risk increase. As such, these HAs may not be submitted to NRC for review

under the current, updated 10 CFR 50.59. However, due to a variety of reasons (e.g. conservative

interpretation by licensees or the HA is part of a larger modification that has risk significance), the HA

may still be submitted to NRC for review. This screening process should then assign such an HA to

Region III, so that it does not receive an unwarranted review, that would expend resources better utilized

elsewhere.
2 RISK SCREENING PROCESS

7

N

Sec. 2.7 

Y

N 

Y

Sec. 2.3 & 2.4

Step 1

Y Y

N N 

Sec. 2.5

Step 2

Sec. 2.6

Step 3

Y Y 

N

N

N

Y

Sec. 3 & 4

Sec. 5

R-I

Submittal ?

Use Generic Method

PSA

Suitable?

Screen Mod. 

per RG 1.174

Reject?

Reject? Region I?

Region I?

Region II?

HA Only

Equip. & HA

RAW & FV Screen

for HA

Region I Review Region II Review Region III Review

Final Decision

Accept/Reject

Qualitative

Evaluation

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

(10)

(12)

(11)

(13)

(14) (15) (16)

(17)

Go to

Box 17

Go to

Box 17

Figure 2.0 Risk informed (R-I) screening & HFE review flow chart

(a discussion of this Figure appears on pp.8-9)
2 RISK SCREENING PROCESS

8

In accordance with RG 1.174, licensee submittals are not necessarily required to include risk information. 

If a licensee is requesting approval of a modification involving changes in HAs and does not wish to have

a risk-informed review, then the NRC must still decide what level of HFE review is necessary. The NRC

may decide the appropriate level of review solely on a deterministic basis. Alternatively, the NRC may

use generic risk information to make a conservative determination as to the appropriate level of review. 

This generic method is discussed near the end of this section and is summarized in Table 2.1. NRC

Regulatory Issue Summary 2000-07, "Use of Risk-informed Decisionmaking in License Amendment

Reviews" (NRC, 2000a), contains additional clarification for unusual situations.

Discussion of Figure 2.0

Box 1: A submittal from a licensee may or may not be risk-informed (R-I). If it is not R-I, then the NRC

may choose to use the Generic Method (Box 2) of Section 2.7 of this document to perform a screening of

the human action and to grade the significance of the review. The Generic Method will assign a

proposed Region (I, II or III) for the review, which is then passed through the qualitative screening 

(Box 10) to see if it needs to be adjusted. Alternatively, the NRC may choose to perform a deterministic

review.

Box 3: The NRC will perform a evaluation to determine whether the PSA information submitted as part

of the R-I submittal is suitable, using criteria defined in RG 1.174 and in Chapter 19 of the Standards

Review Plan (SRP). If it is not suitable, the reviewer can again use the Generic Method for determining

the proper level of review. If it is suitable, then proceed to Step 1 of the screening (Box 4).

Box 4: Step 1 of the screening is described in Section 2.3 and 2.4 of this document and evaluates the

entire proposed modification, including both equipment and HAs. If the modification is a permanent

change (Section 2.3), this evaluation uses RG 1.174 directly. If it is a temporary change, the evaluation

uses Section 2.4.

Box 5: RG 1.174 notes that licensee applications that lie in Region I are not normally permitted. If Step

1 places the entire modification in Region I, then a decision whether to reject it outright will need to be

made (Box 6). If it is rejected, then the reviewer proceeds no further with this Figure or with the

screening and evaluation methodology. If it is not rejected, then a determination is made as to whether

the modification contains only HAs (Box 7) or if it includes both equipment and HAs (Box 8). 

Box 7: If the modification contains only HAs, and is in Region I then, the reviewer proceeds directly to

perform the Region I HFE review (Box 14) as described in Section 3 of this document.

Box 8: If the modification contains both equipment and HAs, then the reviewer proceeds to Step 2 of the

screening method (Box 9), as described in Section 2.5 of this report.

Box 9: This is Step 2 of the screening method which evaluates the HA portion of the modification to

determine the appropriate R-I level of review. This is described in Section 2.5. This is done by

evaluating both the RAW and the FV risk importance measures. This step will preliminarily place the

HA in one of the three Regions. The reviewer then proceeds to the qualitative screen (Box 10).
2 RISK SCREENING PROCESS

9

Box 10: This is the Qualitative Screen that constitutes Step 3 of the screening method and is described in

Section 2.6 of this document. This step of the screening will also determine whether the HA is in Region

I (Box 11). 

Box 11: If the HA is in Region I, the reviewer proceeds to Box 12. If the HA is not in Region I, the

reviewer determines if it is in Region II (Box 13).

Box 12: In this case the HA is in Region I and as in Box 6 above, the reviewer again needs to determine

whether the modification (i.e., the change to the HA) is rejected outright per RG 1.174. If it is rejected,

then the reviewer proceeds no further with this Figure or with the screening and evaluation methodology. 

If it is not rejected, then the reviewer performs the Region I review (Box 14).

Box 13: This box checks if the HA is I Region II. If the HA is in Region II, the reviewer performs the

Region II review (Box 15) described in Section 4 of this document. If it is not in Region II (and in

Region III) then the reviewer performs the minimal Region III review (Box 16), described in Section 2.9

of this document. 

Boxes 14, 15, & 16: These boxes represent the HFE reviews performed for HAs in each of the three

risk-informed regions. The review for Region I is described in Section 3 of this document. The review

for Region II is described in Section 4 of this document. The Region III review is described in Section

2.9 of this document. This review may also include selected review criteria from the Region I or II

review method (contained in Sections 3 or 4 of the document).

Box 17: After the details of the HFE reviews are completed, a final decision as to the acceptability of the

proposed changes is made. Section 5 of this document provides guidance to assist in this final decision.

2.3 Step 1 - Change in Risk from Overall Modification (Permanent Change)

As part of the NRC review of a licensee’s risk submittal, the suitability of the PRA model to support a

risk-informed analysis is evaluated. That is, the reviewer confirms that the modifications have been

appropriately modeled in the PRA, including both equipment and HAs. Guidance for this evaluation is 

in NUREG-0800, Chapter 19 and in RG 1.174. If the PRA is found to be lacking, a few options are

available: the licensee could upgrade the modeling where necessary; the submittal could be withdrawn;

the submittal could be treated as non-risk-informed; and the NRC could use the generic to HAs in Section

2.7 below.

As noted previously, changes proposed by licensees may be permanent or temporary. Permanent changes

are discussed first in this section and then temporary changes are discussed in Section 2.4. For screening

purposes, all modifications should first be passed through this permanent change section. If a temporary

change has risk in Region II or III of the criteria here, then the reviewer may proceed to Step 2 of the

screening per Table 2.1. If the change in risk due to the temporary change is in Region I, then the

reviewer should proceed to the temporary section to evaluate the integrated risk over the time period the

change will be in place.

For permanent changes to the plant, the first step of screening uses RG 1.174 directly. Permanent

changes may include equipment only, human actions only, or a combination of equipment and HAs. 
2 RISK SCREENING PROCESS

10

Equipment-only changes, that have no impact on HAs, are not within the purview of this approach. 

Changes that involve HAs only, HAs plus equipment changes, or equipment changes that affect HAs

should receive an evaluation per RG 1.174 as Step 1 of this approach.

RG 1.174, Figure 3, Acceptance Guidelines for Core Damage Frequency (CDF) and Figure 4,

Acceptance Guidelines for Large Early Release Frequency (LERF) have three risk regions (Regions I, II,

& III), with the most risk significant being Region I. The figures are reproduced here for convenience

and illustration.

 

 

 10-5

 

10-6

Region I

Region II

Region III

 

Figure 2.1 Acceptance Guidelines for Core Damage Frequency (CDF) 

(from RG 1.174 Figure 3)

 C D F

 (Events / Rx Year)

10-4 10-5

Baseline CDF (Events/Rx Year)
2 RISK SCREENING PROCESS

11

 L E R F 



(Large Early Releases / Rx Year)

10-5 10-6

 Baseline LERF (Large Early Releases/Rx Year)

10-6

10-7

Region I

Region II

Region III

Figure 2.2 Acceptance Guidelines for Large Early Release Frequency (LERF)

(from RG 1.174 Figure 4)

The change in risk due to the modification (CDFmod) that includes the new human action is defined as:

CDFmod = [new CDF (with modification in-place) - current baseline CDF]

where: CDFmod is the change in Core Damage Frequency due to the modification.

This value of CDFmod is placed in one of the three Regions of Figure 3 of RG 1.174, for Step 1 of the

screening method.

Similarly, the change in risk due to LERF is evaluated using Figure 4 of the RG. LERF is an important

consideration when the modification affects systems that mitigate offsite releases post-core-damage ,

such as the containment systems. There also may be differential effects on LERF when HAs affect core

damage sequences that contribute heavily to LERF. An experienced reviewer may be able to judge

whether the LERF evaluation is necessary or if the CDF evaluation alone will suffice. This is because

many changes will not affect LERF independently from CDF. Otherwise, per the RG, both CDF and

LERF are checked.

Based on the RG 1.174 evaluation, the modification is placed into one of the three risk regions. If it is in

Region I, the modification will likely be disapproved by NRC. This will be discussed in more detail

below. If the modification is in Region II or III, then the reviewer proceeds to Step 2 of the screening

process. This part of the process is summarized in Table 2.1 below.
2 RISK SCREENING PROCESS

12

Table 2.1 Action on Completion of Step 1

Step 1 Results NRC Review Action

Region I

(Equip. + HA)

- Change generally not permitted.

- If change not disapproved at Step 1, go to Step 2 of screening.

Region I

(HA only)

- Change generally not permitted.

- If change not disapproved at Step 1, perform Region I HF review.

Region II or III - Go to Step 2 of screening

Modification in Region I for Step 1

RG 1.174 notes that licensee applications that are in Region I are not normally permitted. There may,

however, be extenuating circumstances where they would be considered and approved. These could

include: 

 the change is only in place for a short time; 

 there are other compensating actions that lower risk; 

 there are other actions taken that improve safety but are not quantifiable using the PRA; 

 there are no better options available to the licensee; or 

 there is an overriding need to accomplish the actions.

If a Region I modification includes a combination of both equipment and HAs, then NRC may reject the

overall modification. If this happens, clearly no further screening is necessary. If such a modification is

not rejected at this stage, then the screening should proceed to Step 2 to evaluate the risk significance of

only the HA portion of the modification.

If a Region I modification includes only HAs, then NRC may likewise reject the modification. If the

change is not rejected, the reviewer proceeds directly to the Region I HFE review. Since the HA itself is

very risk significant, there is no need to perform Steps 2 & 3 of the screening, and NRC will review the

HA using the Region I HFE review guidance in Section 3 of this document.

Modification in Region II for Step 1

If the overall modification is in Region II, it is still possible that the HA may be placed in either Region I

or Region III. Therefore, it is necessary to perform Steps 2 and 3 of the screening process.

Modification in Region III for Step 1

Consider the case where a Region III modification includes a combination of both equipment and HAs. 

It is possible that there are equipment improvements that result in a decrease in CDF and that may be

masking a risk significant contribution of the HA. That is, even though the overall modification is not

risk significant, the HA may be when considered by itself. Therefore, the reviewer needs to perform

Steps 2 & 3 of the screening. 
2 RISK SCREENING PROCESS

13

If a Region III modification includes only HAs, Steps 2 & 3 of the screening method should still be

performed. This is because the Step 1 risk calculation is based on the base case value of human error

probability (HEP) for the HA. And, there may be the situation where a licensee is replacing a

demonstrated reliable automatic component with presumed reliable HA (with a low HEP). Step 2 will

evaluate this and other possibilities.

2.4 Step 1 - Change in Risk from Overall Modification (Temporary Change) 

Changes associated with HAs are often temporary, implemented to address equipment or analysis

problems until other, more permanent corrective actions can be planned and completed. Sometimes

temporary changes involve substituting HAs for automatic equipment that is temporarily inoperable and

cannot be restored within the time interval required by the plant technical specifications. For temporary

changes, the risk screening also considers the time interval that the modification will be in place and uses

Figures 2.3 and 2.4 in the first step of determining risk information and the level of HFE review. In this

fashion, the screening describes a method to quantitatively evaluate, in an integrated fashion, both the

increase in risk and the length of time of increased risk. 

The risk calculated by a PRA can be expressed in a variety of ways: as an instantaneous value (often

calculated for configuration risk management purposes), an average value of CDF over a reactor year

(the most common value that is cited), or a cumulative value of core damage frequency (CCDF)

computed over a defined time interval. The CCDF can be calculated accurately using statistical

techniques. A simplified method of viewing the cumulative or integrated risk is to multiply the CDF by

time. This gives reasonable results for the type of screening review the NRC is performing for riskimportant HAs. Thus, equations for integrated risk can be written as follows:

Integrated CDF Risk (mod) = CDFmod x time (mod) = ICCDP, or

Integrated LERF Risk (mod) = LERFmod x time (mod) = ICLERP,

where: Integrated Risk (mod) is the integrated risk due to the modification over the time that the change

or modification is to be in place, expressed as CDF or LERF;

time (mod) is the length of time that the change or modification is to be in place; and ICLERP is

incremental conditional large early release probability.

The value of Integrated CDF Risk (mod) can be roughly interpreted as the change in the expected core

damage events in the plant over the time period due to the modification. This concept of integrated risk

is also used in RG 1.177, where the Integrated CDF Risk is called the incremental conditional core

damage probability (ICCDP) and the Integrated LERF Risk (mod) is called the incremental conditional

large early release probability (ICLERP). Incremental in this situation refers to the incremental increase

in risk over the time period for the modification.

RG 1.174 is designed to address changes to the licensing basis of a plant and primarily addresses

permanent changes. As such, Figures 3 and 4 of the RG, that contain the acceptance guidelines for CDF

and LERF, do not explicitly address time. However, RG 1.177 utilizes the integrated risk measure

(ICCDP) similarly for evaluating the acceptability of integrated risk over periods of time that equipment

is out-of-service (due to the allowed outage time or AOT). This RG (see Section 2.4) uses an
2 RISK SCREENING PROCESS

14

acceptability limit of 5 E-7 events per Reactor-year for ICCDP, since that is considered to be a small risk

increase for a single Technical Specification AOT change. Therefore, this value is selected for the

boundary between Regions II and III. Correspondingly we use 5 E-6 events per reactor-year (an increase

of one order of magnitude) as the boundary between Regions I and II. Similarly for ICLERP, RG 1.177

uses 5 E-8 events per reactor-year for the limit on a small LERF increase. This value has been adopted as

well. Thus the two boundary values for integrated risk increase for LERF are 5 E-8 and 5E-7 events per

reactor-year. The resulting new figures are shown below as Figures 2.3 and 2.4. The regions in the

figures can be interpreted similarly to the three regions of the figures of RG 1.174, namely: Region I -

changes normally not permitted without extenuating circumstances; and Regions II and III - changes

permitted, but track cumulative impacts of multiple changes. In addition to screening, the integrated risk

information will also be useful in making the final decision on the implementation of a temporary

modification, as discussed in Section 5.

The above equations calculate the integrated risk due to the modification over time and the figures

contain screening guidelines for the integrated risk. The integrated risk due to the CDFmod and the

LERFmod should be plotted on Figures 2.3 and 2.4. If the modification affects LERF, then both figures

should be used and the most conservative region selected. If LERF is not affected by the change, then

Figure 2.3 will suffice. 

For larger values of integrated risk (e.g., in Region I), the reviewer should consider the potential

synergistic effect on risk of CDF spikes due to plant configuration changes together with the temporary

modification effects. For example, there may need to be temporary restrictions on configurations and

equipment out-of-service during the time period of the temporary modification.

The methods here may allow a larger value of risk increase, if the time that the modification will be in

place is relatively short. Conversely, longer periods of time for changes entail greater integrated risk. 

Similar to the section on permanent changes above, use Table 2.1 to determine the next step in the

screening and evaluation process.
2 RISK SCREENING PROCESS

15

Integrated Risk Increase or ICLERP (# of Large Early Releases)

 Integrated Risk Increase or ICCDP 



(# Core Damage Events)

5x10-6 5x10-7 5x10-7 5x10-8

Baseline CDF (Events/Rx Year)

Region I

Region II

Region III

 

Figure 2.3 Guidelines for Integrated Risk Increase -ICCDP

(Product of CDF and Time)

Region I

Region II

Region III

Figure 2.4 Guidelines for Integrated Risk Increase - ICLERP

(Product of LERF and Time)

10-4 10-5

10-5 10-6

Baseline LERF (Events/Rx Year)
2 RISK SCREENING PROCESS

16

2.5 Step 2 - Risk Evaluation of the Changed Human Action

This step is entered after completing Step 1 of the screening method and directed here by Table 2.1. It

addresses how to evaluate the importance of the HA, using two different, but complementary, risk

importance measures: the Risk Achievement Worth (RAW) of the HA and its Fussell-Vesely (FV) value. 

Both of these risk importance measures are first evaluated relative to the plant's new baseline CDF with

the proposed modification assumed to be in place. Next, if necessary, they are evaluated relative to the

plant’s new baseline LERF with the proposed modification assumed to be in place. 

The “new baseline CDF” is a shortened term for the “new CDF (with modification in-place)” used in the

previous section when defining the CDFmod . Similarly, the new baseline LERF is a shortened term for

the “new LERF (with modification in-place).” 

The RAW measures importance by computing the increase in CDF when the HA fails. That is, the HEP

of the HA is increased from its base case value to 1.0 and the overall CDF is re-computed. Then to

compute the RAW, one either takes a ratio or a difference of the new higher CDF to the baseline CDF.

The more common ratio method of expressing RAW issues . The RAW importance measure was defined

and discussed in NUREG/CR-3385 (Vesely, et al., 1983). One equation for the ratio value of the RAW

for HA “x” is:

RAW (x) = (CDF with x set to 1.0) / CDFnew BL

A high RAW value means that failure of the HA creates a risky situation. Thus, the HA’s reliability shall

be ensured through a thorough HFE review.

The FV importance measure represents a different way of expressing risk significance than RAW and is

included to obtain a more robust evaluation of risk importance. FV is the fraction of the total core

damage cutsets (or sequences) that contain the HA in question. This is expressed for HA “x” in the

following equation:

FV(x) = all CDF sequences containing x / CDFnew BL

If FV is high, the HA contributes to a relatively large portion of risk. Thus, for defense-in-depth

purposes, the HA’s reliability should be insured through a thorough HFE review.

A licensee may want to perform a one-time, plant-specific risk assessment to determine their risk

significant HAs, and to place them in the regions of the figures. Many licensees have already calculated

importance measures in their individual plant examinations (IPEs) and PRAs/PSAs. When a particular

modification affecting HAs is proposed, the licensees can perform a plant-specific and HA specific risk

evaluation for that modification to ensure proper placement on the Figures.
2 RISK SCREENING PROCESS

17

CDF Importance Measure Evaluation of HA

Figures 2.5 and 2.6 following show the regions for RAW and FV. RAW and FV values should be

computed for the HA being evaluated. These computations, together with the new baseline CDF, will

determine which of the three regions the HA is in. If an HA falls very close to the dividing line between

two regions, then the reviewer may want to use the qualitative criteria in Step 3 following to make the

final decision into which of those two regions to place the HA.
2 RISK SCREENING PROCESS

18

RAW vs New Baseline CDF

1

10

100

1000

1.E-06 1.E-05 1.E-04 1.E-03

New Baseline CDF

RAW

Region I

Region II

Region III

FV vs New Baseline CDF

0.001

0.01

0.1

1

1.E-06 1.E-05 1.E-04 1.E-03

New Baseline CDF

FV

Region I

Region II

Region III

Figure 2.5 RAW vs. New Baseline CDF

Figure 2.6 FV vs. Baseline CDF 
2 RISK SCREENING PROCESS

19

LERF Importance Measure Evaluation of HA

Next the reviewer determines if a separate LERF importance measure evaluation is necessary for the HA. 

In general, the default would be that it is not necessary to separately perform a LERF evaluation in this

step. This is because: 

 most HAs affect primarily CDF and one would not obtain a different risk review Region

with the LERF evaluation; 

 LERF importance measures are not routinely computed by the PSA programs, as the

CDF importance measures are; and 

 some PSA Level II models are not structured to support such LERF importance measure

calculations. If the evaluator judges that the HA in question does affect LERF

differently than CDF and that a LERF evaluation should be done here, then proceed as

follows.

The method will use a LERF RAW importance measure, designated as RAW (L) and a LERF FV

importance measure designated as FV (L). These measures are analagous to the RAW and FV for CDF

used above and are being applied by the industry for other regulatory purposes (see NEI 2001). They are

defined as follows:

RAW (L) (x) = (LERF with x set to 1.0) / LERFnew BL

FV (L) (x) = all LERF sequences containing x / LERFnew BL

Figures 2.7 and 2.8 show the risk regions for RAW (L) and FV (L). These were adapted from Figures

2.5 and 2.6 by adjusting the values of the baseline LERF on the x-axis by one order of magnitude to

account for the fact that LERF values and acceptance criteria are generally one order of magnitude less

than CDF values and acceptance criteria.

Figures 2.7 and 2.8 below show the regions for RAW (L) and FV (L). For the HA being evaluated, the

reviewer computes the RAW (L) and FV (L) values. This, together with the new baseline LERF, will

determine in which of the three regions the HA lies. If an HA falls very close to the dividing line

between two regions, then the reviewer may want to use the qualitative criteria in Step 3 below to make

the final decision into which of those two regions to place the HA.
2 RISK SCREENING PROCESS

20

1

10

100

1000

1.E-07 1.E-06 1.E-05 1.E-04

New Baseline LERF

RAW(L)

Region I

Region II

Region III

0.001

0.01

0.1

1

1.E-07 1.E-06 1.E-05 1.E-04

New Baseline LERF

FV(L)

Region I

Region II

Region III

Figure 2.7 RAW (L) vs. New Baseline LERF 

Figure 2.8 FV (L) vs. New Baseline LERF
2 RISK SCREENING PROCESS

21

After the regions for both RAW and FV are determined from Figures 2.5 and 2.6, (plus 2.7 and 2.8 if

used) the HAs should be placed in the most conservative or highest risk region of the two figures (or four

figures, if all were used). If the calculation and placement on the Figures has been performed by the

licensee, the results and placement on the figures should be submitted to the NRC for verification and

use. 

A particular plant modification may encompass more than one HA. Some of these multiple HAs may be

dependent. Also, the PRA modeling techniques for HAs vary, so that a given plant activity may be

modeled as one HA or several. The screening evaluation should consider all of the HAs involved with

the modification. Any dependent HAs should be aggregated together. That is, when computing the

RAW and FV, include all HAs or all aspects of the one HA. This will give the full importance of the

operator actions associated with the modification in question. Any HAs that are not dependent can be

evaluated separately using this screening process.

The next step is to perform the qualitative evaluation in Step 3. This will ensure that any safety

considerations not adequately addressed in the quantitative evaluations are taken into account. Step 3

may keep the region determined in Step 2, but it also may adjust the risk region either up or down.

2.6 Step 3 - Qualitative Evaluation

The qualitative evaluation can be conducted after Step 2 above or after completing the Generic Method

(Section 2.7). The purpose of this step is to ensure that various aspects of a change to HAs, that are not

readily quantifiable, are still appropriately considered when determining the level of HFE review to

perform. These considerations will allow the reviewer to adjust the region for HFE review either up or

down, as appropriate. 

It is also possible that an HA may be in Region III, but the reviewer may identify an aspect of the HA

change that should be reviewed. Even if this does not warrant raising the risk region to Region II, the

reviewer may supplement the small Region III review with a focused review of that particular aspect of

the HA.

The qualitative factors that are addressed here are: Amount of Change to HA, Analyses, Functions and

Tasks, Design Support for Task Performance, and Performance Shaping Factors.

 

1. Amount of Change to HA 

The reviewer evaluates the “amount” of the change to the HA. If the HA is an existing one that was in

the FSAR and the change to the HA portion of the modification is minimal, then the reviewer should

consider reducing the level of review for the HA. This may occur when a plant modification includes

changes to both equipment and HAs.

2. Analyses

The reviewer addresses the following topics as a basis for moving the HA to the next higher (or next

lower) risk region from that determined previously in Step 2.
2 RISK SCREENING PROCESS

22

 Changes in Defense-in-Depth - Has the plant modification resulted in a lessening of defensein-depth that was not captured in the PRA? If so, then move to the next higher risk region.

 Poor HRA analysis – Have the quantitative risk results been based on an HRA that does not

reasonably meet current standards for HA and error analysis (as per the ASME PRA standard

and Chapter 19 of the SRP)? Also, the reviewer should make a judgement about the

uncertainty of the HEP (is this a particularly uncertain calculation compared to typical PRA

modeled actions?). If the HEP is too high or too low due to uncertainty or poor modeling, this

will affect both the RAW and FV measures, but in opposite directions. Thus, Step 2 of the

method tends to be robust in preventing uncertainty in the HEP from affecting the assignment

of the risk region. However, it is possible that the RAW and FV regions may not be affected

by the same amount and that could cause a non-conservative placement in some cases. In these

situations, the reviewer should judge whether such HA modeling or uncertainty may have

notably lowered the risk importance of the HA in question or the CDF of the modification. If

so, then the HA could be increased by one risk region. Alternatively, the reviewer may consult

the generic action tables to determine whether the action is listed, and then use the risk region

indicated by the generic guidance.

Further, if there are judged to be dependent HAs that were not properly modeled in the HRA 

and if the reviewer was not able to adequately address them (as discussed in Step 2), then

consider increasing the risk region for review of the set of dependent HAs.

Another check on the reasonableness of an HRA is the number of HAs that would screen as

Region I. NRC may request RAW and FV values for all HAs in the PRA and note the total

number that fall into Region I. Some example distributions of HAs among the risk regions are

provided in Higgins, 2002.

 Compliance with NRC regulations - Does the proposed change meet current NRC regulations? 

If not, this may be a basis to reject the change, unless a specific exemption under 10 CFR 50.12

is requested. If there are reasons to approve the change despite this issue, the reviewer may

want to elevate the region of review.

3. Functions and Tasks

If the answer is "yes" to any of the questions below, there may be reason to elevate the human action to

the next higher risk region than determined in Step 2.

 Operating Experience – Does the plant modification adversely affect the performance of an

action that has previously been identified as problematic based on experience/events at that

plant or plants of similar design?

 Change in Automation - Has the plant modification given personnel a new function that they

previously did not have and which is different from their normal responsibilities? For

example, are operators now required to take an action in place of a previously automated

action? However, simply being required to open a valve that was previously automatically

operated and where the action required to open the valve is similar to other valve opening
2 RISK SCREENING PROCESS

23

operations the operators are familiar with would not be a sufficient change (in and of itself) to

warrant a "yes" to this question.

 Change in Tasks - Has the plant modification significantly changed the way in which personnel

perform their tasks making them more complex or has the time available to perform the action

been significantly reduced? In this case, personnel are not performing a new function; instead

the way that they perform their function has significantly changed and is different from what

they usually do.

4. Design Support for Task Performance

If the answer is "yes" to any of these questions, then the reviewer should consider at a minimum a Region

II HFE review to address the affected changes in task support.

 Change in HSIs – Has the plant modification resulted in significant changes to the HSIs used

by personnel to perform the task? For example, are personnel now performing their tasks at a

computer terminal where previously the tasks were performed at a control board with analog

displays and controls?

 Change in Procedures – Has the plant modification resulted in significant changes to the

procedures used by personnel to perform the task or is the task not supported by procedures? 

 Change in Training – Has the plant modification resulted in significant changes to the training

or is the task not addressed in training? 

5. Performance Shaping Factors (PSFs)

If the answer is "yes" to any of these questions, then the reviewer should consider increasing the risk

region if (1) the results of the quantitative analysis place the action near the next risk level or (2) if the

PSF concerns are found in combination with the factors discussed above.

 Changes in Teamwork - Has the plant modification significantly changed the team aspects of

performing an action. For example, (1) is one operator now performing the tasks that two or

more operators performed in the past, (2) is it now more difficult to coordinate the actions of

individual crew members, or (3) is task performance more difficult to supervise following the

modification.

 Changes in Skill Level of Individuals Performing the Action - Has the plant modification kept

the same HA but made it necessary for the action to be taken by an individual who is less

trained and has lower qualifications than was the case prior to the modification.

 Change in Communication Demands - Has the plant modification significantly increased the

level of communication needed to perform the task? For example, an operator must now

communicate with other personnel to perform actions that previously could be performed at a

local panel containing all necessary HSIs.
2 RISK SCREENING PROCESS

24

 Change in Environmental Conditions - Has the plant modification significantly increased the

environmental challenges (such as radiation, noise, etc.) that could negatively affect task

performance? 

Based on the results of Step 3, the reviewer will determine a risk region (either Region I, II, or III) for

the HA. The results from Step 2 may have been adjusted by one region in this step. Once the risk region

is determined from Step 3, the reviewer proceeds to Section 2.9 for a discussion of which HFE review

criteria are to be used.

2.7 Generic Method

A Generic Method may be needed if the licensee has chosen not to submit risk information. An

approximation to the risk importance of the HA can be determined generically using Tables A.1 and A.2

in Attachment A, for boiling water reactors (BWRs) and PWRs respectively. These HAs were initially

identified and grouped based on the risk-informed assessment process (Azarm, Higgins, and Chu, 1999)

and from NUREG-1560. The grouping was then updated based on risk information from the latest

licensee PRAs obtained during the site benchmarking of the Risk Informed Inspection Notebooks.

The HAs are organized into two groups. Group 1 contains the most risk-important HAs. RAW

calculations on Group 1 HAs would typically place them in Region I of Figure 2.1. Group 2 HAs are

considered to be “potentially” risk-important. That is, they would appear in Region I for some, but not

all, plants. Typically, they affect risk, but not as significantly as the Group 1 actions. However, at some

plants they may be quite risk-important. They were included in the second section of the plant Risk

Informed Matrices (RIMs) as potentially important HAs.

These two groups of generic risk-important HAs can be used by the NRC and by licensees as a crosscheck on the results of the plant-specific calculations. They can also be used to assist the NRC reviewer

in determining an estimate of the risk importance of HAs associated with a modification, if the licensee

has chosen not to make a risk-informed submittal. This will assist the NRC reviewer in determining the

appropriate level of HFE review for such situations. A summary of the recommended placement of HA’s

into risk regions based on their locations in the generic groups is contained in Table 2.2. below.

The logic for this placement is discussed here. As noted previously, RAW calculations for Group 1 HAs

themselves will typically fall into Region I. However, minor changes to a HA may not significantly alter

the risk associated with the action. If so, the technical basis for this result should be carefully understood

and documented. If no risk submittal is made and the plant modification involves more than a minor

change to a Group 1 action, then the NRC reviewer should assume that it is a Region I change. 

Changes related to Group 2 actions typically fall into Regions I or II. Thus, if such a change is judged to

be in Region III, the reasons should also be documented. If no risk submittal is made and the plant

modification involves more than a minor change to a Group 2 action, then the NRC reviewer needs to

decide whether it merits a Region I or Region II review. The conservative approach is to preliminarily

assign it to Region I and then proceed to the Step 3, Qualitative Evaluation. If the reviewers have

additional information about the risk associated with the action, either from the licensee or from the NRC

risk staff, they may choose to preliminarily assign the Group 2 HA to Risk Region II and then proceed to

the Step 3, Qualitative Evaluation.
2 RISK SCREENING PROCESS

25

It is important to note that, on a plant-specific basis, actions not listed in Tables A.1 and A.2 may also be

risk-significant, and can fall into either Region I or II. This is not common but, one cannot conclude that

if an action is not listed on either table, it is not important to risk. 

Thus, if no risk submittal is made and the plant modification involves an action that is not in Group 1 or

2, then an additional step is taken to determine whether the action involves risk-important systems for the

plant in question. The risk-important systems can be obtained from the plant’s IPE or latest updated

PSA. This information can also be extracted from the plant-specific risk-informed inspection notebooks

and related benchmarking reports that have been completed by the NRC. For example, systems that

benchmark as “Red” items should be considered to have high risk importance. Systems that benchmark

as “Yellow” or “White” should be considered as of moderate risk importance. Systems that are

benchmarked as “Green” would have lower risk importance.

If the action involves a high risk importance system, and there are more than minor changes involved,

then the HA is considered most likely in risk Region I or II. The same logic as discussed for Group II

HAs above applies and the reviewer should preliminarily select risk Region I or II and then proceed to

Step 3, Qualitative Evaluation. Similarly, if the HA involves a system of moderate importance, the HA

should be considered in Region II. If the modification involves only systems with lower risk-importance,

it is considered as a Region III HA. 

The logic applied to HAs in the Generic Method is conservative and will likely place an HA in a higher

risk region than it would receive using plant-specific RAW calculations. Thus, it is beneficial for both

licensees and NRC staff to obtain the plant-specific risk information to more properly allocate review

resources. 

Table 2.2 Preliminary Placement of HAs in Risk Regions 

for Submittals without Risk Information

Generic Groups 

that contain the HA

Systems involving the HA Risk Region 

to place the HA 

Group 1 NA I

Group 2 NA I/II

Neither Group High risk importance (Red) I/II

Neither Group Moderate risk importance

(Yellow or White)

II

Neither Group Lower risk importance (Green) III

After the HAs have been preliminarily placed into risk regions as per Table 2.2, the reviewer should

proceed to Step 3 of the method, Qualitative Evaluation (in Section 2.6). Step 3 can adjust the

preliminary placement into risk regions either up or down, depending on the qualitative factors evaluated.
2 RISK SCREENING PROCESS

26

2.8 Comparison of PRA Results to Acceptance Guidelines

This section provides guidance on comparing the results of the PRA risk calculations for Steps 1 and 2

with the risk guidelines that separate the Regions in Figures 2.1 through 2.6. In the context of integrated

decision-making, as discussed in Section 5, the guidelines should not be interpreted as being overly

prescriptive. They are intended to provide an indication, in numerical terms, of what is considered

acceptable. As such, the numerical values associated with defining the regions in the Figures are

approximate values that provide an indication of the changes that are generally acceptable. Furthermore,

the state-of-knowledge (or epistemic) type of uncertainties associated with PRA calculations, preclude a

definitive decision with respect to which region the application belongs, based purely on the numerical

results. 

The intent of comparing the PRA results with the acceptance guidelines is to demonstrate (with

reasonable assurance) that proposed increases in CDF or risk are generally small. This decision should

be based on a full understanding of the contributors to the PRA results and the impacts of the

uncertainties, both those that are explicitly accounted for in the results and those that are not. RG 1.174,

Section 2.2.5 contains a discussion of the various types of uncertainty that may need to be addressed. 

This is a somewhat subjective process, and the reasoning behind the decisions should be well

documented. Guidance on considerations is also contained in Section 2.2.5 of the RG. 

2.9 Level of HFE Review of the Changed Human Actions

Once the changes in risk and the risk importance associated with the changed HAs in question are placed

in the proper region of the risk figures, the level of HFE review to be performed is determined. The

review guidance is arranged into two levels so that the most risk significant changes (Region I) will

receive a more thorough review. The less risk significant changes (Region II) receive a more efficient

review appropriate to their level of risk. HAs that fall into Region III will receive no (or minimal)

review. Region III HAs are reviewed to verify that they have been properly classified in Region III using

this screening process. They are also reviewed to ensure that they meet current NRC regulations, and to

address any additional specific aspects of the HA that may have been identified as necessary based upon

insights from the qualitative screening in Step 3 above.

Based on the licensee’s 10 CFR 50.59 analysis, if the modification affecting the HA meets any of the

eight criteria of 50.59 (c) (2), then it is submitted to the NRC for review and approval. Licensees may

use the screening techniques provided in this document to assist them in their 50.59 screening and their

submittal to the NRC. The NRC reviewer uses the results of previous Steps 1, 2, and 3 of 10 CFR 50.59

to place the HA into the risk regions, which determines the level of required review (see Table 2.3).

Region I - Using the risk-informed approach, a proposed change in this region would generally

not be permitted. However, there may be extenuating circumstances in which the licensee

justifies the modification, e.g., if the change is temporary and avoids other more serious

problems; or there are other corresponding changes that lower the CDF. Review of an HA in

this Region requires more substantial review by NRC than HAs in the other regions. 

Therefore, these reviews would use the more detailed Region I guidance, in Section 3, which

includes a review of planning, analyses, design, and verification and validation activities, and a

performance monitoring strategy. 
2 RISK SCREENING PROCESS

27

Region II - Changes in this region require a less detailed review. The guidance is contained in

Section 4. In special circumstances, identified in Step 3, NRC may choose to add selected

Region I review criteria to a Region II review, rather than elevating the HA to a full Region I

review.

Region III - The licensee should document and the NRC may verify that the changes in risk

associated with HA are correctly located in Region III. This is done using the screening

process of this section. The NRC may also verify that current regulations are still being met

with the change in place (per Criterion 1 of Section 3.1, "General Deterministic Review

Criteria "). Based on the location in Region III, the modification would be accepted based on

the low risk, without NRC review of its HFE aspects. There may have been one or two

aspects, identified during the Step 3 qualitative screening, that the NRC has chosen to review. 

NRC would use the appropriate portions of the Region II review guidance for these identified

aspects. Licensees should be encouraged to utilize the Region II guidance contained in Section

4 to ensure that the HAs can be accomplished as assumed. Note that even though these HAs

may have met the 10 CFR 50.59 requirements for submittal to NRC, verification of their low

risk by the NRC permits acceptance without a detailed NRC HFE review. 

Table 2.3 Levels of Review for Human Actions

Risk Significance of HA NRC Review Actions

Region I

Change generally not permitted

Licensee may want to make case due to extenuating circumstances, such as

a temporary modification

Requires the full Region I HFE review

Region II Region II HFE review

Region III

Change permitted without detailed NRC review 

Verify change is in Region III and meets current regulations

Address any added criteria from Step 3

Region II HFE review guidance is available for licensee use

29

3 REGION I REVIEW GUIDANCE

The guidance in this section is a tailoring of NUREG-0711, Rev. 1 to plant modifications affecting HAs

of high risk-importance. NUREG-0711, Section 1.4, Graded Approach to Review, indicates that the

level of staff review of an applicant's HFE design should reflect the unique circumstances of the review

and that the guidance should be selectively applied to address the demands of each specific review.

The tailoring was accomplished by selecting the NUREG-0711 criteria that were appropriate to the

review of risk-important (Region I) HAs and then modifying these criteria to better reflect the context of

the types of plant modifications involved. This guidance has been developed to "stand alone." That is,

aspects of the review criteria that were not changed are repeated in this section rather than referring the

reviewer back to NUREG-0711. This makes the guidance easier to use. 

Even with the tailored guidance provided in this section, the reviewer can further adapt the guidance to

meet the unique demands of particular reviews.

3.1 General Deterministic Review Criteria

Objective

The objective of this review is to provide adequate assurance that deterministic aspects of design, as

discussed in RG 1.174, have been appropriately considered by the licensee. Deterministic aspects

include: ensuring the change meets current regulations, and does not compromise defense-in-depth.

Scope

The deterministic review criteria apply to all modifications associated with Region I HAs.

Criteria

(1) The licensee should provide adequate assurance that the change meets current regulations,

except where specific exemptions are requested under 10 CFR 50.12 or 10 CFR 2.802. For

example, a change might be identified as risk significant when using a standard PRA to screen

for risk. However, an exemption might be granted under one or more of the following

regulations: 10 CFR 20, 10 CFR 50 Appendix A, Criterion 19, and10 CFR 50 Appendices C

through R. 

(2) The licensee should provide adequate assurance that the change does not compromise defensein-depth. Defense-in-depth is one of the fundamental principles upon which the plant was

designed and built. Defense-in-depth uses multiple means to accomplish safety functions and

to prevent the release of radioactive materials. Defense-in-depth is important in accounting for

uncertainties in equipment and human performance, and for ensuring some protection remains

even in the face of significant breakdowns in particular areas. Defense-in-depth may be

changed but overall should be maintained. Important aspects of defense-in-depth include:
3 REGION I REVIEW GUIDANCE

30

 A reasonable balance is preserved among prevention of core damage, prevention of

containment failure, and consequence mitigation.

 There is no over-reliance on programmatic activities to compensate for weaknesses in

plant design. This may be pertinent to changes in credited operator actions.

 System redundancy, independence, and diversity are preserved commensurate with the

expected frequency, consequences of challenges to the system, and uncertainties (e.g., no

risk outliers).

 Defenses against potential common cause failures are preserved, and the potential for the

introduction of new common cause failure mechanisms is assessed. Caution should be

exercised in crediting new operator actions to provide adequate assurance that the

possibility of significant common cause operator errors are not created.

 Independence of barriers is not degraded. 

 Defenses against human errors are preserved. One way to help ensure this for riskimportant HAs is to establish procedures for a second check or independent verification

that such important actions have been properly executed. 

 Safety margins often used in deterministic analyses to account for uncertainty and

provide an added margin to provide adequate assurance that the various limits or criteria

important to safety are not violated. Such safety margins are typically not related to

HAs, but the reviewer should take note to see if there are any that may apply to the

particular case under review. It is also possible to add a safety margin (if desired) to the

HA by requiring a demonstration that the action can be performed within some time

interval (or margin) that is less than the time required by the analysis.

3.2 Operating Experience Review

Objective

The objective of this review is to provide adequate assurance that the licensee has identified and

analyzed HFE-related problems and issues encountered previously in designs and human tasks that are

similar to the planned modification so that issues that could potentially hinder human performance can be

addressed.

Scope

The operating experience review (OER) encompasses all proposed changes to HAs and addresses the

operating histories of plant systems, HAs, procedures, and HSI technologies. The scope of the HSI

technology review can be graded as follows:

(1) If existing HSI components are to be used without modification and if they are currently used

for safety-related functions within the plant, then a review of the operating experience with

those HSI components is not necessary.
3 REGION I REVIEW GUIDANCE

31

(2) If existing HSI components are to be used without modification but they are not currently used

for safety-related functions then the operating experience with those HSI components should

be reviewed.

(3) If new HSI components are to be installed or the existing HSI is to be modified using HSI

technologies that have not been previously used in the plant for safety-related functions then

the operating experience with those HSI components should be reviewed.

Criteria

(1) Plant Systems - The licensee’s review should include information pertaining to the operation

and maintenance of the plant system prior to the change in the HAs.

(2) Human Actions - The licensee’s review should identify performance issues associated with

procedural guidance, training, and HAs for the system prior to the proposed change to the

actions, including the types of actions performed, the procedures available for those actions,

and the adequacy of those procedures. 

(3) HSI Technologies - The licensee’s review should identify human performance issues associated

with HSI technologies for the proposed changes in the HAs, if they are different from those

used successfully at their plant.

(4) Issues Identified by Plant Personnel -Interviews and surveys with personnel should be

conducted to determine operating experience related to the plant system before the change in

the HAs. Discussions of plant operations and HFE/HSI design should be limited to topics

relevant to the change in the HAs.

(5) Use for Design Input - Issues identified by the OER should be used for input to the design of

modifications to the HSI, procedures, and training, and tracked to provide assurance that they

are addressed.

3.3 Functional Requirements Analysis And Functional Allocation

(Note: If there are no changes in Functional Requirements or Functional Allocation from the current

plant design, this review element is not needed.)

Objective

The objective of this review is to provide adequate assurance that the licensee has:

(1) Defined any changes in the plant's safety functions (functional requirements analysis), and 

(2) Provided evidence that the allocation of functions between humans and automatic systems

provides an acceptable role for plant personnel; i.e., the allocations take advantage of human

strengths and avoid functions that would be negatively affected by human limitations

(functional allocation). 
3 REGION I REVIEW GUIDANCE

32

Scope

This review addresses all plant functions affected by the change in operator actions including changes to

the functions and to their allocation between personnel and automatic systems. The level of detail in the

functional requirements and allocation analyses may be graded by the reviewer based on: (1) the degree

of difference between the HAs before and after the change; (2) the extent to which difficulties occurred

in prior operations, as identified through the OER; and (3) the risk level associated with the change. The

following additional considerations apply:

(1) If new safety functions are introduced or existing ones changed, then reviews of both the

functional requirements analysis and function allocation analysis should be conducted. (This

situation is not likely to occur since it would involve a significant deviation from the design

basis that was originally approved by the NRC.)

(2) If the function allocation is changed, or if the risk level is well into Region I (as determined by

the PRA/HRA review criteria) then a review of the function allocation should be conducted. 

(Many cases will have changed function allocations. An example may be the reallocation of

responsibility from an automatic system to personnel for the initiation, on-going control, or

termination of a function.)

(3) If the function allocation is not changed then no function allocation analysis is needed and the

licensee should proceed with task analysis. (An example may be a manual action performed

for a safety-related function that is now required under a new scenario. That is, the function is

the same but the initiating circumstances are different.) 

Review Criteria

Functional Requirements

(1) New or changed safety functions should be described, including comparisons before and after

the proposed change. The set of plant system configurations or success paths that are

responsible for or capable of carrying out the safety function should be clearly defined and the

ones affected by the proposed changes in the HAs should be identified. This functional

decomposition should address:

• High-level functions [e.g., maintain reactor coolant system (RCS) integrity] and critical

safety functions (e.g., maintain RCS pressure control)

• Specific plant systems and components

 Technical basis for changes to functions
3 REGION I REVIEW GUIDANCE

33

Functional Allocation

(1) For the functional allocation analysis, a description should be provided for each of the highlevel functions allocated to the human as a result of the proposed change. The description

should include the following:

• Purpose of the high-level function

• Conditions under which the high-level function is required

• Parameters that indicate that the high-level function is available 

• Parameters that indicate the high-level function is operating (e.g., flow indication)

• Parameters that indicate the high-level function is achieving its purpose (e.g., reactor

vessel level returning to normal)

• Parameters that indicate that operation of the high-level function can or should be

terminated

Note that parameters may be described qualitatively (e.g., high or low), rather than as specific

numerical values or setpoints.

(2) The technical basis for all relevant functional allocations should be documented. The basis for

function allocations can be successful operating experience. This analysis should reflect (a)

sensitivity, precision, time, and safety-related requirements; (b) required reliability; and (c) the

number and level of skills of personnel required to operate and maintain the system.

(3) The allocation analysis should consider not only the personnel role of initiating manual actions

but also responsibilities concerning automatic functions, including monitoring the status of

automatic functions to detect system failures. The demands associated with the proposed

allocation of functions should be considered in terms of all other human functions that may

impose concurrent demands upon the personnel. The overall level of workload should be

considered when allocating functions to the personnel. 

3.4 Task Analysis

Objective

 The objective of this review is to provide adequate assurance that the licensee's task analysis

(TA) identifies the behavioral requirements of the tasks personnel are required to perform. The

task analysis should form the basis for specifying the requirements for the HSI, procedures, and

training based on the tasks personnel will perform. The results are also used as basic

information for developing staffing and communication requirements of the plant.

 For a change to an existing action, a new TA may not be necessary.
3 REGION I REVIEW GUIDANCE

34

Scope

The task analysis addresses HAs in their entirety, including all pertinent plant conditions, situational

factors, and performance shaping factors. 

Criteria

(1) The licensee should identify the information that is required to inform personnel that each HA

is necessary, that the HA has been correctly performed, and that the HA can be terminated.

(2) Plant personnel who are affected by the HAs should be identified, including licensed control

room operators as defined in 10 CFR Part 55 and the following categories of personnel defined

by 10 CFR 50.120: nonlicensed operators, shift supervisor, shift technical advisor, instrument

and control technician, electrical maintenance personnel, mechanical maintenance personnel,

radiological protection technician, chemistry technician, and engineering support personnel.

(3) Task analyses should provide detailed descriptions of what the personnel must do. The

licensee should identify how human tasks or performance requirements are being changed. 

(4) The task analysis should address the full range of plant conditions and situational factors, and

performance shaping factors anticipated to influence human performance. The range of plant

operating modes relevant to the HAs (e.g., abnormal and emergency operations, transient

conditions, and low-power and shutdown conditions) should be included in the task analysis. 

(5) The human task requirements that result from the changes in the actions should be assessed to

determine whether they are compatible with each individual’s responsibilities (i.e., will not

interfere with or be disrupted by the cognitive and physical demands of other tasks and

responsibilities). 

(6) The task analysis should identify reasonable or credible, potential errors.

3.5 Staffing

Objective

The objective of this review is to provide adequate assurance that the licensee has analyzed the proposed

change in HAs to determine the number and qualifications of personnel based on task requirements and

applicable regulatory requirements. Adding additional manual actions or shifting tasks to periods of high

workload may increase staffing requirements. An example is a local manual action to temporarily

replace an automatic action.

Scope

The staffing analysis addresses personnel requirements for all conditions in which the HA may be

performed. 
3 REGION I REVIEW GUIDANCE

35

Criteria

(1) Staffing levels should be evaluated to determine their adequacy with respect to any additional

burden that may be imposed by the plant or HA modifications. The staffing levels should be

adjusted if necessary. The evaluation should be based on an analysis of:

• Current nominal (typical shift complement of personnel) and minimal staffing levels (as

identified in administrative procedures)

• Required actions determined from the task analysis, if performed

• The physical configuration of the work environment (e.g., control room and control

consoles configurations that may affect the ability of personnel to work together)

• The availability of plant information from individual workstations from individual and

group view components of the HSI

• Availability of personnel considering other activities that may be ongoing and for other

possible responsibilities outside the control room (e.g., fire brigade)

3.6 Probabilistic Risk and Human Reliability Analysis

Objective

The objectives of this review are to provide adequate assurance that the licensee has: (1) updated the

PRA model to reflect system, component, and HA changes that may be necessary based on the proposed

modification or HAs; (2) performed an analysis of the potential effects of the proposed changes on plant

safety and reliability, in a manner consistent with current, accepted PRA/HRA principles and practices,

and (3) ensured that the risk insights derived from the results are addressed in the selection of HAs;

development of procedures, HSI components, and training in order to limit risk and the likelihood of

personnel error and to provide for error detection and recovery capability. 

Scope

This review addresses PRAs and HRAs conducted by the licensee to evaluate changes in systems,

components, and human tasks that result from the proposed changes in HAs. Some of these items may

have been addressed and reviewed as part of the screening process for the risk-informed submital. In

addition, the NRC human factors engineering reviewers may consult with the NRC risk analysis

specialist on this review.

Criteria

(1) The PRA and HRA should be modified to reflect the changes in systems, components, and

human tasks. Human interactions with plant systems and components should be analyzed at
3 REGION I REVIEW GUIDANCE

36

least at the level modeled in the plant’s current PRA. Alternatively, justification should be

provided as to why the change to the PRA is minimal and not necessary.

(2) The HRA should follow a structured, systematic, and auditable process to provide adequate

assurance that the reliability of each HA is accurately estimated so that its effect on plant

safety using the PRA can be assessed.

(3) The PRA/HRA should address any human interactions that may be involved with the modified

plant systems and components at the level currently modeled in the plant PRA, for example,

 Errors of omission and commission

 Miscalibration and component restoration errors

 Recovery actions

(4) The analysis of HAs should include the identification of performance shaping factors (PSFs),

that is, factors that influence human reliability through their effects on performance. PSFs

include factors such as environmental conditions, HSI design, procedures, training, and

supervision. 

(5) Risk-important HAs associated with the modification should be identified from the PRA/HRA

and used as input to the design of procedures, HSI components, and training. These actions

should be developed from the Level 1 (core damage) PRA and Level 2 (release from

containment) PRA including both internal and external events. They should be developed

using selected (more than one) importance measures and HRA sensitivity analyses to provide

adequate assurance that an important action is not overlooked because of the selection of the

measure or the use of a particular assumption in the analysis.

(6) The licensee should use the information from the modified PRA/HRA to calculate changes in

CDF, LERF, and integrated risk (if a temporary change is involved). 

3.7 Human-System Interface Design

Objective

The objective of this review is to evaluate the HSI design, for those changes in HAs that require changes

to the HSI, to provide adequate assurance that the licensee has appropriately translated function and task

requirements into the detailed design of the HSI through the systematic application of HFE principles and

criteria. 

Scope

This review addresses the design of temporary and permanent modifications to the HSI, including new

HSI components and the modification of existing ones, related to the proposed changes in the HAs, to
3 REGION I REVIEW GUIDANCE

37

ensure that the existing HSI are appropriate for the modified human action. The review addresses

aspects of the HSI and the work environment that affect the ability of the personnel to perform the HAs. 

Criteria

(1) The HSI should be designed consistent with HFE guidelines and the existing HSI to the degree

practical.

(2) The design should seek to minimize the probability that errors will occur and maximize the

probability that errors will be detected and personnel will be able to recovered from them.

(3) When developing HSI components for actions performed either in the control room or locally

in the plant, the following factors should be considered:

 Communication, coordination, and workload 

 Feedback

 Local environment

 Inspection, test, and maintenance

(4) The layout of HSI components within consoles, panels, and workstations should be based upon

(1) analyses of human roles (job analysis) and (2) systematic strategies for organization such as

arrangement by importance, frequency of use, and sequence of use.

(5) HSI characteristics for the changed action should support human performance under the full

range of environmental conditions.

(6) Certain human tasks will need qualified instrumentation in accordance with RG 1.97 (NRC,

1983). The task analysis should identify the necessary safety grade of the control and display

equipment used for human tasks. The RG defines Type A variables as “those variables to be

monitored that provide the primary information required to permit the control room operators

to take the specified manually controlled actions for which no automatic control is provided

and that are required for safety systems to accomplish their safety function for design basis

accident events” (NRC, 1983, p. 1.87-4). Primary information is further defined in the RG as

information that is essential for the direct accomplishment of the specified safety functions, but

does not include those variables that are associated with contingency actions that may also be

identified in written procedures. Table 1 of RG 1.97 provides detailed Category 1 criteria that

Type A variables should meet. In general, these Category 1 criteria provide for environmental

and seismic qualification, redundancy, quality assurance, continuous display, good human

factors design, and an emergency power supply. Therefore, HAs, which are required for safety

systems to accomplish their safety function for design basis accident events and for which no

automatic control is provided, will need control and display instrumentation in accordance with

RG 1.97. (This RG allows for consideration of alternative approaches that are adequately

justified and include consideration of the risk significance of the actions involved.) Thus,
3 REGION I REVIEW GUIDANCE

38

credit should only be given for these types of HAs if they can be completed using control and

display instrumentation that is consistent with RG 1.97.

3.8 Procedure Design

Objective

The objective of this review is to provide adequate assurance that applicable plant procedures have been

appropriately modified, where needed, to provide adequate guidance for the successful completion of the

HAs, and that the procedures adequately reflect changes in plant equipment and HAs. In the procedure

development process, HFE principles and criteria should be applied along with all other design

requirements to develop procedure modifications that are technically accurate, comprehensive, explicit,

easy to use, and validated. 

Scope

This review addresses all plant procedures that provide guidance to personnel for the affected actions,

including the following types:

 Emergency operating procedures (EOPs)

 Plant and system operations (including startup, power, and shutdown operations)

 Abnormal and emergency operations

 Alarm response

The scope includes both temporary and permanent modifications to these procedures.

Criteria

(1) Where applicable, plant procedures should be modified to provide new instructions for the

proposed changes in the HAs. Exceptions may be made where the adequacy of the existing

procedures can be justified. Such a justification should indicate how the existing procedures

provide necessary and sufficient guidance for the changed HAs and do not contain information

that is inaccurate or no longer relevant.

(2) Where appropriate, procedures should identify how the operating crew should independently

verify that the HAs have been successfully performed.

(3) All procedures should be verified and validated to provide adequate assurance that they are

correct and can be carried out. Their final validation should be performed as part of the

validation activities described in Section 3.11.

(4) Any changes in the HSI should be reflected in the modifications of the procedures.
3 REGION I REVIEW GUIDANCE

39

(5) Procedural modifications should be integrated across the full set of procedures; alterations in

particular parts of the procedures should not conflict nor be inconsistent with other parts. For

example, an HSI component that is modified for a HA may also affect other actions that have

not been modified. Therefore, procedure changes should not be limited to only the changed

HAs.

3.9 Training Program Design

Objective

The objective of this review is to provide adequate assurance that the licensee’s training program results

in adequate training for the HAs. The review should provide adequate assurance that appropriate training

has been developed and conducted for the HAs, including any changes in qualifications, as described in

NRC Information Notice 97-78 (NRC, 1997). 

Scope

This review addresses the licensee’s training programs for all licensed and non-licensed personnel who

perform the changed HAs. The scope includes both temporary and permanent modifications to training

programs.

Criteria

(1) The licensee’s training program should be modified to address the knowledge and skill

requirements for all changes in HAs for the licensed and non-licensed personnel. 

(2) Learning objectives should be derived from an analysis that describes desired performance for

the HAs after training has been completed. 

3.10 Human Factors Verification and Validation

Objective

The objective of this review is to provide adequate assurance that the licensee’s verification and

validation (V&V) program:

• Provides adequate assurance that the HFE/HSI design contains all necessary alarms,

displays, and controls to support plant personnel tasks (HSI Task Support Verification).

• Provides adequate assurance that the HFE/HSI design conforms to HFE principles,

guidelines, and standards (HFE Design Verification).

• Provides adequate assurance that the HFE/HSI design can be effectively operated by

personnel within all performance requirements applicable to the HAs, including the

following Integrated System Validation):
3 REGION I REVIEW GUIDANCE

40

 All pertinent staffing considerations are acceptable for nominal and minimal

shift levels, such as shift staffing, assignment of tasks to crew members, and

crew coordination within the control room and between the control room and

local control stations and support centers.

 The HAs can be accomplished within time and performance criteria

 The integrated system performance is consistent with all functional

requirements, including tolerance of failures of individual HSI features

Scope

(1) The general scope of V&V includes the following factors as applicable to the proposed

changes to the HAs:

 HSI hardware and software

 Procedures

 Workstation and console configurations

(2) The typical order of V&V activities is:

 HSI task support verification

 HFE design verification

 Integrated system validation

(3) All V&V activities are applicable regardless of whether the change in HAs involve changes in

the HSI.

Criteria

HSI Task Support Verification

(1) All aspects of the HSI (e.g., controls, displays, procedures, and data processing) that are

required to accomplish the HAs should be verified as available through the HSI. For HAs that

require qualified instrumentation in accordance with RG 1.97, it should be verified that the HSI

provides such qualified instrumentation.

HFE Design Verification

(1) All aspects of the HSI (e.g., controls, displays, procedures, and data processing) used for the

HAs should be verified as consistent with accepted HFE guidelines, standards, and principles.
3 REGION I REVIEW GUIDANCE

41

(2) Deviations from accepted HFE guidelines, standards, and principles should be acceptably

justified on the basis of a documented rationale such as trade study results, literature-based

evaluations, demonstrated operational experience, or tests and experiments.

Integrated System Validation

Validation Testbeds

(1) For HAs performed in the main control room, the plant training simulator should be used as the

testbed when conducting the validation tests.

(2) For HAs performed at locations outside of the main control room, the use of a simulation or

mockup can be used or drills conducted in the plant. The conduct of these drills should not

interfere with plant operations (e.g., drills may be conducted when the plant is shutdown or the

affected systems are removed from service).

Plant Personnel

(1) Participants in the validation tests should be the plant personnel who will perform the changed

actions. Actions that will be performed by licensed personnel should be validated using

licensed personnel rather than training or engineering personnel. Similarly, actions allocated

to non-licensed personnel should be validated using non-licensed personnel.

(2) To properly account for human variability, more than one crew should participate in the

validation tests. This will help provide adequate assurance that variation along most of the

significant dimensions that influence human performance are included in the validation tests. 

Participation is not necessary for personnel who do not normally operate or maintain the plant

(e.g., administrative personnel who hold operating licenses).

(3) In selection of personnel, consideration should be given to the assembly of nominal and

minimum crew configurations, including shift supervisors, reactor operators, shift technical

advisors, etc., that will participate in the validation tests. The composition of operations

personnel need only include categories of personnel that are relevant to the HAs.

Operational Conditions

(1) Integrated system validation should consider the operational conditions for which each HA is

required. 

(2) The operational conditions should be developed into scenarios. The following information

should be defined to provide adequate assurance that important performance dimensions are

addressed and to allow scenarios to be accurately presented for repeated trials:

 Description of the scenario mission and any pertinent "prior history"

 Specific initial conditions
3 REGION I REVIEW GUIDANCE

42

 Events (e.g., failures) to occur and their initiating conditions, e.g., time, parameter

values, or events

 Data to be collected and the specification of what, when and how data are to be obtained

and stored

 Specific criteria for terminating the scenario

(4) Scenarios should have appropriate task fidelity so that realistic task performance will be

observed in the validation tests and so that results can be generalized to actual operation in the

real plant.

(5) When evaluating performance associated with the use of HSI components located remote from

the main control room, the effects on crew performance due to potentially harsh environments

(i.e., high radiation) should be simulated (i.e., additional time to don protective clothing and

access radiologically controlled areas).

Plant Performance Measurement

(1) The variables used in the performance measures should include performance of the plant and

personnel, as described below. 

(2) Measures that assess personnel task performance should be used, including the following: 

 For each specific scenario, the tasks that personnel are required to perform should be

identified and assessed. Such tasks can include necessary primary (e.g., start a pump) as

well as secondary (e.g., access the pump status display) tasks. This analysis should be

used for the identification of errors of omission by identifying tasks which should be

performed. The proper completion of required tasks should be verified.

 The tasks that are actually performed by personnel during simulated scenarios should be

identified and quantified. The variable(s) used to quantify tasks should be chosen to

reflect the important aspects of the task with respect to system performance, such as:

- Task success or failure

- Task completion time

- Errors (omission and commission)

- Subjective reports of participants

(3) Performance criteria for the measures used in the evaluations should be established. 
3 REGION I REVIEW GUIDANCE

43

Data Analysis and Interpretation

(1) Validation test data, time and errors, should be analyzed through a combination of quantitative

and qualitative methods. For example, task time can be statistically compared with time

available to perform the task and subjective reports of participants can be qualitatively

evaluated to identify potential obstacles to performance.

3.11 Human Performance Monitoring Strategy

Objective

The objective of this review is to provide adequate assurance that the licensee has prepared a human

performance monitoring strategy for ensuring that no adverse safety degradation occurs because of the

changes that are made and to provide adequate assurance that the conclusions that have been drawn from

the evaluation remain valid over time. A human performance monitoring strategy will help to ensure that

the confidence developed by the completion of the integrated system validation is maintained over time. 

There is no intent to periodically repeat the full integrated system validation, however, there should be

sufficient evidence to provide reasonable confidence that operators have maintained the skills necessary

to accomplish the assumed actions.

The results of the monitoring need not be reported to the NRC, but should be retained onsite for

inspection. 

Scope

The scope of the performance monitoring strategy should provide adequate assurance that the:

 HFE/HSI design can be effectively operated by personnel, both within the control room and

between the control room and local control stations and support centers.

 HAs can be accomplished within time and performance criteria.

 Integrated system performance is maintained within the performance established by the

integrated system validation. 

Criteria

(1) A human performance monitoring strategy should be developed and documented by the

licensee. The strategy should be capable of tracking human performance after the changes

have been implemented to demonstrate that performance is consistent with that assumed in the

various analyses that were conducted to justify the change. Licensees may integrate, or

coordinate, their performance monitoring for risk-informed changes with existing programs for

monitoring operator performance, such as the licensed operator training program. If a plant

change requires monitoring of actions that are not included in existing training programs, it

may be advantageous for a licensee to adjust the existing training program rather than to

develop additional monitoring programs for risk-informed purposes.
3 REGION I REVIEW GUIDANCE

44

(2) The program should be structured such that (1) HAs are monitored commensurate with their

safety importance, (2) feedback of information and corrective actions are accomplished in a

timely manner, and (3) degradation in performance can be detected and corrected before plant

safety is compromised (e.g., by use of the plant simulator during periodic training exercises).
45

4 REGION II REVIEW GUIDANCE

The guidance in this section also is a tailoring NUREG-0711, Rev. 1 to plant modifications affecting

risk-important HAs of medium risk significance. The guidance in this section reflects a further reduction

of the criteria to reflect the level of risk imposed by the modification in Region II. Even with the tailored

guidance provided in this section, the reviewer can further adapt the guidance to meet the unique

demands of particular reviews.

4.1 General Deterministic Review Criteria

Objective

The objective of this section is to provide adequate assurance that deterministic aspects of design, as

discussed in RG 1.174, have been appropriately considered by the licensee. Deterministic aspects

include: ensuring the change meets current regulations; and does not compromise defense-in-depth.

Scope

The deterministic review criteria are applicable to all modifications associated with Region II HAs.

Criteria

(1) The licensee should provide adequate assurance that the change meets current regulations,

except where specific exemptions are requested under 10 CFR 50.12 or 10 CFR 2.802. 

Examples of regulations that may be affected by a change, but that may be identified as risk

significant when using a standard PRA to screen for risk include the following: 10 CFR 20, 10

CFR 50 Appendix A, Criterion 19, and10 CFR 50 Appendices C through R.

(2) The licensee should provide adequate assurance that the change does not compromise defensein-depth. Defense-in-depth is one of the fundamental principles upon which the plant was

designed and built. Defense-in-depth uses multiple means to accomplish safety functions and

to prevent the release of radioactive materials. It is important in accounting for uncertainties in

equipment and human performance, and for ensuring some protection remains even in the face

of significant breakdowns in particular areas. Defense-in-depth may be changed but overall

should be maintained. Important aspects of defense-in-depth include:

 A reasonable balance is preserved among prevention of core damage, prevention of

containment failure, and consequence mitigation.

 There is no over-reliance on programmatic activities to compensate for weaknesses in

plant design.

 System redundancy, independence, and diversity are preserved commensurate with the

expected frequency, consequences of challenges to the system, and uncertainties (e.g., no

risk outliers).

 Defenses against potential common cause failures are preserved, and the potential for the

introduction of new common cause failure mechanisms is assessed.
4 REGION II REVIEW GUIDANCE

46

 Independence of barriers is not degraded. 

 Defenses against human errors are preserved.

4.2 Analysis

Objective

The objective of the review is to provide adequate assurance that the licensee has analyzed the changes to

HAs and identified HFE inputs for any modifications to the HSI, procedures, and training that may be

necessary.

Scope

The review criteria are applicable to all modifications associated with Region II HAs.

Criteria

(1) Functional and Task Analysis

 The licensee should identify how the personnel will know when the HA is necessary, that

is performed correctly, and when it can be terminated.

 Task analyses should provide a description of what the personnel must do. The licensee

should identify how human tasks or performance requirements are being changed. 

The task analysis should identify reasonable or credible, potential errors and their

consequences.

(2) Staffing - The effects of the changes in HAs upon the number and qualifications of current

staffing levels of operations personnel for normal and minimal staffing conditions.

4.3 Design of HSIs, Procedures, and Training

Objective

The objective of the review is to provide adequate assurance that the licensee has supported the HAs by

appropriate modifications to the HSI, procedures, and training.

Scope

The review criteria are applicable to all modifications associated with Region II HAs.
4 REGION II REVIEW GUIDANCE

47

Criteria

(1) HSIs - Temporary and permanent modifications to the HSI should be identified and described.

The modifications should be based on task requirements, HFE guidelines, and resolution of any

known operating experience issues.

(2) Procedures - Temporary and permanent modifications to plant procedures should be identified

and described. The modifications should be based on task requirements and resolution of any

known operating experience issues. Justification should be provided when the plant procedures

are not modified for changes in operator tasks.

(3) Training - Temporary and permanent modifications to the operator training program should be

identified and described. The modifications should be based on task requirements and

resolution of operating experience issues. Justification should be provided when the training

program is not modified for changes in operator tasks.

4.4 Human Action Verification

Objective

The objective of this review is to provide adequate assurance that the licensee has demonstrated that the

HAs can be successfully accomplished with the modified HSI, procedures, and training.

Scope

The review criteria are applicable to all modifications associated with Region II HAs.

Criteria

(1) An evaluation should be conducted at the actual HSI to determine that all required HSI

components, as identified by the task analysis, are available and accessible.

(2) A walk-through of the HAs under realistic conditions should be performed to determine that:

 The procedures are complete, technically accurate, and usable

 The training program appropriately addressed the changes in plant systems and HAs

 The HAs can be completed within the time criterion for each scenario that is applicable

to the HAs.

The scenario used should include any complicating factors that are expected to affect the crews

ability to perform the HAs. 

(3) The walk-throughs should include at least one crew of actual operators.

49

5 FINAL DECISION ON ACCEPTANCE OF HUMAN ACTIONS

Once the NRC review of a proposed change in HAs is completed, a final decision regarding the

acceptability of the modification must be made. At this point a significant amount of information has

been gathered, reviewed, and evaluated that can be used to assist in the final decision-making. This

information includes: 

 the risk values related to the change or modification, including their location on the acceptance

guideline figures

 the time associated with the change

 the results of the Region I or Region II review, which includes both HFE information relating to

the ability of operators to reliably perform the actions in question and deterministic aspects of

the proposed change

 answers to request for additional information (RAIs) that NRC has developed providing

additional information or commitments 

 other factors related to the plant in question that may bear on the decision

This information needs to be considered in an integrated fashion, that considers risk, but does not wholly

base the final decision on risk alone. RG 1.174 notes that the use of PRA technology should be increased

in all regulatory matters, but it should be done in a manner that complements the NRC's deterministic

approach and supports the NRC's traditional defense-in-depth philosophy. RG 1.174 also notes that

decisions concerning proposed changes are expected to be reached in an integrated fashion, considering

traditional engineering and risk information, and may be based on qualitative factors as well as

quantitative analyses and information. The review guidance in this document takes these concepts into

consideration.

RG 1.174 notes that HAs in the high-risk area of Region I are generally not desired, but there are

certainly examples of such actions in plants today, e.g., the PWR ECCS switchover situation described in

Generic Issue B-17. Also, there may be extenuating circumstances in which the licensee can adequately

justify a modification to add a Region I HA, e.g., if the change is temporary or if there are other changes

that lower the CDF.

Another important consideration is whether and how well the licensee has addressed the HFE aspects of

the modification. The results of the HFE analyses discussed in Sections 2, 3, and 4 must be considered in

an integrated manner. No individual analysis is sufficient in and of itself. Thus, the decision will not be

driven solely by the numerical results of the PRA. Each type of information helps in building an overall

picture of the implications of the proposed change on risk. The PRA has an important role in putting the

change into its proper context as it impacts the plant as a whole. As the discussions in the previous

section indicate, both quantitative and qualitative arguments may be brought to bear. The different

pieces of evidence that are used to make a final decision may not be combined in a formal way, but they

do need to be clearly documented. The proposed change should be given increased NRC management

attention when the calculated values of the changes in the risk metrics approach the criterion levels of

current, accepted guidelines. 
5 FINAL DECISION ON ACCEPTANCE OF HUMAN ACTIONS

50

The main factors in the decision process are discussed here first and then supplementary decision factors

are listed that may assist when the decision is difficult to make.

Main Decision Factors

 Change in CDF - One consideration is the value of CDFmod or the increase in Core Damage

Frequency due to the modification. The placement of this value into the regions of Figure 2.1

can also be considered. The confidence one has in the PRA HEP value and hence that the

change in CDF is at the value shown by CDFmod is partially determined by the results of the

HFE review noted in #3 below.

 Change in LERF - Another consideration is  LERF, similar to CDF in #1 above.

 Risk Importance Measures for the HA - The values of RAW and FV give a measure of the risk

importance of the HA in question. The specific meaning of these measures is discussed in

Section 2. These provide insight on the contribution of the HA to risk.

 Time and Integrated Risk - A further consideration is the length of time that the change will be

in place, if only a temporary modification. The integrated risk over time (or the ICCDP and

ICLERP) can be considered, per Section 2.4 above.

 Human Factors - A most important consideration is the degree of confidence that operators can

perform the actions required for the modification in question. This is determined by the

aggregate evaluation in Sections 3.2 through 3.12 of the Region I review guidance and Sections

4.2 through 4.4 of the Region II review guidance.

 Deterministic Criteria - Another consideration is the more traditional deterministic review

guidance provided in Section 3.1 of the Region I review guidance and Section 4.1 of the Region

II review guidance.

Supplemental Decision Factors

Additional factors may also be used, as appropriate, to determine the acceptability of a change. These

were adapted from RG 1.174 Section 2.2.6, Integrated Decisionmaking and include:

 The cumulative impact of previous changes and the trend in CDF (the licensee's risk

management approach)

 The cumulative impact of previous changes and the trend in LERF (the licensee's risk

management approach)

 The impact of the proposed change on operational complexity, burden on the operating

staff, and overall safety practices

 Plant-specific performance and other factors (for example, siting factors, inspection

findings, performance indicators, and operational events), and Level 3 PRA information,

if available
5 FINAL DECISION ON ACCEPTANCE OF HUMAN ACTIONS

51

 The benefit of the change in relation to its CDF/LERF increase

 The practicality of accomplishing the change with a smaller CDF/LERF impact

 The practicality of reducing CDF/LERF when there is reason to believe that the baseline

CDF/LERF are above the guideline values (i.e., 10-4 and 10-5 per reactor year) 

53

6 REFERENCES

10CFR50.59 Changes, Tests, and Experiments, January, 2000.

ANSI/ANS. (1984). Time response design criteria for safety-related operator actions (ANSI/ANS-58.8-

1984). La Grange Park, Illinois: American Nuclear Society.

ANSI/ANS. (1994). Time response design criteria for safety-related operator actions (ANSI/ANS-58.8-

1994). La Grange Park, Illinois: American Nuclear Society.

Azarm, A., Higgins, J., and Chu, L. (1999). Development of a risk-informed baseline inspection program

(BNL Report JCN W6234). Upton, New York: Brookhaven National Laboratory.

Gertman, D., Gilmore, W., Galyean, W., Groh, M., Gentiollion, C., Gilbert, B., and Reece, W. (1990).

Nuclear Computerized Library for Assessing Reactor Reliability: Summary Description (NUREG/ CR4639 ). Washington, D.C.: U.S. Nuclear Regulatory Commission.

Higgins, J. and Nasta, K. (1996). HFE insights for advanced reactors based upon operating experience

(NUREG/CR-6400). Washington, D.C.: U.S. Nuclear Regulatory Commission.

Higgins, J., O’Hara, J., Stubler, W., and Deem, R. (1999). Summary of credit of past operator action

cases (Report No. W6022-T1-1-7/99). Upton, New York: Brookhaven National Laboratory.

Higgins, J., et al. (2000). The development of guidance for the review of plant modifications involving

risk-important human actions. (Report No. W6022-T2-1-2/00). Upton, New York: Brookhaven National

Laboratory.

Higgins, J. , O’Hara, J., Lewis, P. , Persensky, J. (2002). Development of a Risk Screening Methodology

for Credited Operator Actions, In Proceedings for the 2002 IEEE 7th Conference on Human Factors and

Power Plants. Phoenix, Arizona: IEEE.

Hollnagel, E. (1998). Cognitive Reliability Analysis and Error Analysis Method. New York: Elsevier

Science Ltd.

Nuclear Energy Institute (2001). Option 2 Implementation Guidance, NEI 00-04, Draft Rev. B. 

Washington, DC: Nuclear Energy Institute.

NRC (1980). Criteria for preparation and evaluation of radiological emergency response plans and

preparedness in support of nuclear power plants (NUREG-0654). Washington, D.C.: U.S. Nuclear

Regulatory Commission.

NRC (1983). Instrumentation for light-water-cooled nuclear power plants to assess plant and

environmental conditions during and following an accident (Regulatory Guide 1.97). Washington, D.C.:

U.S. Nuclear Regulatory Commission.

NRC (1990). Severe accident risks: An assessment for five U.S. nuclear power plants (NUREG-1150). 

Washington, D.C., U.S. Nuclear Regulatory Commission.
6 REFERENCES

54

NRC. (1991). Information to Licensees regarding two NRC inspection manual sections on resolution of

degraded and nonconforming conditions and on operability (Information Notice 91-18). Washington,

D.C.: U.S. Nuclear Regulatory Commission.

NRC. (1996a). Standard review plan for the review of safety analysis reports for nuclear power plants

(NUREG-0800). Washington, D.C.: U.S. Nuclear Regulatory Commission. 

NRC (1997). Crediting of operator action in place of automatic actions and modification of operator

actions, including response times (Information Notice 97-78). Washington, D.C.: U.S. Nuclear

Regulatory Commission. 

NRC (1998). An approach to using probabilistic risk assessment in risk-informed decisions on plantspecific changes to the licensing basis (Regulatory Guide 1.174). Washington, D.C.: U.S. Nuclear

Regulatory Commission.

NRC (1998). An Approach for Plant-Specific, Risk-Informed Decisionmaking: Technical Specifications

(Regulatory Guide 1.177). August, 1998, Washington, D.C.: U.S. Nuclear Regulatory Commission.

NRC (2000a). NRC regulatory issue summary 2000-07: Use of risk-informed decision-making in license

amendment reviews (RIS 2000-07). Washington, D.C.: U.S. Nuclear Regulatory Commission.

NRC (2000b). Technical Basis and Implementation Guidelines for a Technique for Human Event

Analysis (ATHEANA) (NUREG-1624, Rev.1 ). Washington, DC: U.S. Nuclear Regulatory

Commission.

NRC (2002a). Human factors engineering program review model (NUREG-0711, Rev. 1). J. O’Hara, J.

Higgins, J. Persensky, P. Lewis, and J. Bongarra. Washington, D.C.: U.S. Nuclear Regulatory

Commission

NRC (2002b). Human-system interface design review guideline (NUREG-0700, Rev. 2). J. O’Hara, W.

Brown, P. Lewis, and J. Persensky. Washington, DC: U.S. Nuclear Regulatory Commission.

O'Hara, J., Higgins, J., and Stubler, W. (2000). The Development of Guidance for the Review of Plant

Modifications Involving Risk-Important Human Actions (BNL Report W6022-T2-1-1/2000). Upton, New

York: Brookhaven National Laboratory.

O’Hara, J., Stubler, W., Higgins. J., and Brown, W. (1997). Integrated system validation: Methodology

and review criteria (NUREG/CR-6393). Washington, DC: U.S. Nuclear Regulatory Commission.

Swain, A. and Guttmann, H. (1983). Handbook of human reliability analysis with emphasis on nuclear

power applications - Final report (NUREG/CR-1278). Washington, DC: U.S. Nuclear Regulatory

Commission.

Thadani, A. (1998). NRC internal memo: Thadani to Collins, Report on Risk-Informed 50.59 Options,

December 28, 1998. Washington, DC: U.S. Nuclear Regulatory Commission.
6 REFERENCES

55

Vesely, W., Davis, T., Denning, R., and Saltos, N. (1983). Measures of risk importance and their

applications (NUREG/CR-3385). Washington, DC: U.S. Nuclear Regulatory Commission.

57

GLOSSARY

Component - An individual piece of equipment such as a pump, valve, or vessel; usually part of a plant

system.

Function - An action that is required to achieve a desired goal. Safety functions are those functions that

serve to ensure higher-level objectives and are often defined in terms of a boundary or entity that is

important to plant integrity and the prevention of the release of radioactive materials. A typical safety

function is "reactivity control." A high-level objective, such as preventing the release of radioactive

material to the environment, is one that designers strive to achieve through the design of the plant and

that plant operators strive to achieve through proper operation of the plant. The function is often

described without reference to specific plant systems and components or the level of human and machine

intervention that is required to carry out this action. Functions are often accomplished through some

combination of lower-level functions, such as "reactor trip." The process of manipulating lower-level

functions to satisfy a higher-level function is defined here as a control function. During function

allocation the control function is assigned to human and machine elements.

Human-system interface (HSI) - The means through which personnel interact with the plant, including

the alarms, displays, controls, and job performance aids. Generically this includes maintenance, test, and

inspection interfaces as well.

Human factors - A body of scientific facts about human characteristics. The term covers all biomedical,

psychological, and psychosocial considerations; it includes, but is not limited to, principles and

applications in the areas of human factors engineering, personnel selection, training, job performance

aids, and human performance evaluation (see "Human factors engineering").

Human factors engineering (HFE) - The application of knowledge about human capabilities and

limitations to plant, system, and equipment design. HFE ensures that the plant, system, or equipment

design, human tasks, and work environment are compatible with the sensory, perceptual, cognitive, and

physical attributes of the personnel who operate, maintain, and support it (see "Human factors").

Mockup - A static representation of an HSI (see "Simulator").

Performance criteria - The criteria against which measured performance is compared in order to judge

its acceptability. Approaches to the establishment of performance criteria include:

Requirement Referenced - This is a comparison of the performance of the integrated system with

respect to an accepted, quantified, performance requirement. For many variables a requirementreferenced approach can be used; i.e., requirements for plant, system, and operator performance

can be defined through engineering analysis as part of the design process. Plant parameters

governed by technical specifications and time requirements for critical operator actions are

examples of performance measures for which a requirement-referenced criteria can be

determined. For performance measures where such specific requirement referenced criteria

cannot be used alternative criteria development methods must be used.

Benchmark Referenced - This is a comparison of the performance of the integrated system with

that of a benchmark system which is predefined as acceptable under the same conditions or

equivalent conditions. Such an approach is typically employed when no accepted independent

performance requirements can be established. Performance is evaluated through comparisons to
GLOSSARY

58

an accepted benchmark rather that through an absolute measurement. For example, the

evaluation may test whether the plant under review can be operated to stay within a level of

operator workload not exceeding that associated with Plant X. Plant X is identified as acceptable

for reasons such as its acceptable operating history and operators report their workload levels to

be acceptable. In this case the performance measure must be obtained for Plant X and the new

system, under similar operational conditions, and then compared. In the establishment of

benchmark-referenced criteria, similar test conditions should be established for the benchmark

system and system under evaluation.

Normative Referenced - Normative-referenced comparison is similar to a benchmark reference

comparison, however, the performance criterion is not based upon a single comparison system, it

is based upon norms established for the performance measure through its use in many system

evaluations. The new system performs as compared to the norms established under the same

conditions or equivalent conditions. This approach can be used when no accepted independent

performance requirements can be established, but repeated use of the same performance measure

enables the development of performance norms for acceptable and unacceptable systems.

Expert-Judgement Referenced - This is a comparison of the performance of the integrated system

with criteria established through the judgement of SMEs.

Performance shaping factors (PSFs) - Factors that influence human reliability through their effects on

performance. PSFs include factors such as environmental conditions, HSI design, procedures, training,

and supervision.

Primary tasks - Those tasks performed by the operator to supervise the plant; i.e., monitoring, detection,

situation assessment, response planning, and response implementation.

Risk-important human action - Actions that must be performed successfully by operators to ensure

plant safety. There are both absolute and relative criteria for defining risk important actions. From an

absolute standpoint, a risk-important action is one whose successful performance is needed to ensure that

predefined risk criteria are met. From a relative standpoint, the risk- important actions constitute the

most risk-significant human action identified. 

Safety-related operator action - A manual action required by plant emergency procedures that is

necessary to cause a safety-related system to perform its safety-related function during the course of any

Design Basis Event. The successful performance of a safety-related operator action might require that

discrete manipulations be performed in a specific order. 

Secondary tasks - Those tasks that the operator must perform when interfacing with the plant, but are

not directed to the primary task. Secondary tasks may include: navigating through and paging displays,

searching for data, choosing between multiple ways of accomplishing the same task, and making

decisions regarding how to configure the interface. 

Simulator - A facility that physically represents the HSI configuration and that dynamically represents

the operating characteristics and responses of the plant in real time (see "Mockup").
GLOSSARY

59

System - An integrated collection of plant components and control elements that operate alone or with

other plant systems to perform a function. 

Task - A group of activities that have a common purpose, often occurring in temporal proximity, and that

utilize the same displays and controls 

Testbed - The representation of the human-system interface and the process model used in testing.

Validation - The process by which the integrated system (consisting of hardware, software, and

personnel elements) is evaluated to determine whether it acceptably supports safe operation of the plant.

Validity - The characteristics of the methods and tools used in the validation process. See the specific

uses of the term: construct validity, convergent validity, performance representation validity, statistical

conclusion validity, system representation validity, and test design validity.

Verification - The process by which the human-system interface design is evaluated to determine

whether it acceptably reflects personnel task requirements and HFE design guidance.

Vigilance - The degree to which an operator is alert.

Workload - The physical and cognitive demands placed on plant personnel.

APPENDIX

A-1

APPENDIX 

Generic Risk-Important Human Actions

This attachment contains two tables of generic risk-important HAs for BWRs and PWRs, respectively. 

Each table is further divided into “Group 1" risk-important HAs and “Group 2" potentially risk-important

HAs. To facilitate readability of the tables, the names of common events and plant systems are given in

acronyms. These acronyms are defined in the acronym list on page xiii of this document. These Tables

are for use in the Generic Method described in Section 2.7 of this document.

Table A.1 Generic BWR Risk-Important Human Actions

Group 1: BWR Risk-Important Human Actions

Human Actions Description and Reasons for Risk-Importance

Perform Manual

Depressurization

On selected sequences, such as station blackout (SBO), manual depressurization is required

after failure of high pressure injection systems to allow for injection with low pressure systems. 

A complicating factor is that some procedures initially direct the operator to inhibit ADS. In

some PRAs this appears in cutsets up to 45 % of CDF. Operators typically depressurize by

manually operating the safety relief valves (SRV).

Vent Containment On a transient or loss-of-coolant accident (LOCA) sequence, with failure of the PCS,

containment temperature and pressure increase and must be controlled. This can be done by

containment heat removal, suppression pool cooling, or containment venting. Actions are

required to remove DH before adverse conditions are reached (e.g., high Suppression Pool

temperature leading to loss of ECCS pumps).

Align Containment or

Suppression Pool Cooling

Actions During Shutdown Almost all actions, including actuation of various equipment, are done manually during

shutdown. The operator’s understanding of the plant configuration is necessary for the

successful manual actions.

Group 2: BWR Potentially Risk-Important Human Actions

Human Actions Description and Reasons for Risk-Importance

Level Control in

anticipated transient

without scram (ATWS) 

Effective Rx Vessel level manual control at lower than normal levels (e.g., near the top of the

active fuel) is needed during an ATWS in order to reduce core power.

Initiate Standby Liquid

Control (SLC)

Manual initiation of SLC is needed for ATWS sequences.

Inhibit ADS Some IPEs conclude that core damage will occur if ADS is not manually inhibited in an ATWS

event due to instabilities created at low pressures.

Mis-calibrate Pressure

Switches

Various pressure switches are important for initiating ECCS and operating ECCS permissives. 

Common cause mis-calibration of these switches can affect multiple trains of safety systems.

Initiate isolation

condenser (IC)

For the early design BWR plants, this action is important during accidents to ensure the continued

viability of the cooling from the IC.

Control feedwater (FW)

events 

The actions of operators to properly control the FW system as an injection source after loss-ofinstrument air or other loss of FW events can be important in various sequences such as, transients

and small LOCAs.

Recover Offsite Power The actions of operators to recover offsite power after a total loss of offsite power (LOOP) is

important to limit the risk due to station blackout and other LOOP core damage sequences. These

are modeled with various recovery times in PRAs.
ATTACHMENT A

A-2

Shedding of DC Load

After SBO

While often not well modeled, operator action to shed DC loads is needed to extend the battery

charge in order to operate the AC independent HPCI and RCIC systems and to keep the SRVs

open (to allow low pressure vessel injection from a diesel-driven fire pump). This extends the

time to core damage and the time that operators have for recovery of AC power.

Similar actions to those

in Group I

Actions that are substantially similar (but not identical) to those contained in Group 1 of this Table

should be considered as potentially risk-important, if they involve the same systems, components,

or actions.

Actions involving the

most risk-important

systems 

Each plant has one or two systems that are clearly the most risk significant in the plant. Human

actions associated with these systems should be considered as potentially risk-important. When

modifications associated with these risk-important systems are being considered, new human

actions may be created that were not in the original PRA, but that will be risk-important.

Table A.2 Generic PWR Risk-Important Human Actions

Group 1: PWR Risk-Important Human Actions

Establish

Recirculation

In LOCA scenarios, the switching of ECCS lines from the injection to the recirculation mode is done

manually. Failure to do so or human error involving the valve alignment is important. Both low

pressure and high pressure recirculation modes were noted to be important.

Feed and Bleed Failure of the operator to initiate and perform the feed and bleed operation of the reactor coolant

system as a last resort of heat removal is important. Of particular importance is the bleed portion

using the pressurizer PORVs.

Provide Water

Supply for AFW

Use of water pumps to transfer water, from other sources of make up to the CST for use by AFW, is

considered important in scenarios when long term cooling through SG is needed.

RCP Trip On a loss of cooling to the RCP seals, it is important for operators to quickly trip the pumps to

prevent an RCP seal LOCA.

Action During

Shutdown

Almost all actions, including actuation of various equipment, are done manually during shutdown. 

The operator’s understanding of the plant configuration is necessary for the successful manual

actions.

Group 2: PWR Potentially Risk-Important Human Actions

Human Actions Description and Reasons for Risk-Importance

Recover of RCP Seal

Cooling

In some plants there are means of alternate cooling for RCP seals that could be relied on in scenarios

involving loss of CCW. However, the alignment of the system is manual and requires operator

action.

Recover Emergency

AC or Offsite Power

Some losses of AC power can be recovered by either manual transfer of the source of power, or

recovery of onsite normal/emergency AC power. This recovery action is considered risk significant

in many PRAs.

Actions in Response

to ATWS

Upon failure of RPS, the operator should perform several actions, starting with manual scram,

ensuring turbine trip, and most importantly initiating emergency boron injection.

DEP and

Equalization during

SGTR Event

An important strategy during an SGTR event is the depressurization of primary and secondary

systems and the equalization of pressures between primary and secondary. These all help to limit the

leakage.

Isolate SG During both a MSLB and an SGTR event, isolation of the affected SG is important.

Shut PORV Block

Valve

During a stuck open PORV event, shutting the PORV block is an important action to eliminate the

leak.
APPENDIX

A-3

Isolate ISLOCA In some plants there is a capability to isolate an interfacing systems LOCA through manual actions. 

Operator failure to isolate an interfacing LOCA in the LPI system is considered risk significant in

these plants.

Similar Actions to

those in Group I

Actions that are substantially similar to those contained in Group 1 of this Table should be

considered as potentially risk-important, if they involve the same systems, components, or actions.

Actions Involving

the Most RiskImportant Systems 

Each plant has one or two systems that are clearly the most risk significant in the plant. Human

actions associated with these systems should be considered as potentially risk-important. When

modifications associated with these risk-important systems, are being considered new human actions

may be created that were not in the original PRA, but that will be risk-important.