I

NUREG-0711, Rev. 2

Human Factors Engineering

Program Review Model

U.S. Nuclear Regulatory Commission

Office of Nuclear Regulatory Research

Washington, DC 20555-0001
AVAILABILITY OF REFERENCE MATERIALS

IN NRC PUBLICATIONS

NRC Reference Material

As of November 1999, you may electronically access

NUREG-series publications and other NRC records at

NRC's Public Electronic Reading Room at

http://www.nrc.pov/reading-rm.htrnl.

Publicly released records include, to name a few,

NUREG-series publications; Federal Register notices;

applicant, licensee, and vendor documents and

correspondence; NRC correspondence and internal

memoranda; bulletins and information notices;

inspection and investigative reports; licensee event

reports; and Commission papers and their attachments.

NRC publications in the NUREG series, NRC

regulations, and Title 10, Energy, in the Code of

Federal Regulations may also be purchased from one

of these two sources.

1. The Superintendent of Documents

U.S. Government Printing Office

Mail Stop SSOP

Washington, DC 20402-0001

Internet: bookstore.gpo.gov

Telephone: 202-512-1800

Fax: 202-512-2250

2. The National Technical Information Service

Springfield, VA 22161-0002

www.ntis.gov

1-800-553-6847 or, locally, 703-605-6000

A single copy of each NRC draft report for comment is

available free, to the extent of supply, upon written

request as follows:

Address: Office of the Chief Information Officer,

Reproduction and Distribution

Services Section

U.S. Nuclear Regulatory Commission

Washington, DC 20555-0001

E-mail: DlSTRIBUTION~nrc.gov

Facsimile: 301-415-2289

Some publications in the NUREG series that are

posted at NRC's Web site address

http://www.nrc.gov/reading-rmIdoc-collections/nuregs

are updated periodically and may differ from the last

printed version. Although references to material found

on a Web site bear the date the material was accessed,

the material available on the date cited may

subsequently be removed from the site.

Non-NRC Reference Material

Documents available from public and special technical

libraries include all open literature items, such as

books, journal articles, and transactions, Federal

Register notices, Federal and State legislation, and

congressional reports. Such documents as theses,

dissertations, foreign reports and translations, and

non-NRC conference proceedings may be purchased from their sponsoring organization.

Copies of industry codes and standards used in a

substantive manner in the NRC regulatory process are

maintained atThe NRC Technical Library -

Two White Flint North

11545 Rockville Pike

Rockville, MD 20852-2738

These standards are available in the library forreference use by the public. Codes and standards are

usually copyrighted and may be purchased from the

originating organization or, if they are American

National Standards, fromAmerican National Standards Institute

11 West 42'd Street

New York, NY 10036-8002

www.ansi.org

212-642-4900

Legally binding regulatory requirements are stated

only in laws; NRC regulations; licenses, including technical specifications; or orders, not in

NUREG-series publications. The views expressed in contractor-prepared publications in this series are

not necessarily those of the NRC.

The NUREG series comprises (1) technical and

administrative reports and books prepared by the

staff (NUREG-XXXX) or agency contractors

(NUREG/CR-XXXX), (2) proceedings of

conferences (NUREG/CP-XXXX), (3) reports

resulting from international agreements

(NUREGIIA-XXXX), (4) brochures

(NUREG/BR-XXXX), and (5) compilations of legal

decisions and orders of the Commission and Atomic

and Safety Licensing Boards and of Directors'

decisions under Section 2.206 of NRC's regulations (NUREG-0750).

.1 -

I:It.,

I

I

I

1,

s

Z I

.IiIi

.I

ij -. ; ,i
NUREG-0711, Rev. 2

Human Factors Engineering

Program Review Model

Manuscript Completed: January 2004

Date Published: February 2004

Prepared by

J.M. O'Hara*, J.C. Higgins*

J.J. Persensky", P.M. Lewis", J.P. Bongarra**

*Brookhaven National Laboratory

Energy Sciences and Technology Department

Upton, NY 11973-5000

P.M. Lewis, NRC Project Manager

Division of Systems Analysis and Regulatory Effectiveness

Office of Nuclear Regulatory Research

**U.S. Nuclear Regulatory Commission

Washington, DC 20555-0001

NRC Job Code Number Y6022
I

ABSTRACT

This document is used by the staff of the Nuclear Regulatory Commission to review the human factors

engineering (HFE) programs of applicants for construction permits, operating licenses, standard design

certifications, combined operating licenses, and for license amendments. The purpose of these reviews is

to verify that accepted HFE practices and guidelines are incorporated into the applicant's HFE program.

The review methodology provides a basis for performing reviews that address the twelve elements of an

HFE program: HFE Program Management, Operating Experience Review; Functional Requirements

Analysis and Function Allocation, Task Analysis, Staffing, Human Reliability Analysis, Human-System

Interface Design, Procedure Development, Training Program Development, Human Factors Verification

and Validation, Design Implementation, and Human Performance Monitoring. Each review element is

divided into four sections: Background, Objective, Applicant Submittals, and Review Criteria.

References to sources of additional information are also provided for each element.

iii
CONTENTS

Page

ABSTRACT ......................................................... iii

EXECUTIVE SUMMARY ......................................................... Xi

FOREWORD..................................................................... xvii

ACRONYMS ................................................................... XiX

1 INTRODUCTION ............................................................... I

1.1 Background . ........................................................... I

1.2 General Description of the Program Review Model ............................ 2

1.2.1 Review Elements ....................................................... 2

1.2.2 Sources of Additional Information ............. ............................. 3

1.3 Applications........................................................... 4

1.3.1 Application to Current Applicants Under 10 CFR Part 50 ....... ................ 4

1.3.2 Application to Advanced NPP Applicants Under 10 CFR Part 52 ...... ........... 5

1.4 Graded Approach to Review ................... ........................... 6

2 HFE PROGRAM MANAGEMENT. . . 7

2.1 Background. 7

2.2 Objective. 7

2.3 Applicant Submittals. 8

2.4 Review Criteria. 8

2.4.1 General HFE Program Goals and Scope. 8

2.4.2 HFE Team and Organization. 9

2.4.3 HFE Process and Procedures .0

2.4.4 HFE Issues Tracking .0

2.4.5 Technical Program .11

2.5 Sources of Additional Information .12

3 OPERATING EXPERIENCE REVIEW ....... ............ .......... 15

3.1 Background ..................................... 15

3.2 Objective ............................. 16

3.3 Applicant Submittals ............................. 16

3.4 Review Criteria ............................. 16

3.4.1 Scope ............................. 16

3.4.2 Issue Analysis, Tracking, and Review ............ ................. 17

3.5 Sources of Additional Information .......... ................... 17

4 FUNCTIONAL REQUIREMENTS ANALYSIS AND FUNCTION ALLOCATION ... 19

4.1 Background .19

4.2 Objective .19

4.3 Applicant Submittals .19

4.4 Review Criteria .20

4.5 Sources of Additional Information .22

v
CONTENTS (continued)

Page

5 TASK ANALYSIS ............................ 25

5.1 Background ....................... 25

5.2 Objective ....................... 25

5.3 Applicant Submittals ........................ 25

5.4 Review Criteria ........................ 25

5.5 Sources of Additional Information ........................ 27

6 STAFFING AND QUALIFICATIONS ............. .............. 29

6.1 Background ............. 29

6.2 Objective ............. 29

6.3 Applicant Submittals .............. 29

6.4 Review Criteria .......... 29

6.5 Sources of Additional Information ........................ 31

7 HUMAN RELIABILITY ANALYSIS ........... ........................ 3

7.1 Background ....................... 33

7.2 Objective ....................... 35

7.3 Applicant Submittals ........................ 35

7.4 Review Criteria ........................ 35

7.5 Sources of Additional Informnation ........................ 36

8 HUMAN-SYSTEM INTERFACE DESIGN .................................... 37

8.1 Background ................. ................. 37

8.2 Objective ..... ............ ................. 37

8.3 Applicant Submittals ................. ................... 37

8.4 Review Criteria ................. ................... 37

8.4.1 HSI Design Inputs ................. ................... 37

8.4.2 Concept of Operations .................................... 38

8.4.3 Functional Requirement Specification ....................... 39

8.4.4 HSI Concept Design ................ .................. 39

8.4.5 HSI Detailed Design and Integration ....................................... 40

8.4.6 HSI Tests and Evaluations ....... 42

8.4.6.1 Trade-Off Evaluations ....... 42

8.4.6.2 Performance-Based Tests ....... 42

8.4.7 HSI Design Documentation ....... 43

8.5 Sources of Additional Information ........................................ 43

9 PROCEDURE DEVELOPMENT .................................................. 45

9.1 Background .45

9.2 Objective .45

9.3 Applicant Submittals .45

9.4 Review Criteria .45

9.5 Sources of Additional Information .47

vi
CONTENTS (continued)

Page

10 TRAINING PROGRAM DEVELOPMENT .......................................... 49

10.1 Background . . . ................ 49

10.2 Objective ................... 49

10.3 Applicant Submittals .............. .... 49

10.4 Review Criteria ................... 49

10.4.1 General Approach .49

10.4.2 Organization of Training .50

10.4.3 Learning Objectives .50

10.4.4 Content of Training Program .51

10.4.5 Evaluation and Modification of Training .52

10.4.6 Periodic Re-training ....... 52

10.5 Sources of Additional Information ........................................ 52

1 1 HUMAN FACTORS VERIFICATION AND VALIDATION ...... ..................... 55

11.1 Background .......................................................... 55

11.2 Objective ............................................................ 56

11.3 Applicant Submittals ................................................... 56

11.4 Review Criteria ........................................................ 56

11.4.1 Operational Conditions Sampling ................................... 56

11.4.1.1 Operational Conditions Sampling Review Objectives ............... .... 56

11.4.1.2 Operational Conditions Review Criteria ....... ...................... 57

11.4.1.2.1 Sampling Dimensions ................................... 57

11.4.1.2.2 Identification of Scenarios ................................ 60

11.4.1.2.3 Special Considerations for Plant Modernization Programs .... ... 60

11.4.2 Design Verification ........... ................................... 61

11.4.2.1 Inventory and Characterization ........ ............................ 61

11.4.2.1.1 Inventory and Characterization Review Objectives ..... ........ 61

11.4.2.1.2 Inventory and Characterization Review Criteria ..... .......... 61

11.4.2.2 HSI Task Support Verification ........ ............................ 62

11.4.2.2.1 HSI Task Support Verification Review Objectives ..... ........ 62

11.4.2.2.2 HSI Task Support Verification Review Criteria ..... ........... 62

11.4.2.3 HFE Design Verification ......................................... 63

11.4.2.3.1 HFE Design Verification Review Objective ..... ............. 63

11.4.2.3.2 HFE Design Verification Review Criteria ..... ............... 63

11.4.3 Integrated System Validation ....................................... 65

11.4.3.1 Integrated System Validation Review Objective ................... .... 65

11.4.3.2 Integrated System Validation Review Criteria ........ .. .............. 66

11.4.3.2.1 Test Objectives ......................................... 66

11.4.3.2.2 Validation Testbeds ..................................... 66

11.4.3.2.3 Plant Personnel ........................................ 67

11.4.3.2.4 Scenario Definition ..................................... 68

11.4.3.2.5 Performance Measurement ............................... 69

11.4.3.2.5.1 Measurement Characteristics ...................... 69

11.4.3.2.5.2 Performance Measure Selection ..... ............... 69

11.4.3.2.5.3 Performance Criteria ............................ 71

vii
CONTENTS (continued)

Page

11.4.3.2.6 Test Design .............. ............................. 71

11.4.3.2.6.1 Coupling Crews and Scenarios ...... .............. 71

11.4.3.2.6.2 Test Procedures ........ ........................ 71

11.4.3.2.6.3 Test Personnel Training ....... ................... 72

11.4.3.2.6.4 Participant Training ....... ...................... 72

11.4.3.2.6.5 Pilot Testing ......... .......................... 73

11.4.3.2.7 Data Analysis and Interpretation ....... .................... 73

11.4.3.2.8 Validation Conclusions ......... ......................... 73

11.4.4 Human Engineering Discrepancy Resolution ........... ............... 74

11.4.4.1 Human Engineering Discrepancy Resolution Review Objective ..... ..... 74

11.4.4.2 Human Engineering Discrepancy Resolution Review Criteria ..... ....... 74

11.5 Sources of Additional Information ................... .............. 78

12 DESIGN IMPLEMENTATION ................................. 79

12.1 Background ................................. 79

12.2 Objective ........................................ 80

12.3 Applicant Submittals ................................. 80

12.4 Review Criteria ................................. 80

12.4.1 General Criteria ................................................. 80

12.4.2 Modernization Programs Consisting of Many Small Modifications .... .... 81

12.4.3 Modernization Programs Consisting of Large Modifications During Multiple

Outages .81

12.4.4 Modernization Programs Where Both Old and New Equipment

are Left in Place .82

12.4.5 Modernization Programs Where New Non-Functional HSls are in Place in

Parallel with Old Functional HSIs ............. ............. 82

12.4.6 Final Plant HFE Design Verification . .......................... 83

12.5 Sources of Additional Information ................................. 83

13 HUMAN PERFORMANCE MONITORING ................................. 85

13.1 Background ........................ 85

13.2 Objective ............................. 85

13.3 Applicant Submittals ........................ 85

13.4 Review Criteria ........................ 85

13.5 Sources of Additional Information ........................ 86

14 REFERENCES ............ 87

GLOSSARY. ...................................................................... 93

APPENDIX A: HFE Design Team Composition . A-I

viii
CONTENTS (continued)

Pace

List of Figures

1.1 HFE program review model review elements . ......................................... 3

4.1 Allocation of functions to human and machine resource ................................ 20

7.1 The role of human reliability analysis in the HFE program .............................. 3. 4

11.1 Overview of verification and validation activities ..................................... 56

11.2 The HED evaluation process ........................ ............................. 77

List of Tables

3.1 The role of operating experience review in the HFE program ......... ................... 15

5.1 Task considerations ................... .............................. 26

10.1 Some knowledge and skill dimensions for learning objectives identification ..... ........... Si

12.1 Typical advantages & disadvantages of different methods of modernization program

implementation ............................................................ 79

ix..
EXECUTIVE SUMMARY

Nuclear power plant (NPP) personnel play a vital role in the productive, efficient, and safe generation of

electric power. Operators monitor and control plant systems and components to verify their proper

functioning. Test and maintenance personnel check that plant equipment is functioning properly and

restore components when malfunctions occur.

The human factors engineering (HFE) staff of the Nuclear Regulatory Commission's (NRC's) Office of

Nuclear Reactor Regulation (NRR) evaluates HFE programs of applicants for construction permits (CPs),

operating licenses (OLs), standard design certifications (DCs), combined operating licenses (COLs), and

for license amendments. The purpose of these reviews is to verify that accepted HFE practices and

guidelines are incorporated into the applicant's HFE program. The HFE review includes the design

process, the final design, its implementation, and ongoing performance monitoring. Therefore, these

reviews support public health and safety by verifying that accepted HFE practices and guidelines are

incorporated into the design.

The HFE aspects of the plant should be developed, designed, and evaluated based on the basis of a

structured analysis using accepted HFE principles. Therefore, the review method reflects a top-down

approach for conducting an NRC safety evaluation so that the significance of individual topics may be

seen in relationship to the high-level goal of plant safety. Top-down refers to an approach starting at the

"top" with the plant's high-level mission goals and dividing them into the functions necessary to achieve

the goals. Functions are allocated to human and system resources and are separated into tasks. Personnel

tasks are analyzed to identify the alarms, displays, procedures, and controls that will be required for task

performance. Tasks are arranged into meaningful jobs and the human-system interface (HSI), procedures,

and training are designed to best support them. The detailed design (of the HSI, procedures, and training)

is the "bottom" of the top-down process. The HFE safety evaluation is broad-based and includes normal

and emergency operations, maintenance, test, inspection, and surveillance activities.

The HFE Program Review Model consists of twelve review elements; Each element is divided into four

sections: Background, Objective, Applicant Submittals, and Review Criteria.

* Background - A brief explanation is given of the rationale and purpose of each element.

* Objective - The review objective(s) of the element is defined.

* Applicant Submittals - Materials to be provided for the NRC's review are listed.

* Review Criteria - The acceptance criteria for the review elements are provided.

References to sources of additional information are also provided for each element.

An overview of each of the review elements follows.

HFE Proeram Manaeement

The overall purpose of the HFE program review is to verify that

* the applicant has integrated HFE into the plant's development, design, and evaluation

xi
* the applicant has provided HFE products (e.g., HSIs, procedures, and training) that make

it possible to perform operation, maintenance, test, inspection, and surveillance tasks

safely, efficiently, and reliably

* the HFE program and its products reflect "state-of-the-art human factors principles"

[10 CFR 50.34(f) and 10 CFR 52.47(a)(l)(ii)] and satisfy all specific regulatory

requirements

The objective of this review element is to verify that the applicant has an HFE design team with the

responsibility, authority, placement within the organization, and composition to provide reasonable

assurance that the design commitment to HFE is met. Also, the team should be guided by a plan to verify

that the HFE program is properly developed, executed, overseen, and documented. This plan should

describe the technical program elements ensuring that all aspects of the HSI, procedures, and training are

developed, designed, and evaluated on the basis of a structured analysis using accepted HFE principles.

Operatine Experience Review

The main purpose of conducting an operating experience review (OER) is to identify HFE-related safety

issues. The OER should provide information on the past performance of predecessor designs. In the case

of new plants this may be earlier designs on which the new design is based. In the case of plant

modifications, it may be the design of the systems being changed. The issues and lessons learned from

operating experience provide a basis for improving the plant design in a timely way; i.e., at the beginning

of the design process.

The objective of this element is to verify that the applicant has identified and analyzed HFE-related

problems and issues in previous designs that are similar to the current design under review. In this way,

negative features associated with predecessor designs may be avoided in the current one while retaining

positive features. The OER should address the predecessor systems upon which the design is based,

selected technological approaches (e.g., if touch-screen interfaces are planned, the HFE issues associated

with using them should be reviewed), and the plant's HFE issues (e.g., generic safety issues defined by

the NRC).

Functional Requirements Analysis and Function Allocation

The purpose of this review element is to verify that the applicant has defined the plant's safety functional

requirements' and that the function allocations take advantage of human strengths and avoid allocating

functions that would be negatively affected by human limitations. The operator's role is examined in two

steps: functional requirements analysis, and function allocation (assignment of levels of automation).

Functional requirements analysis is the identification of those functions which must be performed to

satisfy the plant's safety objectives, i.e., to prevent or mitigate the consequences of postulated accidents

that could cause undue risk to the health and safety of the public. This analysis determines the objectives,

' The term "requirements" as used here and elsewhere in this document, refers to requirements that are established as

part of the design process. The term requirements is used in this context as a term-of-art. These are not "regulatory"

requirements. There are no regulatory requirements in this document, only review guidance.

xii
performance requirements, and constraints of the design, and sets a framework for understanding the role

of controllers (whether personnel or system) in regulating plant processes.

Function allocation is the analysis of the requirements for plant control and the assignment of control

functions to (I) personnel (e.g., manual control), (2) system elements (e.g., automatic control and passive,

self-controlling phenomena), and (3) combinations of the two (e.g., shared control and automatic systems

with manual backup). Plant safety and reliability are enhanced by exploiting the strengths of personnel

and system elements, including improvements that can be achieved through assigning control to these

elements with overlapping and redundant responsibilities. Function allocation should be based upon HFE

principles using a structured and well-documented methodology that provides personnel with logical,

coherent, and meaningful tasks.

Task Analysis

Task analysis is the identification of task requirements for accomplishing the functions allocated to plant

personnel. The objective of a task analysis review is to verify that the applicant's task analysis identifies

the requirements of the tasks that personnel must perform. The task analysis should (I) provide one of

the bases for making decisions on design, (2) verify that human-performance requirements do not exceed

human capabilities, (3) be used as basic input for developing procedures, (4) be used as basic information

for developing the staffing, training, and communication requirements of the plant, and (5) form the basis

for specifying the design requirements for the displays, data processing, and controls needed to carry out

tasks.

Staffin! and Oualifications

Plant staffing and their qualifications are important consideration throughout the design process. Initial

staffing levels may be established early in the process based on experience with previous plants, staffing

goals (such as for staffing reductions), initial analyses, and government regulations. However, the

acceptability of the staffing goals and assumptions should be examined as the design of the plant

proceeds. The objective of the staffing review is to verify that the applicant has systematically analyzed

the requirements for the number and qualifications of personnel that includes a thorough understanding of

task requirements and regulatory requirements.

Human Reliability Analysis

Human reliability analysis (HRA) seeks to evaluate the potential for, and mechanisms of, human error

that may affect plant safety. Thus, it is an essential element in achieving the HFE design goal of.

providing operator interfaces that will minimize personnel errors, allow their detection, and provide

recovery capability. The HRA should be conducted as an integrated activity to support both the HFE

design and probabilistic risk assessment (PRA). The PRA and HRA should be performed early in the

design process to provide insights and guidance both for systems design and for HFE purposes. The

robustness of the HRA depends, in large part, on the analyst's understanding of personnel tasks, the

information related to them, and the factors which influence human performance. Accordingly; the HRA

might be carried out interactively as the design progresses. By developing an'understanding of the causes,

modes, and probabilities of human error, the HRA can provide valuable insights into the desirable

characteristics of the HSI design; consequently, special attention should be paid to those scenarios,

xiii
critical human actions, and HSI components that were identified by HRA and PRA analyses as being

important to the plant's safety and reliability.

The objectives of this review element are to verify that (1) the applicant has addressed human-error

mechanisms in the design of the HFE aspects of the plant to minimize the likelihood of personnel error,

and verify that errors are detected and recovered from; and (2) the HRA activity effectively integrates the

HFE program with the PRA and risk analysis.

Human-System Interface Design

The objective of this review element is to evaluate the process by which HSI design requirements are

developed and HSI designs are identified and refined. The review should verify that the applicant has

appropriately translated functional and task requirements to the detailed design of alarms, displays,

controls, and other aspects of the HSI. The HSI should be designed using a structured methodology that

should guide designers in identifying and selecting candidate HSI approaches, defining the detailed

design, and performing HSI tests and evaluations. It should cover the development and use of HFE

guidelines that are tailored to the unique aspects of the applicants's design, e.g., a style guide to define the

design-specific conventions. The availability of an HSI design methodology will help verify

standardization and consistency in applying HFE principles. The process and the rationale for the HSI

design should be documented for review (including the results of trade-off studies, other analyses and

evaluations, and the rationale for choosing design and evaluation tools).

Procedure Develonment

Procedures are essential to plant safety because they support and guide personnel interactions with plant

systems and their response to plant-related events. Procedures should be developed from the same design

process and analyses as the HSIs and training. This will result in a well-integrated design with a highdegree of consistency. The objective of the review is to verify that human engineering principles and

guidance are applied, along with all other design requirements, to develop procedures that are technically

accurate, comprehensive, explicit, easy to use, and validated.

Training Program Development

The NRC requires a systems approach to training and also requires that it be based on the systematic

analysis ofjob and task requirements. The HFE analyses provide a valuable understanding of such task

requirements. The objective of the training review is to verify that the applicant establishes an approach

for developing personnel training that incorporates the elements of a systems approach, and

* evaluates the knowledge and skill-requirements of personnel

* coordinates the development of the training program with the other elements of the HFE

design process

* implements the training effectively in a manner consistent with human factors principles

and practices

xiv
Human Factors Verification and Validation

Verification and validation (V&V) evaluations comprehensively determine that the final design conforms

to HFE design principles, and enables personnel to successfully and safely perform their tasks to achieve

operational goals. This element involves three evaluations, the objectives of which are to verify that the

applicant has performed the following activities:

* HSI Task Support Verification - an evaluation to verify that the HSI supports personnel

task requirements as defined by task analyses.

* HFE Design Verification - an evaluation to verify that the HSI is designed to

accommodate human capabilities and limitations as reflected in HFE guidelines such as

those provided in NUREG-0700.

* Integrated System Validation - an evaluation using performance-based tests to determine

whether an integrated system design (i.e., hardware, software, and personnel elements)

meets performance requirements and acceptably supports safe operation of the plant.

These evaluations identify human engineering discrepancies (HEDs). The staffs review of the applicant's

HED resolution is an activity that can be performed iteratively with V&V. HED resolution review

verifies that the applicant has assessed the importance of HEDs, corrected important HEDs, and that the

results are confirmed to be acceptable.

Design Implementation

This element addresses implementation of the HFE aspects of the plant design for both new plants and

plant modifications. For a new plant, the implementation phase is well defined and carefully monitored

through start-up procedures and testing; implementation of plant modifications is more complex.

The objectives of this review element are to verify that:

* the applicant's implementation of modernized plant systems, HSls, procedures, and

training considers their effect on personnel performance and provides the necessary

support to verify safe operations

* the applicant's as-built design conforms to the verified and validated design that resulted

from the HFE design process.

Human Performance Monitoring

A human performance monitoring strategy will help to verify that the confidence developed by the

completion of the integrated system validation is maintained over time. There is no intent to periodically

repeat the full integrated system validation; however, there should-be sufficient evidence to provide

reasonable confidence that personnel have maintained the skills necessary to accomplish the assumed

actions.

The objective of this review is to verify that the applicant has prepared a human performance monitoring

strategy for ensuring that no safety degradation occurs because of any changes that are made in the plant

and to verify that the conclusions that have been drawn from the evaluation remain valid over time.

xv
FOREWORD

NUREG-071 1, Revision 0 (I 994) was published to provide criteria for the review of the human factors

aspects of design certification submittals for advanced nuclear power plants. Revision I, published in

May 2002, (1) provided additional human factors engineering (HFE) review guidance for hybrid humansystem interfaces (HSIs); (2) revised the sections on Functional Requirements Analysis and Function

Allocation, Human Reliability Analysis, Human-System Interface Design, and HFE Verification and

Validation; (3) added new sections on Design Implementation and Human Performance Monitoring; and

(4) integrated the NRC's HFE review processes into a single document.

Revision I was submitted for public comment in December, 2002. This document, Revision 2,

incorporates changes the NRC made to the document in response to public comments.

The review criteria in are organized into 12 elements: (1) HFE Program Management, (2) Operating

Experience Review, (3) Functional Requirements Analysis and Function Allocation, (4) Task Analysis,

(5) Staffing and Qualifications, (6) Human Reliability Analysis, (7) Human-System Interface Design,

(8) Procedure Development, (9) Training Program Development, (10) Human Factors Verification and

Validation, (I1) Design Implementation, and (12) Human Performance Monitoring.

NUREG-07 11 has been used as the basis for the three main applications in NUREG-0800, Standard

Review Plan, Chapter 18, "Human Factors Engineering": (I) new plants, (2) human-system interface

modifications, and (3) modifications involving human actions. The review guidance contained in

NUREG-0711 and NUREG-0800 is adapted to address specific types of HFE reviews. For example, a

review of a new nuclear power plant will likely use all the elements, while a review of changes to the

HSIs of an existing plant will likely use only a subset of the elements. Thus the staff will tailor the

guidance used based on the unique circumstances of an individual design review.

Farouk Eltawila, Director

Division of Systems Analysis and Regulatory Effectiveness

Office of Nuclear Regulatory Research

xvii
PAPERWORK REDUCTION ACT STATEMENT

Any information collections referenced in this NUREG are covered by the requirements of

10 CFR Parts 50 and 51, which were approved by the Office of Management and Budget,

approvals numbers 3150-0011 and 3150-0151.

xviii
ACRONYMS

ALWR advanced light water reactor

ANS American Nuclear Society

ANSI American National Standards Institute

ATWS anticipated transients without scram

BNL Brookhaven National Laboratory

CAP corrective action program

CAS central alarm station

CFR U.S. Code of Federal Regulations

COL combined operating license

CP construction permit

CR control room

DAC design acceptance criteria

DBA design basis accident

EOF emergency offsite facility

EOP emergency operating procedure

EPG emergency procedure guideline

EPRI Electric Power Research Institute

GDC General Design Criteria

GTG generic technical guidelines

HA human action

HED human engineering discrepancies

HFE human factors engineering

HRA human reliability analysis

HSI human-system interface

I&C instrumentation and control

IAEA International Atomic Energy Agency

IEC International Electrotechnical Commission

IEEE Institute of Electrical and Electronics Engineers

ISO International Standards Organization

ITAAC inspections, tests, analyses, and acceptance criteria

LCS local control station

LER licensee event report

LOCA loss-of-coolant accident

MCR main control room

MUX multiplexer

NASA National Aeronautics and Space Administration

NEI Nuclear Energy Institute

NPP nuclear power plant

NRC Nuclear Regulatory Commission

NRR Office of Nuclear Reactor Regulation (NRC)

OER operating experience review

OL operating license

PRA probabilistic risk assessment

PSF performance shaping factor

RCS reactor coolant system

xix
RG regulatory guide (NRC)

SAR safety analysis report

SAS secondary alarm station

SGTR steam generator tube rupture

SPDS safety parameter display system

SRP Standard Review Plan

SSAR standard safety analysis report

SSC structure, system, and component

TMI Three Mile Island

TR technical report

TSC technical support center

V&V verification and validation

xx
1 INTRODUCTION.

1.1 Background

Nuclear power plant (NPP) personnel play a vital role in the productive, efficient, and safe generation of

electric power. Operators monitor and control plant systems and components to verify their proper

functioning. Test and maintenance personnel help verify that plant equipment is functioning properly

and restore components when malfunctions occur; One of the important insights from studies of the

Three Mile Island (TMI), Chernoybl, and other NPP events is that errors resulting from human factors

deficiencies such as poor control room design, procedures, and training are a significant contributing

factor to NPP incidents and accidents.

Plant safety requires "defense in depth." Defense in depth includes the use of multiple barriers to prevent

the release of radioactive materials and uses a variety of programs to verify the integrity of barriers and

related systems (IAEA, 1988). These programs include conservative design, quality assurance,

administrative controls, and human factors. Human factors plays a significant role in supporting plant

safety and providing defense in depth.

The human factors engineering (HFE) staff of the Nuclear Regulatory Commission's (NRC's) Office of

Nuclear Reactor Regulation (NRR) evaluates HFE programs of applicants for construction permits (CPs),

operating licenses (OLs), standard design certifications (DCs), combined operating licenses (COLs), and

for license amendments. In this document, the term "applicant" is subsequently used to succinctly refer to

both licensees and applicants for design certification or plant licensing. The purpose of these reviews is

to support public health and safety by verifying that accepted HFE practices and guidelines are

incorporated into the applicant's HFE program. The HFE reviews include the design process, the final

design, its implementation, and ongoing performance monitoring.

General guidance to the staff for the performance of HFE reviews is in Chapter 18 of the Standard

Review Plan, NUREG-0800 (NRC, 2004). This document, the Human Factors Engineering Program

Reviewv Model (NUREG-071 1), supports the staff's HFE reviews by providing detailed review criteria.

The review process reflects a "top-down" approach to the conduct of an HFE program safety evaluation.

"Top-down" refers to the fact that the review approach starts at the "top" with high-level plant mission

goals. These goals are divided into the functions necessary to achieve the mission goals. Functions are

allocated to human and system resources and are separated into tasks for the purposes of specifying the

alarms, information, and controls that will be needed to accomplish function assignments. Tasks are

arranged into meaningful jobs and the human-system interfaces (HSIs), procedures, and training are

designed to support job task performance. The detailed design of the HSls, procedures, and training is the

bottom of the "top-down" process. The HFE safety evaluation is broad-based and includes normal and

emergency operations, maintenance, test, inspection, and surveillance activities.

NRC regulations in I OCFR Parts 50 and 52 require a variety of control and displays to be used by

operators. They also require a control room that reflects state-of-the-art human factors principles. This

document provides detailed guidance for NRC staff to use in verifying that these requirements are met.

NUREG-1649 (NRC, 2000) describes the NRC's Reactor Inspection and Oversight Program. This

program is outlined using "cornerstones" for reactor safety, radiation safety, and security. They are

initiating events, mitigation systems, barrier integrity, and emergency preparedness. Well-designed HSns,

procedures and training are important to optimizing each of these four cornerstones and the guidance

contained herein will help to verify that they are well-designed. They are also important to help verify

1
the radiation safety cornerstone goals of minimizing the radiation exposure of plant workers and the

general public during routine operations. Additionally, these guidelines could be applied to the plant

security program's central alarm station (CAS) and secondary alarm station (SAS) in order to improve

their functionality. In addition, there are three cross-cutting elements; one of which is human

performance, and supporting human performance is one of the principal aspects of this document.

An applicant's HFE program best provides reasonable assurance of plant safety when it is: (1) developed

by a qualified HFE design team with all skills, using an acceptable HFE program plan; (2) the result of

appropriate HFE studies and analyses that provide accurate and complete inputs to the design process and

inputs to verification and validation (V&V) assessment criteria; (3) designed using proven technology

based on human performance and task requirements 2 incorporating accepted HFE standards and

guidelines; (4) evaluated with a thorough V&V test program; (5) implemented in a manner that

effectively supports operations; and (6) monitored after operations to detect changes in safe performance

standards.

1.2 General Description of the Program Review Model

1.2.1 Review Elements

This document is organized into twelve elements arranged in four general activities as shown in Figure

1. 1. A brief description follows of the review objectives, acceptance criteria, and applicant products

reviewed for each element. The review elements are described in more detail in remaining sections of this

report.

Each element is divided into four sections: Background, Objective, Applicant Submittals, and Review

Criteria.

(1) Background - A brief explanation of the rationale and purpose is provided for each element.

(2) Objective - The review objective(s) of the element is defined.

(3) Applicant Submittals - In general, applicants are expected to submit two reports for NRC review:

* An implementation plan gives the applicant's proposed methodology for meeting the

acceptance criteria of the element. An implementation plan review gives the applicant

the opportunity to obtain staff review of and concurrence in the applicant's approach

before conducting the activities associated with the element. Such a review is desirable

from the staffs perspective because it provides the opportunity to resolve methodological

issues and provide input early in the analysis or design process when staff concerns can

more easily be addressed than when the effort is completed.

* A results summary report gives the results of the applicant's efforts related to each

element. The NRC staff will use the report as the main source of information for

assessing the applicant's efforts using the review criteria contained in this document.

This information may be submitted in a form other than in two reports. In some cases

2 The term "requirements" as used here and elsewhere in this document, refers to requirements that are established as

part of the design process. The term requirements is used in this context as a term-of-art. These are not "regulatory"

requirements. There are no regulatory requirements in this document, only review guidance.

2
an applicant may choose to provide this information in a single~report. It is also possible that, for

more complex elements such as HSI Design or V&V, more than two reports may be submitted in

order to address all criteria. When additional information, is needed, it is identified in this section

of each element. In addition to reports, the reviewer may review sample work products.

(4) Reviiew Criteria - This section contains the acceptance criteria for design process products and for

the final design review. Where appropriate, references to more detailed NRC guidance, e.g.,

NUREG-0700, are provided.

Planning and

Analysis

HFE Program

I Management I

Design Verification

and Validation

Implementation

and Operation

I

I Operating Experience

Review I _

I I Human-System Interface

IDesign

Functional

Requirements Analysis

and Function Allocation 1 I

Design Implementation 1

Task Anas

. .

-

I

Procedure Development

I .I: , Human Factors

. cation and ValidationI . .

I

Human Performance

Monitoring I .

. Training Program

I - Development I

I

Staffing & Qualification

I . .

i

Human Reliability

Analysis I .

Figure 1.1 HFE program review model review elements

1.2.2 Sources of Additional Information

Applicants should conduct their activities relative to each element using accepted HFE practices as

specified by applicable regulatory documents and codes, HFE standards, and guidelines. Therefore, for

most elements, a list of documents that contain generally recognized acceptable approaches is provided.

However, there are some qualifiers:

3
* References include documents that are periodically updated, such as NUREG-0700. The

reference contained herein is to the latest version of the document available. One should

consult the latest version of the document at the time of usage.

* Each individual document listed for a given element does not necessarily address all

aspects of that element. In the conduct of a review of each element, a combination of the

applicable sections of several of the identified documents may be appropriate.

* All of the documents referenced are not necessarily applicable to every design review.

* Where inconsistencies or contradictions within and between documents exist, they should

be resolved and justified on a case-by-case basis.

* It should not be inferred that the listed documents provide complete guidance for each

and every activity encompassed by the element. HFE is still an evolving discipline;

therefore, not all HFE activities are adequately covered in codes, standards, and

guidelines.

* Alternative approaches to those described in the HFE standards and guidelines may be

acceptable if they have defensible rationales. The NRC will evaluate alternative

approaches proposed by applicants on a case-by-case basis.

1.3 Applications

The review methodology presented in this document addresses the scope of NRC HFE reviews identified

in NUREG-0800 and can be used to review applications for construction permits, operating licenses,

standard design certifications, and combined operating licenses. This document can also be used to

review changes or modifications to licenses for nuclear power plants that include changes to human

actions, e.g., a license amendment request. The NRC, the nuclear industry, and the public, have moved to

a broader consideration of risk in many activities associated with NPPs. Therefore, risk importance

should be taken into account when deciding which particular items to review and the depth of reviews to

be undertaken. The purpose of these HFE reviews is to support safety by verifying that accepted human

factors engineering practices and guidelines are incorporated into the plant and program designs.

1.3.1 Application to Current Applicants Under 10 CFR Part 50

This document can be used by the NRC staff in the following circumstances:

Review of the HFE Aspects of a New Plant

If an applicant proposes to build a new plant under 10 CFR Part 50 requirements, an HFE review would

be performed in accordance with NUREG-0800. NUREG-0800 describes the staffs review activities to

verify that accepted human factors engineering principles are incorporated during the design process and

that the HFE reflects a state-of-the-art HFE design. NUREG-0800 provides a reference to NUREG-07 11

for detailed review.

Voluntary Modifications to Plants Affecting Personnel Performance

The NRC staff would use the guidance in this document to verify that voluntary modifications are

acceptable. NUREG-07 11 also contains risk considerations, which consider risk-informed evaluations

and regulation. The term modification is used generically to include any type of change or modernization

made to plant systems, HSIs, procedures, or training that may influence personnel performance. For

4
example, when computer-based HSI technology is integrated into existing control rooms, hybrid HSIs are

created; i.e., HSIs containing a mixture of conventional and advanced technology. An HFE review

should be conducted if the modification affects the role of personnel or the tasks they perform and is

potentially significant to plant safety. Modifications affect the role or tasks of perIsonnel if they impose

new or different demands.

A modification may be considered potentially significant to plant safety, if it is identified in 10 CFR

50.59(c)(2). In addition, the staff may use this document when evaluating changes to the plant under 10

CFR 50.59 that are related to the HSI and personnel performance.

Additional guidance related to 10 CFR 50.59 is provided in RG 1.187 (NRC, 2000) and Nuclear Energy

Institute (NEI) publication 96-07, entitled Guidelinesfor 10 CFR 50.59 Implementation, (NEI, 2000).

The following sections from NEI 96-07 identify plant modifications for which NUREGs-0711 and -0700

would be used:

In Section 4.3.2, Does the Activity Result in More Than a Minimal Increase in the

Likelihood of Occurrence of a Malfunction of an SSC (Structures, Systems, and

Components) Important to Safety, examples are provided of cases that would require

prior NRC approval. Example 7 states that, "The change would (permanently) substitute

manual action for automatic actions for performing UFSAR-described design functions"

(p. 48).

Section 4.3.6, Does the Activity Create a Possibilityfor a Malfunction of an SSC

Important to Safety with a Different Result, indicates that, "An example of a change that

would create the possibility for a malfunction with a different result is a substantial

modification to control station alarms, controls, or displays that are associated with SSCs

important to safety that creates a new or common cause failure that is not bounded by

previous analyses or evaluations" (p. 55).

Staff Investigation of Events Involving Human Performance

NRC inspectors would use selected elements of the guidance in this document to support the review of

those aspects of incidents that have important human performance contributions.

1.3.2 Application to Advanced NPP Applicants Under 10 CFR Part 52

Nuclear power plant (NPP) designers and vendors may submit designs of advanced standardized NPPs to

the NRC for review and approval under 10 CFR Part 52, "Early Site Permits; Standard Design

Certifications; and Combined Licenses for Nuclear Power Plants," (see Part 52 Subpart B, "Standard

Design Certifications"). To obtain a standard design certification under Part 52, applicants must submit

technical information which is technically relevant to the design. The technical information should

include the HFE program. However, since technology is continually advancing, details of the applicant's

HFE design might not be complete before the NRC issues a design certification. In such cases, reviews

under 10 CFR Part 52 would primarily focus on the HFE design process.

An applicant may apply for a COL to operate a standardized NPP that has already received a design

certification under 10 CFR Part 52. Portions of the facility design not covered by the design certification

are reviewed at the COL stage. Thus, for advanced NPPs, HFE reviews can occur at different points

within the 10 CFR Part 52 application and licensing process. These reviews can include the following:

5
* Design documentation, such as design-specific HFE guidance documents and specifications

* Prototype designs

* Completed designs

* HFE related inspections, tests, analyses, and acceptance criteria (ITAAC) (to provide

reasonable assurance that an as-built plant will be built and will operate to the standard design

certification)

* HFE related design acceptance criteria (DAC) (to verify that the applicant properly executes

the design process after certification)

For advanced NPPs (under 10 CFR Part 52), some HFE program elements may be deferred to the COL

applicant. However, all HFE review criteria are addressed before plant startup.

In addition, once an advanced reactor is operated under an NRC license, the staff can use the guidance in

this document, as per the SRP's reference to this document for detailed review guidance, to support

reviews of voluntary modifications to the advanced reactor, and to support investigations of events

involving human performance, as discussed in Section 1.3 above.

1.4 Graded Approach to Review

The guidance in the Standard Review Plan (SRP), NUREG-0800, indicates that for any given application,

the staff reviewers may select and emphasize particular aspects of each SRP section as is appropriate for

the application. Thus the level of staff review of an applicant's HFE design should reflect the unique

circumstances of the review.

The review methodology presented in this document is discussed generically. Therefore, the guidance

should be selectively applied to address the demands of each specific review. In its complete form, the

review process provides a comprehensive, detailed evaluation. This approach should be used for the

review of a new plant design or an extensive control room modernization. Under certain circumstances,

such as a preliminary review of an applicant's design prototype, or a focused review of one aspect of the

design that might be implicated in an incident involving human performance, the focus of the staffs'

review will be more limited. In that case, the reviewer should select the relevant criteria to meet the

demands of the review.

6
2 HFE PROGRAM MANAGEMENT

2.1 Background

The overall purpose of the HFE program review is to verify that:

* The applicant has integrated HFE into plant development, design, and evaluation.

* The applicant has provided HFE products (e.g., HSIs, procedures, and training) that allow

safe, efficient, and reliable performance of operation, maintenance, test, inspection, and

surveillance tasks.

* The HFE program and its products reflect "state-of-the-art human factors principles"

[10 CFR 50.34(f)(2)(iii)) and by 10 CFR 52.47(a)(1)(ii)] and satisfies all specific regulatory

requirements.

10 CFR 52.47 requires that applications for design certification of new reactor designs meet the

technically relevant portions of the TMI requirements contained in 10 CFR 50.34(f). Pertinent to this

document, 50.34 (f)(2)(iii) requires a control room design that reflects state-of-the-art human factors

principles. As further examples, 50.34 also requires: a safety parameter display system (SPDS) console,

automatic indication of bypassed and operable status of safety systems, and monitoring capability in the

control room of a variety of system parameters. 10 CFR 55.46 also requires a plant-referenced simulator

capability.

State-of-the-art human factors principles are defined as those principles currently accepted by human

factors practitioners. "Current" is defined with reference to the time when a program management or

implementation plan is prepared. "Accepted", is defined as a practice, method, or guide that is

(1) documented in the human factors literature within a standard or guidance document that has

undergone a peer-review process or (2) can be justified through scientific research and/or industry

practices.

To accomplish these purposes, an applicant should have an HFE program plan which is implemented by a

qualified HFE design team. The term "HFE design team" generically refers to the primary organization

or function within the organization that is responsible for HFE within the scope of the staff's review.

There is, however, no assumption that HFE is the responsibility of a single organization or that there is an

organizational unit called the HFE design team.

2.2 Objective

The objective of this review element is to verify that the applicant has an HFE design team with the

responsibility, authority, placement within the organization, and composition to verify that the design

commitment to HFE is met. Also, the team should be guided by a plan to provide reasonable assurance

that the HFE program is properly developed, executed, overseen, and documented. This plan should

describe the technical program elements verifying that all aspects of the HSI, procedures, and training are

developed, designed, and evaluated on the basis of accepted HFE principles. In addition, the HFE

program as a whole should appropriately consider and address the deterministic aspects of design, as

discussed in RG 1.174.

7
2.3 Applicant Submittals

The applicant should provide the following for staff review: HFE program plan describing the applicant's

HFE goals/objectives, technical program to accomplish the objectives, a system to track HFE issues, the

HFE design team, and the management and organizational structure to allow the technical program to be

accomplished.

2.4 Review Criteria

HFE Program Management review topics include:

* general HFE program goals and scope

* HFE team and organization

* HFE process and procedures

* HFE issues tracking

* technical program

2.4.1 General HFE Program Goals and Scope

(1) HFE Program Goals - The general objectives of the program should be stated in "humancentered" terms, which, as the HFE program develops, should be defined and used as a basis for

HFE test and evaluation activities. Generic "human-centered" HFE design goals include the

following:

* personnel tasks can be accomplished within time and performance criteria

* the HSIs, procedures, staffing/qualifications, training and management and organizational

support will support a high degree of operating crew situation awareness

* the plant design and allocation of functions will maintain operation vigilance and provide

acceptable workload levels i.e., to minimize periods of operator underload and overload

* the operator interfaces will minimize operator error and will provide for error detection and

recovery capability

(2) Assumptions and Constraints - An assumption or constraint is an aspect of the design, such as a

specific staffing plan or the use of specific HSI technology, that is an input to the HFE program

rather than the result of HFE analyses and evaluations. The design assumptions and constraints

should be clearly identified.

(3) Applicable Facilities - The HFE program should address the main control room, remote shutdown

facility, technical support center (TSC), emergency operations facility (EOF), and local control

stations (LCSs).

(4) Applicable HSIs, Procedures and Training - The applicable HSIs, procedures, and training

included in the HFE program should include all operations, accident management, maintenance,

test, inspection and surveillance interfaces (including procedures).

(5) Applicable Plant Personnel - Plant personnel who should be addressed by the HFE program

include licensed control room operators as defined in 10 CFR Part 55 and the following

8
categories of personnel defined by 10 CFR 50.120: nonlicensed operators, shift supervisor, shift

technical advisor, instrument and control technician, electrical maintenance personnel,

mechanical maintenance personnel, radiological protection technician, chemistry technician, and

engineering support personnel. In addition, any other plant personnel who perform tasks that are

directly related to plant safety should be addressed.

For plant modifications, the HFE program should include the involvement of plant personnel to

provide reasonable assurance that the following are considered from a user's perspective in

establishing modification requirements and evaluating the design process's outputs:

* user's understanding of how plant systems are structured and behave

* task demands and constraints of the existing work environment

* existing work processes

* organizational goals that affect the implementation and use of the modification

(6) Effects of Modiflcations on Personnel Performance - The goals of the HFE program should

address the need to consider the effects that the modification may have on the performance of

personnel. The transition from the existing plant configuration to the modification configuration

can pose demands on human performance that differ from either the initial or final configurations.

Therefore, it should be planned so it places minimal demands for adapting to the change. The

considerations should include the following:

* planning the installation to minimize disruptions to work

* coordinating training and procedure modifications with implementing the modification to

provide reasonable assurance that both accurately reflect its characteristics.

* conducting training to maximize personnel's knowledge and skill with the new design before

its implementation

2.4.2 HFE Team and Organization

(I) Responsibility - The team should be responsible (with respect to the scope of the HFE program)

for (a) the development of all HFE plans and procedures; (b) the oversight and review of all HFE

design, development, test, and evaluation activities; (c) the initiation, recommendation, and

provision of solutions through designated channels for problems identified in the implementation

of the HFE activities; (d) verification of implementation of team recommendations; (e) assurance

that all HFE activities comply with the HFE plans and procedures; and (f) scheduling of activities

and milestones.

(2) Organizational Placement andAuthority - The primary HFE organization(s) or function(s) within

the organization of the total program should be identified, described, and illustrated (e.g., charts

to show organizational and functional relationships, reporting relationships, and lines of

communication). When more than one organization is responsible for HFE, the lead

organizational unit responsible for the HFE program plan should be identified. The team should

have the authority and organizational placement to provide reasonable assurance that all its areas

of responsibility are accomplished and to identify problems in the implementation of the overall

plant design. The team should have the authority to control further processing, delivery,

9
installation, or use of HFE products until the disposition of a nonconformance, deficiency, or

unsatisfactory condition has been achieved.

(3) Composition - The HFE design team should include the expertise described in the Appendix.

(4) Team Staffing - Team staffing should be described in terms ofjob descriptions and assignments

of team personnel.

2.4.3 HFE Process and Procedures

(1) General Process Procedures - The process through which the team will execute its responsibilities should be identified. The process should include procedures for:

* assigning HFE activities to individual team members

* governing the internal management of the team

* making management decisions regarding HFE

* making HFE design decisions

* governing equipment design changes

* design team review of HFE products

(2) Process Management Tools - Tools and techniques (e.g., review forms) to be utilized by the team

to verify they fulfill their responsibilities should be identified.

(3) Integration ofHFE and Other Plant Design Activities - The integration of design activities should

be identified, that is, the inputs from other plant design activities to the HFE program and the

outputs from the HFE program to other plant design activities. The iterative nature of the HFE

design process should be addressed.

(4) HFE Program Milestones - HFE milestones should be identified so that evaluations of the

effectiveness of the HFE effort can be made at critical check points and the relationship to the

integrated plant sequence of events is shown. A relative program schedule of HFE tasks showing

relationships between HFE elements and activities, products, and reviews should be available for

review.

(5) HFE Documentation - HFE documentation items should be identified and briefly described along

with the procedures for retention and access.

(6) Subcontractor HFE Efforts - HFE requirements should be included in each subcontract and the

subcontractor's compliance with HFE requirements should be periodically verified.

2.4.4 HFE Issues Tracking

(1) Availability - A tracking system should be available to address human factors issues that are

(a) known to the industry (defined in the Operating Experience Review element, see Section 3)

and (b) identified throughout the life cycle of the HFE aspects of design, development, and

evaluation. Issues are those items that need to be addressed at some later date and thus need to be

tracked to provide reasonable assurance that they are not overlooked. It is not necessary to

establish a new system to track HFE issues that is independent from the rest of the design effort.

10
An existing tracking syste m may be adapted to'serve this purpose (such as a plant's corrective

action program, CAP).

(2) Method - The method should document and track HFE issues from identification until the

potential for negative effects on human performance has been reduced to an acceptable level.

(3) Documentation - Each issue or concern that meets or exceeds the threshold established by the

design team should be entered into the system when first identified, and each action taken to

eliminate or reduce the issue or concern should be thoroughly documented. The final resolution

of the issue should be documented in detail, along with information regarding design team

acceptance.

(4) Responsibility - When an issue is identified, the tracking procedures should describe individual

responsibilities for issue logging, tracking and resolution, and resolution acceptance.

2.4.5 Technical Program

(1) The general development of implementation plans, analyses, and evaluation of the following

should be identified and described:

* operating experience review

* functional requirements analysis and function allocation

* task analysis

* staffing and qualifications

* human reliability analysis:

* HSI design

* procedure design

* training design

* human factors verification and validation

* design implementation

* human performance monitoring

(2) The HFE requirements imposed on the design process should be identified and described. The

standards and specifications that are sources of HFE requirements should be listed.

(3) HFE facilities, equipment, tools, and techniques (such as laboratories, simulators, rapid

prototyping software) to be utilized in the HFE program should be specified.

(4) The applicant should provide assurance in the HFE plan that a plant modification meets current

regulations, except where specific exemptions are requested under 1O CFR 50.12 or 1O CFR

2.802. An exemption might be granted under one or more of the following regulations: 10 CFR

20,10CFR50 Appendix A, Criterion 19, and 10 CFR50 Appendices C through R.

(5) The applicant should provide assurance in the HFE plan that a modification'does not compromise

defense-in-depth. Defense-in-depth'is one of the fundamental principles upon which the plant

was designed and built. Defense-in-depth uses multiple means to accomplish safety functions

11
and to prevent the release of radioactive materials. Defense-in-depth is important in accounting

for uncertainties in equipment and human performance, and for ensuring some protection remains

even in the face of significant breakdowns in particular areas. Defense-in-depth may be changed

but should be maintained overall. Important aspects of defense-in-depth are identified in RG

1.174, and include:

* A reasonable balance is preserved among prevention of core damage, prevention of

containment failure, and consequence mitigation.

* There is no over-reliance on programmatic activities to compensate for weaknesses in plant

design. This may be pertinent to changes in credited human actions (HAs).

* System redundancy, independence, and diversity are preserved commensurate with the

expected frequency, consequences of challenges to the system, and uncertainties (e.g., no risk

outliers).

* Defenses against potential common cause failures are preserved, and the potential for the

introduction of new common cause failure mechanisms is assessed. Caution should be

exercised in crediting new HAs to verify that the possibility of significant common cause

errors is not created.

* Independence of barriers is not degraded.

* Defenses against human errors are preserved. For example, establish procedures for a second

check or independent verification for risk-important HAs to determine that they have been

performed correctly.

* The intent of the General Design Criteria (GDC) in Appendix A to 10 CFR Part 50 is

maintained. GDC that may be relevant are 3 - Fire Protection, 13 - Instrumentation and

Control, 17 - Electric Power Systems, 19 - Control Room, 34 - Residual Heat Removal, 35 -

Emergency Core Cooling System, 38 - Containment Heat Removal, and 44 - Cooling Water.

* Safety margins often used in deterministic analyses to account for uncertainty and provide an

added margin to provide adequate assurance that the various limits or criteria important to

safety are not violated. Such safety margins are typically not related to HAs, but the reviewer

should take note to see if there are any that may apply to the particular case under review. It

is also possible to add a safety margin (if desired) to the HA by demonstrating that the action

can be performed within some time interval (or margin) that is less than the time identified by

the analysis.

2.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

IEC 964: Design for Control Rooms of Nuclear Power Plants (International Electrochemical

Commission, 1989).

IEEE Std. 1023-1988: IEEE Guide to the Application of Human Factors Engineering to Systems,

Equipment, and Facilities of Nuclear Power Generating Stations (Institute of Electrical and

Electronics Engineers, 1988).

ISO 11064- 1: Ergonomic Design of Control Centres -- Part 1: Principles for the Design of

Control Centres (International Standards Organization, 2000).

12
Regulatory Guide 1.174: An approach for using probabilistic risk assessment in risk-informed

decisions on plant-specific changes to the licensing basis (NRC, 1998).

U.S. Code ofFederal Regulations, Part 55, "Operator's Licenses," Title 1O, "Energy."

13
3 OPERATING EXPERIENCE REVIEW

3.1 Background

Applicant's should provide administrative procedures of evaluating operating, design and construction

experience and for ensuring that applicable important industry experiences will be provided in a timely

manner to those designing and construction the plant [10 CFR 50.34(f)(3)(i)]. The main purpose of

conducting an operating experience review (OER) as part of the HFE review is to identify HFE-related

safety issues. The OER should provide information on the past performance of predecessor designs. In

the case of new plants this may be earlier designs on which the new design is based. In the case of plant

modifications, it may be the design of the systems being changed. The issues and lessons learned from

operating experience provide a basis for improving the plant design in a timely way; i.e., at the beginning

of the design process.

The resolution of OER issues may involve function allocation, changes in automation, HSI equipment

design, procedures, training, and so forth. Thus, negative features encountered in previous designs can be

identified and analyzed so that they are avoided in the development of the current system and positive

features can be retained.

OER information contributes to other review elements. These inputs are summarized in Table 3.1. As

indicated in the table, OER can contribute to review and evaluation considerations as well as system

design considerations. 'For example, OER can be used in the selection of specific failure scenarios to

incorporate in validation testing and can be used as a basis to select specific performance measures for the

evaluation (e.g., to measure an aspect of human performance identified in OER as being problematic).

Table 3.1 The role of operating experience review in the HFE program

HFE ELEMENT

Functional Requirements Analysis

and Function Allocation

Task Analysis, Human Reliability

Analysis, and Staffing/Qualifications

Human-System Interface, Procedures,

and Training Development

Human Factors Verification and Valid

OER CONTRIBUTION

* Basis for initial requirements

* Basis for initial allocations

* Identification of need for modifications

* Risk-important human actions and errors

* Problematic operations and tasks

* Staffing shortfalls

* Trade study evaluations

* Potential design solutions

* Potential design issues

lation * Tasks to be evaluated

* Event and scenario selection

* Performance measure selection

* Issue resolution verification

15
3.2 Objective

The objective of reviewing operating experience is to verify that the applicant has identified and analyzed

HFE-related problems and issues in previous designs that are similar to the current design under review.

In this way, negative features associated with predecessor designs may be avoided in the current one

while retaining positive features.

3.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for conducting a review of operating experience. Upon completion of the applicant's

OER, a results summary report should be submitted so that the staff can review the identification and

analysis of HFE-related problems and issues using the criteria provided in Section 3.4 below. In addition,

the reviewer may also audit the issue tracking system for examination of OER issue treatment.

3.4 Review Criteria

3.4.1 Scope

(1) Predecessor/Related Plants and Systems - The review should include information pertaining to

the human factors issues related to the predecessor plant(s) or highly similar plants and plant

systems. For a review of plant modifications, the scope of the OER should be focused to provide

information relevant to the plants' systems, HSIs, procedures, or training that are being modified.

It should address the operating experience of the plant that will be modified, including

experiences with the systems that will be modified and with technologies that are similar to those

under consideration for it. Some useful information may be found in the plant's CAP. Also,

when personnel are unfamiliar with the proposed technology, attention should be paid to the

operating experience of other plants that already have the technology.

(2) Recognized Industry HFE Issues - NUREG/CR-6400 (Higgins and Nasta, 1996) issues should be

addressed. The issues are organized into the following categories:

* unresolved safety issues/generic safety issues

* TMI issues

* NRC generic letters and information notices

* reports of the former NRC Office for Analysis and Evaluation of Operational Data

* low power and shutdown operations

* operating plant event reports

(3) Related HFE Technology - The OER should address related HFE technology. For example, if

touch screen interfaces or computerized procedures are planned, HFE issues associated with their

use should be reviewed.

(4) Issues Identified by Plant Personnel - Personnel interviews should be conducted to determine

operating experience related to predecessor plants or systems. The following topics should be

included in the interviews as a minimum:

* Plant Operations

- normal plant evolutions (e.g., startup, full power, and shutdown)

16
- instrument failures [e.g., safety-related system logic and control unit, fault

tolerant controller (nuclear steam supply system), local "field unit" for

multiplexer (MUX) system, MUX controller (balance of plant), break in MUX

line]

- HSI equipment and processing failure (e.g., loss of video display units, loss of

data processing, loss of large overview display)

transients (e.g., turbine trip, loss of offsite power, station blackout, loss of all

feedwater, loss of service water, loss of power to selected buses or control room

(CR) power supplies, and safety/relief valve transients)

- accidents (e.g., main steam line break, positive reactivity addition, control rod

insertion at power, control rod ejection, anticipated transients without scram

(ATWS), and various-sized loss-of-coolant accidents (LOCA))

- reactor shutdown and cooldown using remote shutdown system

HFE Design Topics

- alarm and annunciation

- display

- control and automation

- information processing and job aids

- real-time communications with plant personnel and other organizations

- procedures, training, staffing/qualifications, and job design

(5) Risk-Important Human Actions - The OER should identify risk-important HAs that have been

identified as different or where errors have occurred. The human actions should be identified as

requiring special attention during the design process to lessen their probability.

3.4.2 Issue Analysis, Tracking, and Review

(1) Analysis Content - The issues should be analyzed with regard to the identification of

* human performance issues, problems, and sources of human error

* design elements that support and enhance human performance

(2) Documentation - The analysis of operating experience should be documented in an evaluation

report.

(3) Incorporation Into the Tracking System - Each operating experience issue determined to be

appropriate for incorporation in the design (but not already addressed in the design) should be

documented in the issue tracking system.

3.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

IAEA Safety Series No. 75-INSAG-3: Basic Safety Principles for Nuclear Power Plants

(International Atomic Energy Agency (1988).

17
IEEE Std. 1023-1988: IEEE Guide to the Application of Human Factors Engineering to Systems,

Equipment, and Facilities of Nuclear Power Generating Stations (Institute of Electrical and

Electronics Engineers, 1988).

IEEE Std. 845-1999: IEEE Guide to the Evaluation of Human-System Performance in Nuclear

Power Generating Stations (Institute of Electrical and Electronics Engineers, 1999).

NUREG/CR-6400: HFE Insights For Advanced Reactors Based Upon Operating Experience

(Higgins and Nasta, 1996).

18
4 FUNCTIONAL REQUIREMENTS ANALYSIS

AND FUNCTION ALLOCATION

4.1 Background

Functional requirements analysis is the identification of functions that must be performed to satisfy plant

safety objectives; that is, to prevent or mitigate the consequences of postulated accidents that could

damage the plant or cause undue risk to the health and safety of the public. A functional requirements

analysis is conducted to (1) determine the objectives, performance requirements, and constraints of the

design, (2) define the high-level functions that have to be accomplished to meet the objectives and desired

performance, (3) define the relationships between high-level functions and plant systems (e.g., plant

configurations or success paths) responsible for performing the function, and (4) provide a framework for

understanding the role of controllers (whether personnel or system) for controlling the plant.

Function allocation is the analysis of the requirements for plant control and the assignment of control

functions to (1) personnel (e.g., manual control), (2) system elements (e.g., automatic control and passive,

self-controlling phenomena), and (3) combinations of personnel and system elements (e.g., shared control

and automatic systems with manual backup). Plant safety and reliability are enhanced by exploiting the

strengths of personnel and system elements, including improvements that can be achieved through the

assignment of control to these elements with overlapping and redundant responsibilities. In addition to

technological and economic considerations, function allocation should be based on HFE principles using

a structured and well-documented methodology that seeks to provide personnel with logical, coherent,

and meaningful tasks. It should not be based solely on technology considerations that allocate to plant

personnel everything the designers cannot automate. Such an approach results in an ad hoc set of

activities that may negatively affect operator performance.

The purpose of the functional requirements analysis and function allocation review is to verify that the

plant's safety functions have been defined and that the allocation of those functions to human and system

resources has resulted in a role for personnel that takes advantage of human strengths and avoids human

limitations.

Functional requirements analysis is not only a consideration for new designs. Plant modifications can

change the level of automation of the original design; e.g., full-range feedwater control systems, which in

turn effects the roles and responsibilities of plant personnel.

4.2 Objective

The objective of the functional requirements analysis and functional allocation review is to verify that the

applicant has (I) defined the plant's functions that must be performed to satisfy plant safety objectives,

and (2) that the allocation of those functions to human and system resources has resulted in a role for

personnel that takes advantage of human strengths and avoids human limitations.

4.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for conducting functional requirements analysis and functional allocation. Upon

completion of the applicant's efforts, a results summary report should be submitted so that the staff can

review the applicant's definition of the plant's functions and the allocation of functions to human and

system resources using the criteria provided in Section 4.4 below.

19
4.4 Review Criteria

(1) Functional requirements analysis and function allocation should be performed using a structured,

documented methodology reflecting HFE principles. An example functional allocation process

and considerations is shown in Figure 4.1. The functional requirements analysis and function

allocation may be graded based on:

* the degree to which the functions of the new design differ from those of the predecessor

* the extent to which difficulties related to plant functions were identified in the plant's

operating experience and will be addressed in the new design.

Identification of

Functions to be Performed

i

Specification of

Functional Requirements

-T r

Analysis of

Function Allocation

* Performance demands

* Human and machine capabilities/limitations

* Existing practices

* Operating experience

* Regulatory requirements

* Technical feasibility

* Cost

t , f

Human Control

(Manual)

Machine Control

(Automatic)

Shared Control _ (Human and Machine)

4'

Design Development

and Modification

Function Verification

Figure 4.1 Allocation of functions to human and machine resources

(2) The functional requirements analysis and function allocation should be kept current over the life

cycle of design development and held until decommissioning so that it can be used as a design

20
basis when modifications are considered. Control functions should be re-allocated in an iterative

manner, in response to developing design specifics, operating experience, and the outcomes of

ongoing analyses and trade studies.

(3) A description of the functions and systems should be provided along with a comparison to the

reference plants/systems, i.e., the previous plants or plant systems on which the new system is

based. This description should identify differences that exist between the proposed and reference

plants/systems. Safety functions (e.g., reactivity control) include functions needed to prevent or

mitigate the consequences of postulated accidents that could cause undue risk to the health and

safety of the public. For each safety function, the set of plant system configurations or success

paths that are responsible for or capable of carrying out the function should be clearly defined.

Function decomposition should start at "top-level" functions where a very general picture of

major functions is described, and continue to lower levels until a specific critical end-item

requirement emerges (e.g., a piece of equipment, software, or HA). The functional

decomposition should address the following levels

* high-level functions [e.g., maintain reactor coolant system (RCS) integrity] and critical

safety functions (e.g., maintain RCS pressure control)

* specific plant systems and components

(4) A description should be provided for each high-level function which includes:

* purpose of the high-level function

* conditions that indicate that the high-level function is needed

* parameters that indicate that the high-level function is available

* parameters that indicate the high-level function is operating (e.g., flow indication)

* parameters that indicate the high-level function is achieving its purpose (e.g., reactor

vessel level returning to normal)

* parameters that indicate that operation of the high-level function can or should be

terminated

Note that parameters may be described qualitatively (e.g., high or low). Specific data values or

setpoints are not necessary at this stage.

(5) The technical basis for modifications to high-level functions in the new design (compared to the

predecessor design) should be documented.

(6) The technical basis for all function allocations should be documented; including the allocation

criteria, rationale, and analyses method. The technical basis for functional allocation can be any

one or combination of the evaluation factors (see Fig 4;3). For example, the performance

demands to successfully achieve the function, such as degree of sensitivity needed, precision,

time, or frequency of response, may be so stringent that it would be difficult or error prone for

personnel to accomplish. This would establish a basis for automation (assuming acceptability of

other factors, such as technical feasibility or cost).

(7) The OER should be used to identify modifications to function allocations, if necessary.

If problematic OER issues are identified, then an analysis should be performed to (a) justify the

original analysis of the function, (b) justify the original human-machine allocation, and

(c) identify solutions such as training, personnel selection, and procedure design that will be

implemented to address the OER issues.

21
(8) The allocation analysis should consider not only the primary allocations to personnel, but also

their responsibilities to monitor automatic functions and to assume manual control in the event of

an automatic system failure.

(9) A description of the integrated personnel role across functions and systems should be provided in

terms of personnel responsibility and level of automation.

(10) The functional requirements analysis and function allocation should be verified:

* all the high-level functions necessary for the achievement of safe operation are identified.

* all requirements of each high-level function are identified.

* the allocations of functions result in a coherent role for plant personnel

(11) When the analyses address plant modifications, the following considerations should also be

addressed:

* Functional requirements analyses for modifications that are likely to change existing

safety functions, introduce new functions for systems supporting safety functions, or

involve unclear functional requirements that may be important to safety. The functional

requirements analysis should address new functions resulting from changes in the degree

of integration between plant systems. For example, installing higher-level automation

may bring systems that were formerly controlled separately under a single controller.

Also, the modifications may change the degree to which different plant systems share

common resources (e.g., power sources, cooling water, and data-transmission buses).

These may be important in diagnosing malfunctions or planning responses. The

functional requirements analyses should be revised and updated to reflect the

modification; the scope of the analyses may be restricted to functions related to the

modification.

* Function allocation analyses for modifications that are likely to change the allocation

between personnel and plant systems of functions important to safety. The analyses

should be revised and updated to reflect the modification; their scope may be restricted to

functions involving the modification.

* A change in an operator's role due to a modification should be examined within the

context of its effects on the operator's overall responsibilities. Increases in certain task

demands may affect the ability of the operator to carry out others that are risk-important.

4.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

HWR-639: FAME-A Function Allocation Design Tool (Bye and Hoffinan, 2001).

IAEA-TECDOC-668: The Role ofAutomation and Humans in Nuclear Power Plants

(International Atomic Energy Agency, 1992).

IEC 964: Design for Control Rooms of Nuclear Power Plants (International Electrochemical

Commission, 1989).

NASA Technical Memo No. 103885: Human-Centered Aircraft Automation: A Concept and

Guidelines (Billings, 1991).

22
NUREG/CR-2623: The Allocation of Functions in Man-Machine Systems: A Perspective and

Literature Review ( Price, et al., 1982).

NUREG/CR-333 1: A MethodologyforAllocation of Nuclear Power Plant Control Functions to

Human and Automated Control (Pulliam et al., 1983).

23
5 TASK ANALYSIS

5.1 Background

The functions allocated to plant personnel define their roles and responsibilities. Human actions are

performed to accomplish these functions. HAs can be further divided into tasks. A task is a group of

related activities that have a common objective or goal. Task analysis is the identification of

requirements for accomplishing these tasks, i.e., for specifying the requirements for the displays, data

processing, controls, and job support aids needed to accomplish tasks. As such, the results of task

analysis are identified as inputs in many HFE activities; e.g., it forms the basis for

* staffing, qualifications, job design, and training

* HSIs, procedures, and training program design

I task support verification criteria definition (see Human Factors Verification and

Validation in Section 11).

5.2 Objective

The objective of this review is to verify that the applicant's task analysis identifies the specific tasks that

are needed for function accomplishment and their information, control and task-support requirements.

5.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for conducting task analysis. Upon completion of the applicant's efforts, a results

summaty report should be submitted so that the staff can review the applicant's identification of tasks that

are needed for function accomplishment and the information, control and task-support requirements using

the criteria provided in Section 5.4 below.

5.4 Review Criteria

(1) The scope of the task analysis should include:

* selected representative and important tasks from the areas of operations, maintenance,

test, inspection, and surveillance

* full range of plant operating modes, including startup, normal operations, abnormal and

emergency operations, transient conditions, and low-power and shutdown conditions

* HAs that have been found to affect plant risk by means of PRA importance and

sensitivity analyses should also be considered risk-important. Internal and external

initiating events and actions affecting the PRA Level I and II analyses should be

considered when identifying risk-important actions

* where critical functions are automated, the analyses- should consider all human tasks

including monitoring of the automated system and execution of backup actions if the

system fails.

(2) Tasks should be linked using a technique such as operational sequence diagrams. Task analyses

should begin on a gross level and involve the development of detailed narrative descriptions of

what personnel have to do. The analyses should define the nature of the input, process, and

output needed by and of personnel. Detailed task descriptions should address (as appropriate) the

topics listed in Table 5.1

25
Table 5.1 Task considerations

Type of Information

Information Requirements

Decisionmaking Requirements

ResponseRequirements

Communication Requirements

Workload

Example

alarms and alerts

parameters (units, precision, and accuracy)

feedback needed to indicate adequacy of actions taken

decisions type (relative, absolute, probabilistic)

evaluations to be performed

type of action to be taken

task frequency, tolerance and accuracy

time available and temporal constraints (task ordering)

physical position (stand, sit, squat, etc.)

biomechanics

- movements (lift, push, turn, pull, crank, etc.)

- forces needed

personnel communication for monitoring information or control

cognitive

physical

overlap of task requirements (serial vs. parallel task elements)

Task Support Requirements

Workplace Factors

Situational and Performance

Shaping Factors

Hazard Identification

special and protective clothing

job aids or reference materials needed

tools and equipment needed

ingress and egress paths to the worksite

workspace envelope needed by action taken

typical and extreme environmental conditions, such as lighting, temp, noise

stress

reduced manning

identification of hazards involved, e.g., potential personal injury

(3) The task analysis should be iterative and become progressively more detailed over the design

cycle. It should be detailed enough to identify information and control requirements to enable

specification of detailed requirements for alarms, displays, data processing, and controls for

human task accomplishment.

26
(4) The task analysis should address issues such as:

* the number of crew members

* crew member skills

* allocation of monitoring and control tasks to the (a) formation of a meaningful job and

(b) management of crew member's physical and cognitive workload.

(5) The task analysis results should be used to define a minimum inventory of alarms, displays, and

controls necessary to perform crew tasks based on both task and instrumentation and control

requirements.

(6) The task analysis results should provide input to the design of HSIs, procedures, and personnel

training programs.

(7) The following considerations should be addressed for plant modifications that are likely to affect

HAs previously identified as risk-important, cause existing HAs to become risk-important, or

create new actions that are risk-important.

* The tasks analyses should be revised and updated to reflect requirements of the

modification; the scope should include tasks involving the modification and its

interactions with the rest of the plant, including those resulting from functions addressed

in the analyses of functional requirements and function allocation. For maintenance,

tests, inspections, and surveillances, attention should be given to risk-important actions

that are new or supported by new technologies (e.g., new capabilities for on-line

maintenance).

* The task analysis should identify the design characteristics of the existing HSIs that

support the performance of experienced personnel (e.g., support high levels of

performance during demanding situations). They may include the spatial arrangement of

control- and display-devices and the ability to adjust controls and displays to deal with

special tasks. These design characteristics should be considered in developing new

design requirements. That is, the new design should have features performing similar

functions, or should eliminate the need for them by performing these functions

differently. In addition, the task analysis should identify and examine adjustments made

to the HSIs by users, such as notes and external memory-aids, which suggest that the

users' needs may not be fully met by its current design. All task demands should be

adequately addressed by the new design requirements. Design features identified during

OERs should be considered in these analyses.

5.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

A Guide to TaskAnalysis (Kirwan and Ainsworth, 1992).

Cognitive Task Analysis (Shraagen, Chipman, and Shalin, 2000).

Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work

(Vicente, 1999).

IEC 964: Design for Control Rooms of Nuclear Power Plants (International Electrotechnical

Commission, 1989).

27
NUREG/CR-3371: TaskAnalysis of Nuclear Power Plant Control Room Crews (Burgy et al.,

1983).

28
6 STAFFING AND QUALIFICATIONS

6.1 Background

Plant staff and their qualifications arean important consideration throughout the design process. Initial

staffing levels may be established based on experience with previous plants, staffing goals (such as for

staffing reductions), initial analyses, and government regulations. Staffing levels are also an important

consideration when plant modifications are designed. For example, when plant modifications impact

credited operator actions, the applicant may review the staffing needed to successfully accomplish that

action. Many such actions require teamwork and communication between control room staff, auxiliary

operators, and other plant staff. The NRC reviews the applicant's analysis used to determine the staffing

requirements for accomplishing that action.

As a second example, when a plant and control room modernization program is proposed and the

technology underlying control room operations changes significantly, the applicants evaluate the impact

of the change on the qualifications of plant staff. Here too, this element is used to review the applicant's

analysis.

The review criteria in this element address these situations.

6.2 Objective

The objective of the staffing review is to verify that the applicant has systematically analyzed the need for

the number and qualifications of personnel and has demonstrated a thorough understanding of task

requirements and regulatory requirements.

6.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for staffing and qualifications analysis. -Upon completion of the applicant's efforts, a

results summary report should be submitted so that the staff can review the applicant's evaluation of the

requirements for the number and qualifications of personnel using the criteria provided in Section 6.4

below.

6.4 Review Criteria

(1) Staffing and qualifications should address applicable guidance in NUREG-0800 Section 13.1 and

10 CFR 50.54.

(2) The staffing analysis should determine the number and background of personnel for the full range

of plant conditions and tasks including operational tasks (normal, abnormal, and emergency),

plant maintenance, and plant surveillance and testing. The scope of personnel that should be

considered is identified in the HFE Program Management element (see Section 2.4.1, Criterion

5).

(3) The staffing analysis should be iterative; that is, initial staffing goals should be reviewed and

modified as the analyses associated with other elements are completed.

(4) The basis for staffing and qualifications should be modified to address these issues:

Operating Experience Review

29
- operational problems and strengths that resulted from staffing levels in

predecessor systems

- initial staffing goals and their bases including staffing levels of predecessor

systems and a description of significant similarities and differences between

predecessor and current systems

- staffing considerations described in NRC Information Notice 95-48, "Results of

Shift Staffing Study"

- staffing considerations described in NRC Information Notice 97-78, "Crediting

of Operator Actions in Place of Automatic Actions and Modifications of

Operator Actions, Including Response Times"

* Functional Requirements Analysis and Function Allocation

- mismatches between functions allocated to personnel and their qualifications

- changes the roles of personnel due to plant system and HFE modifications

* Task Analysis

- the knowledge, skills, and abilities needed for personnel tasks addressed by the

task analysis

- personnel response time and workload

- personnel communication and coordination, including interactions between them

for diagnosis, planning, and control activities, and interactions between personnel

for administrative, communications, and reporting activities

- the job requirements that result from the sum of all tasks allocated to each

individual both inside and outside the control room

- decreases in the ability of personnel to coordinate their work due to plant and

HFE modifications

- availability of personnel considering other activities that may be ongoing and for

which operators may take on responsibilities outside the control room (e.g., fire

brigade)

- actions identified in 10 CFR 50.47, NUREG-0654, and procedures to meet an

initial accident response in key functional areas as identified in the emergency

plan

- staffing considerations described by the application of ANSI/ANS 58.8-1994,

"Time Response Design Criteria for Safety-Related Operator Actions"

* Human Reliability Analysis

- the effect of overall staffing levels on plant safety and reliability

- the effect of overall staffing levels and crew coordination for risk-important HAs

- the effect of overall staffing levels and the coordination of personnel on human

errors associated with the use of advanced technology

* HSI Design

- staffing demands resulting from the locations and use (especially concurrent use)

of controls and displays

- coordinated actions between individuals

- decreases the availability or accessibility of information needed by personnel due

to plant system and HFE modifications

30
- the physical configuration of the control room and control consoles

- the availability of plant information from individual workstations and group-view

interfaces

Procedure Development

- staffing demands resulting from requirements for concurrent use of multiple

procedures

- personnel skills, knowledge, abilities, and authority identified in procedures

Training Program Development

- crew coordination concerns that are identified during the development of training

6.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

10 CFR 50.47: U.S. Code of Federal Regulations, Part 50, "Domestic Licensing of Production

and Utilization Facilities," Title 10, "Energy."

10 CFR 50.54: US. Code of Federal Regulations, Part 50, "Domestic Licensing of Production

and Utilization Facilities," Title 10, "Energy."

10 CFR 26: U.S. Code of Federal Regulations, Part 26, "Fitness for Duty Programs," Title 10,

"Energy."

ANSI/ANS 58.8: Time Response Design Criteria for Safety-Related OperatorActions (American

Nuclear Society, 1994).

ANSI/ANS 3.1: Selection, Qualification, and Training of Personnelfor Nuclear Power Plants,

(American Nuclear Society, 1993).

ANSI/ANS 3.5: Nuclear Power Plant Simulatorsfor Use in Operator Training (American

Nuclear Society, 1998).

Generic Letter No. 82-12: Policy on Factors Causing Fatigue of Operating Personnel at Nuclear

Reactors (NRC, 1982).

Information Notice 95-48: Results of Shift Staffing Study (NRC, 1995).

Information Notice 97-78: Crediting of OperatorActions in Place ofAutomaticActions and

Modifications of OperatorActions, Including Response Times (NRC, 1997).

NUREG-0654: Criteriafor Preparation and Evaluation of Radiological Emergency Response

Plans and Preparedness in Support of Nuclear Power Plants (NRC, 1980).

NUREG-0737 and Supplements: Clarification of TMIAction Plan Requirements (NRC, 1980).

NUREG-0800: Standard Review Plan, Sections 13.1.1 - 13.1.3 (NRC, 1999).

31
Regulatory Guide 1.114: Guidance to Operators at the Controls and to Senior Operators in the

Control Room of a Nuclear Power Unit (NRC, 1986).

Regulatory Guide 1.149: Nuclear Power Plant Simulators for Use in Operator Training (NRC,

2001).

Regulatory Guide 1.8: Personnel Selection and Training (NRC, 2000).

32
7 HUMAN RELIABILITY ANALYSIS

7.1 Background

Human reliability analysis (HRA) is an integral activity of a complete probabilistic risk assessment

(PRA). A PRA is submitted in accordance with current NRC requirements, if applicable. Human

reliability analysis (HRA) seeks to evaluate the potential for, and mechanisms of, human error that may

affect plant safety. Thus, it is an essential element in achieving the HFE design goal of providing a

design that will minimize personnel errors, allow their detection, and provide recovery capability.

The HRA should be conducted as an integrated activity to support both the HFE design and PRA

activities. Figure 7.1 illustrates the relationship between the PRA/HRA and the rest of the HFE program,

including the concept of an initial PRA/HRA and then a final one at completion of design. The quality of

the HRA depends in large part on the analyst's understanding of personnel tasks, the information related

to those tasks, and the factors that influence human performance of those tasks. The development of

information to facilitate the understanding of causes and modes of human error is an important human

factors activity. The HRAs should make use of descriptions and analyses of operator functions and tasks

as well as the operational characteristics of HSIs. HRA can provide valuable insight into desirable

characteristics of the HSI design. Consequently,- the HFE design effort should give special attention to

those plant scenarios, risk-important human actions, and HSIs that have been identified by PRA/HRA as

being important to plant safety and reliability.

The discussions in the remainder of this HRA element should be applied as appropriate to the earliest

PRA/HRA (depending on the amount of design information that is available) and applied in full to the

final PRA/HRA. By developing an understanding of the causes, modes, and probabilities of human error,

the HRA can provide valuable insights into the desirable characteristics of the design; consequently,

special attention should be paid to those scenarios, HAs, and HFE components that were identified by

HRA and PRA analyses as being important to the plant's safety and reliability.

The HRA should be performed iteratively as the design progresses. The PRA and HRA should be

performed early in the'design process to provide insights and guidance both for systems design and for

HFE purposes. The robustness of the HRA depends, in large part, on the analyst's understanding of

personnel tasks, the information related to them, and the factors which influence human performance.

Accordingly, the HRA should be carried out interactively as the design progresses. At the very least, the

initial PRA/HRA should be finalized when the plant design and HFE are complete.

Although there are many different approaches to the conduct of HRA, there are several analysis

components that verify the quality of the HRA. These include

* meeting all applicable 10 CFR 50.34 (f)(l)(i) requirements

* use of a multidisciplinary team to analyze human actions within the context of the PRA

* availability of information related to those factors that affect human performance, such as

accident analyses (indicating time available for action), task analyses, procedures, and

HSI design details

* consideration of the effects of advanced technology on human performance and the

potential for different types of human error that may be associated with the technology

* detailed analyses of human actions with an emphasis on human error mechanisms

* availability of appropriate sources of human error data for the types of human actions that

are modeled

* sensitivity and uncertainty analyses to evaluate human error probability estimates

33
integration of PRA and HRA activities into plant design activities

thorough documentation of the HRA process

Figure 7.1 The role of human reliability analysis in the HFE program

34
7.2 Objective

The objectives of this review are to verify that (1) the applicant has addressed human-error mechanisms in

the design of the HFE aspects of the plant to minimize the likelihood of personnel error, and verify errors

are detected and recovered from; and (2) the HRA activity effectively integrates the HFE program and

PRA and risk analysis.

7.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for human reliability analysis. Upon completion of the applicant's efforts, a results

summary report should be submitted so that the staff can review the-applicant's evaluation of human-error

mechanisms in the design of the HFE aspects of the plant and their integration of the HFE program and

PRA and risk analysis using the criteria provided in Section 7.4 below.

7.4 Review Criteria

(1) Risk-important human actions should be identified from the PRAIHRA and used as input to the

HFE design effort.

* These actions should be developed from the Level 1 (core damage) PRA and Level 2

(release from containment) PRA including both internal and external events. They should

be developed using selected (more than one) importance measures and HRA sensitivity

analyses to provide reasonable assurance that an important action is not overlooked

because of the selection of the measure or the use of a particular assumption in the

analysis.

* When upgrading plant systems, HSIs, procedures, and training the scope of the analysis

should address personnel actions resulting from the modification and its interactions with

the rest of the plant. Consideration should be given to the following effects of these

modifications on the existing HRA:

- whether the original HRA assumptions are valid for the modified design

- whether the human errors analyzed in the existing HRA are still relevant

- whether the probability of errors by operators and maintenance personnel may

change

- whether errors may be introduced that are not modeled by the existing HRA and

PRA

- whether the consequences of errors, established in the existing HRA, may change

(2) Risk-important HAs and their associated tasks and scenarios should be specifically addressed

during function allocation analyses, task analyses, HSI design, procedure development, and

training. This will help verify that these tasks are well supported by the design and within

acceptable human performance-capabilities (e.g. within time and workload requirements).

(3) The use of PRA/HRA results by the HFE design team should be specifically addressed; that is,

how are risk-important HAs addressed (through HSI design, procedural development, and

training) under the HFE program to minimize the likelihood of operator error and provide for

error detection and recovery capability.

35
(4) HRA assumptions such as decisionmaking and diagnosis strategies for dominant sequences

should be validated by walkthrough analyses with personnel with operational experience using a

plant-specific control room mockup or simulator. Reviews should be conducted before the final

quantification stage of the PRA.

7.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

IEEE Std. 1082-1997: IEEE Guidefor Incorporating Human Action Reliability Analysisfor

Nuclear Power Generating Stations. (Institute of Electrical and Electronics Engineers, 1997).

NUREG-0800: Standard Review Plan: Chapter 19, Use of Probabilistic Risk Assessment in

Plant-Speciflc, Risk-Informed Decision Making: General Guidance (NRC (2002)

NUREG-1560: Individual Plant Examination Program: Perspectives on Reactor Safety and Plant

Performance (NRC, 1997).

NUREG- 1624, Rev. 1.: Technical Basis and Implementation Guidelinesfor A Techniquefor

Human EventAnalysis (ATHEANA) (NRC, 2000).

NUREG/CR-6689: Proposed Approach for Reviewing Changes to Risk-Important Human

Actions (Higgins and O'Hara, 2000).

Regulatory Guide 1.174: An approach for using probabilistic risk assessment in risk-informed

decisions on plant-specific changes to the licensing basis (NRC, 1998).

Regulatory Guide 1.177: An Approach for Plant-Specifc, Risk-Informed Decisionmaking:

Technical Specifications (NRC, 1998).

Risk-informed inspection notebooks for each plant.

36
8 HUMAN-SYSTEM INTERFACE DESIGN

8.1 Background

The HSI design process represents the translation of function and task requirements into HSI

characteristics and functions. The HSI should be designed using a structured methodology that should

guide designers in identifying and selecting candidate HSI approaches, defining the detailed design, and

performing HSI tests and evaluations. It should cover the development and use of HFE guidelines that are

tailored to the unique aspects of the applicants's design, e.g., a style guide to define the design-specific

conventions. The availability of an HSI design methodology will help verify standardization and

consistency in applying HFE principles. The process and the rationale for the HSI design should be

documented for review (including the results of trade-off studies, other analyses and evaluations, and the

rationale for choosing design and evaluation tools).

Issues related to the detailed design of specific aspects of the HSIs should be resolved during HSI design

activities rather than at verification and validation (V&V). For example, considerations as to acceptable

display formats or alarm system processing should be resolved during the HSI design activities rather

than deferred to V&V (as described in Section I 1), at which point making modifications to the design is

significantly more difficult.

8.2 Objective

The objective of this review element is to evaluate the process by which HSI design requirements are

developed and HSI designs are identified and refined. The review should verify that the applicant has

appropriately translated functional and task requirements to the detailed design of alarms, displays,

controls, and other aspects of the HSI through the systematic application of HFE principles and criteria.

8.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for human-system interface design process. Upon completion of the applicant's

efforts, a results summary report should be submitted so that the staff can review the applicant's

development of design requirements and the HSI design using the criteria provided in Section 8.4 below.

8.4 Review Criteria

8.4.1 HSI Design Inputs

The following sources of information should provide input to the HSI design process:

(1) Analysis of Personnel Task Requirements - The analyses performed in earlier stages of the design

process should be used to identify requirements for the HSIs. These analyses include:

* Operational experience review - Lessons learned from other complex human-machine

systems, especially predecessor designs and designs involving similar HSI technology

should be used as an input to HSI design.

* Functional requirement analysis and function allocation - The HSIs should support the

operator's role in the plant, e.g., appropriate levels of automation and manual control.

* Task analysis - The set of requirements to support the role of personnel is provided by

task analysis. The task analysis should identify:

37
- tasks that are necessary to control the plant in a range of operating conditions for

normal through accident conditions;

- detailed information and control requirements (e.g., requirements for display

range, precision, accuracy, and units of measurement);

- task support requirements (e.g., special lighting and ventilation requirements);

and

- risk-important HAs and their associated performance shaping factors, as

identified through HRA should be given special attention in the HSI design

process.

Staffing/qualifications and job analyses - The results of staffing/qualifications analyses

should provide input for the layout of the overall control room and the allocation of

controls and displays to individual consoles, panels, and workstations. They establish the

basis for the minimum and maximum number of personnel to be accommodated and

requirements for coordinating activities between personnel.

(2) System Requirements - Constraints imposed by the overall instrumentation and control (I&C)

system should be considered throughout the HSI design process.

(3) Regulatory Requirements - Applicable regulatory requirements should be identified as inputs to

the HSI design process.

(4) Other Requirements - The applicant should identify other requirements that are inputs to the HSI

design.

8.4.2 Concept of Operations

(1) A concept of operations should be developed indicating crew composition and the roles and

responsibilities of individual crew members based on anticipated staffing levels. The concept of

operations should:

* Identify the relationship between personnel and plant automation by specifying the

responsibilities of the crew for monitoring, interacting, and overriding automatic systems

and for interacting with computerized procedures systems and other computerized

operator support systems.

* Provide a high-level description of how personnel will work with HSI resources.

Examples of the types of information that should be identified is the allocation of task to

the main control room or local control stations, whether personnel will work at a single

large workstation or individual workstations, what types of information each crew

member will have access to, and what types of information should be displayed to the

entire crew.

* Address the coordination of crew member activities, such as the interaction with auxiliary

operators and coordination of maintenance and operations should be addressed.

38
8.4.3 Functional Requirement Specification

(1) Functional requirements for the HSIs should be developed to address:

* the concept of operations

* personnel functions and tasks that support their role in the plant as derived from function,

task, and staffing/qualifications analyses

* personnel requirements for a safe, comfortable working environment

(2) Requirements should be established for various types of HSIs, e.g., alarms, displays, and controls.

8.4.4 HSI Concept Design

(I) The functional requirement specification should serve as the initial source of input to the HSI

design effort. If the design is a direct evolution from a predecessor, rather than a new design

concept, the criteria in this section should be considered relative to operating experience of the

predecessor and the design features (e.g., aspects of the process, equipment, or operations) of the

new design that may be different from the predecessor. Human performance issues identified

from operating experience with the predecessor design should be resolved.

(2) Alternative approaches for addressing HSI functional requirements should be considered. A

survey of the state-of-the-art in HSI technologies should be conducted to:

* support the development of concept designs that incorporate advanced HSI technologies

* provide assurance that proposed designs are technically feasible

* support the identification of human performance concerns and tradeoffs associated with

various HSI technologies

(3) Alternative approaches for addressing HSI functional requirements should be considered.

Evaluation methods can include operating experience and literature analyses, tradeoff studies,

engineering evaluations and experiments.

(4) Alternative concept designs should be evaluated so that one can be selected for further

development. The evaluation should provide reasonable assurance that the selection process is

based on a thorough review of design characteristics and a systematic application of selection

criteria. Tradeoff analyses, based on the selection criteria, should provide a rational basis for the

selection of concept designs.

(5) HSI design performance requirements should be identified for components of the selected HSI

concept design. These requirements should be based on the functional requirement specifications

but should be refined to reflect HSI technology considerations identified in the survey of the state

of the art in HSI technologies and human performance considerations identified in the human

performance research.

39
8.4.5 HSI Detailed Design and Integration

(1) Design-specific HFE design guidance (style guide) should be developed. HFE Guidelines should

be utilized in the design of the HSI features, layout, and environment.

The content of the Style Guide should be derived from (1) the application of generic HFE

guidance to the specific application, and (2) the development of the applicant's own

guidelines based upon design-related analyses and experience. Guidelines that are not

derived from generic HFE guidelines may be justified by the applicant based on an

analysis of recent literature, analysis of current industry practices and operational

experience, tradeoff studies and analyses, and the results of design engineering

experiments and evaluations. The guidance should be tailored to reflect design decisions

by the applicant to address specific goals and needs of the HSI design.

* The topics in the Style Guide should address the scope of HSIs included in the design and

address the form, function, and operation of the HSIs as well as environmental

characteristics relevant to human performance.

* The individual guidelines should be expressed in concrete, easily observable terms. In

general, generic HFE guidelines should not be used in their abstract form. Such generic

guidance should be translated into more specific design guidelines that can, as much as

possible, provide unambiguous guidance to designers and evaluators. They should be

detailed enough to permit their use by design personnel to achieve a consistent and

verifiable design that meets the applicant's guideline.

* The Style Guide should provide procedures for determining where and how HFE

guidance is to be used in the overall design process. The Style Guide should be written so

it can be readily understood by designers. The Style Guide should support the

interpretation and comprehension of design guidance by supplementing text with

graphical examples, figures, and tables.

* The guidance should be maintained in a form that is readily accessible and usable by

designers and that facilitates modification when the contents require updating as the

design matures. Each guideline included in the guidance documentation should include a

reference to the source upon which it is based.

* The Style Guide should address HSI modifications. This guidance should specifically

address consistency in design across the HSIs.

(2) The HSI detailed design should support personnel in their primary role of monitoring and

controlling the plant while minimizing personnel demands associated with use of the HSIs (e.g.,

window manipulation, display selection, display system navigation). NUREG-0700 describes

high-level HSI design review principles that the detailed design should reflect.

(3) For risk-important HAs, the design should seek to minimize the probability that errors will occur

and maximize the probability that an error will be detected if one should be made.

(4) When developing functional requirements for monitoring and control capabilities that may be

provided either in the control room or locally in the plant, the following factors should be

considered:

* communication, coordination, and workload

feedback

40
* local environment

* inspection, test, and maintenance

* importance to safety

(5) The layout of HSIs within consoles, panels, and workstations should be based upon (1) analyses

of operator roles (ob analysis) and (2) systematic strategies for organization such as arrangement

by importance, frequency of use, and sequence of use.

(6) Personnel and task performance should be supported during minimal, nominal, and high-level

staffing.

(7) The design process should take into account the use of the HSIs over the duration of a shift where

decrements in performance due to fatigue may be a concern.

(8) HSI characteristics should support human performance under the full range of environmental

conditions, e.g., normal as well as credible extreme conditions. For the main control room

requirements should address conditions such as loss of lighting, loss of ventilation, and main

control room evacuation. For the remote shutdown facility and local control stations,

requirements should address constraints imposed by the ambient environment (e.g., noise,

temperature, contamination) and by protective clothing (if necessary).

(9) The HSIs should be designed to support inspection, maintenance, test, and repair of (1) plant

equipment and (2) the HSIs. The HSIs should be designed so that inspection, maintenance, test,

and repair of the HSIs do not interfere with other plant control activities (e.g., maintenance tags

should not block the operators' views of plant indications).

(10) The following considerations should be addressed in the review of design modifications:

* 1HSI modifications should be designed, to the extent possible, to be consistent with users'

existing strategies for gathering and processing information and executing actions,

identified in the task analysis. Consistency with existing strategies can reduce the

learning personnel need to become proficient in using the modification.

* Design requirements for computer-based HSI modifications should include requirements

for crew coordination and define design characteristics for supporting it. Design

characteristics that may limit crew coordination include features that limit the ability of

personnel to have a shared view of plant information (e.g., decision-aids and display

devices that can only be accessed by one individual), maintain an awareness of others'

actions, and communicate effectively with others from anticipated work locations.

* If the degree of integration between plant systems is changed, then design requirements

should be developed to verify that the HSIs support personnel in controlling these

systems. The design requirements of the HSIs should provide reasonable assurance that

the relationships between plant systems are clearly and accurately depicted.

41
8.4.6 HSI Tests and Evaluations

Testing and evaluation of HSI designs should be conducted throughout the HSI development process and

evaluations should be performed iteratively. The methodology used for testing should be reviewed using

the appropriate criteria provided below. Note the types of tests and evaluations performed will vary

depending on the specific applicant's design process.

8.4.6.1 Trade-Off Evaluations

(1) Aspects of human performance that are important to task performance should be carefully

selected and defined so that the differential effects of design options on human performance can

be adequately considered in the selection of design approaches. The following factors should be

considered when developing selection criteria:

* personnel task requirements

* human performance capabilities and limitations

* HSI system performance requirements

* inspection and testing requirements

* maintenance requirements

* use of proven technology and the operating experience of predecessor designs.

(2) The selection process should make explicit the relative benefits of design alternatives and the

basis for their selection.

8.4.6.2 Performance-Based Tests

(1) Performance-based tests can have many different purposes, therefore, the hypotheses should be

structured to address the specific questions being addressed.

(2) The general approach to testing should be based on the test objective. The design of

performance-based tests should be driven by the purpose of the evaluation and the maturity of the

design.

(3) The specific design features or characteristics of design features should be carefully defined.

If the characteristics are to be manipulated in the test, i.e., systematically varied, the differences

between test conditions should be specified in detail.

(4) The selection of testbeds for the conduct of performance-based tests should be based upon the

requirements imposed by the test hypotheses and the maturity of the design.

(5) The selection of performance measures should be based on a consideration of:

* measurement characteristics

* identification and selection of variables to represent measures of the aspects of

performance under investigation

* development of performance criteria.

(6) The selection of participants for HSI design tests should be based on the nature of the questions

being addressed in test objectives and the level of design maturity.

42
(7) The test design should permit the observation of performance in a manner that avoids or

minimizes bias, confounds, and error variance (noise).

(8) Test data should be analyzed using established analysis techniques.

(9) Design solutions, such as modifications of the HSIs or user training requirements, should be

developed to address problems that are identified during the testing and evaluation of the HSI

detailed design.

8.4.7 HSI Design Documentation

(1) The HSI design should be documented to include:

* the detailed HSI description including its form, function and performance characteristics

* the basis for the HSI requirements and design characteristics with respect to operating

experience and literature analyses, tradeoff studies, engineering evaluations and

experiments, and benchmark evaluations

* records of the basis of the design changes

(2) The outcomes of tests and evaluations performed in support of HSI design should be documented.

8.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

ANSI/AIAA G-035-1992: Guide to Human Performance Measurements (American National

Standards Institute, 1993).

ANSI HFS-I00: American National Standardfor Human Factors Engineering of Visual Display

Terminal Workstations (American National Standards Institute, 1988).

BNL TR E2090-T4-1-9/96: Human-System Interface Design Process and Review Criteria

(Stubler and O'Hara, 1996).

BNL TR E2090-T4-4-12/94, Rev. 1: Group-View Displays (Stubler and O'Hara, 1996).

EPRI NP-3659: Human Factors Guide for Nuclear Power Plant Control Room Development

(Kinkade and Anderson, 1984).

EPRI NP-4350: Human Engineering Design Guidelinesfor Maintainability (Pack et al., 1985).

EPRI-ALWR URD: Advanced Light Water Reactor Utility Requirements Document, Volume II,

Evolutionary Plant, Rev. 4 (Electric Power Research Institute, 1992).

IEC-964: Design for Control Rooms of Nuclear Power Plants (International Electrotechnical

Commission, 1989).

IEEE Std. 1023-1988: IEEE Guide to the Application ofHuman Factors Engineering to Systems,

Equipment, and Facilities of Nuclear Power Generating Stations (Institute of Electrical and

Electronics Engineers, 1988).

43
NUREG/CR-6684: Advance alarm systems: Guidance Development and Technical Basis

(Brown, et al., 2000)

NUREG/CR-6637: Human-System Interface and Plant Modernization Process: Technical Basis

and Human Factors Review Guidance (Stubler, O'Hara, Higgins, and Kramer, 2000).

NUREG/CR-63 93: Integrated System Validation: Methodology and Review Criteria (O'Hara, et

al., 1997).

NUREG/CR-6635: Soft Controls: Technical Basis and Human Factors Review Guidance (W.

Stubler, O'Hara, and Kramer, 2000).

NUREG/CR-6634: Computer-Based Procedure Systems: Technical Basis and Human Factors

Review Guidance (O'Hara, Higgins, Stubler, and Kramer, 2000).

NUREG/CR-6633: Advanced Information Systems: Technical Basis and Human Factors Review

Guidance (O'Hara, Higgins, and Kramer, 2000).

NUREG/CR-6636: Maintenance of Digital Systems: Technical Basis and Human Factors Review

Guidance (Stubler, Higgins, and Kramer, 2000).

NUREG-0696: Functional Criteria for Emergency Response Facilities (NRC, 1980).

NUREG-0700: Human-System Interface Design Review Guidelines (NRC, 2002).

NUREG-0800: Standard Review Plan, Chapter 18 Human Factors Engineering (NRC, 2004).

Regulatory Guide 1.22: Periodic Testing of Protection System Actuation Functions (NRC, 1972).

Regulatory Guide 1.105: Instrumentation Setpoints (NRC, 1999).

Regulatory Guide 1.81: Shared Emergency and Shutdown Electrical Systems for Multi-Unit

NPPs (NRC, 1975).

Regulatory Guide 1.97: Instrumentation for Light-Water-Cooled Nuclear Power Plants to Assess

Plant and Environmental Conditions During and Following an Accident (NRC, 1983).

Regulatory Guide 1.62: Manual Initiation of Protective Actions (NRC, 1973).

Regulatory Guide 1.47: Bypassed and Inoperable Status Indication for NPP Safety Systems

(NRC, 1973).

UCRL- 15673: Human Factors Design Guidelines for Maintainability of Department of Energy

Nuclear Facilities (Bongarra, et al., 1985).

44
9 PROCEDURE DEVELOPMENT

9.1 Background

Procedures are essential to plant safety because they support and guide personnel interactions with plant

systems and their response to plant-related events. In the nuclear industry, procedure development has

historically been considered the responsibility of individual utilities. Procedures should be derived from

the same design process and analyses as the HSIs and training and subject to the same evaluation

processes. The same human factors principles should be applied to both aspects of the interface to verify

complete integration and consistency.

For new plant designs and advanced reactors, the generic technical guidance (GTG), if available, and

procedures should be developed as part of the same design process as the HSIs and training to verify a

high degree of integration and consistency. For plants that modernize, the procedure modifications

should address all personnel tasks that are affected by the changes in plant systems and HSIs. Procedures

should be developed or modified to reflect the characteristics and functions of the modification.

9.2 Objective

The objective of the review is to verify that the applicant has applied HFE principles and guidance, along

with all other design requirements, to develop procedures that are technically accurate, comprehensive,

explicit, easy to use, and validated.

9.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for procedure development. Upon completion of the applicant's efforts, a results

summary report should be submitted so that the staff can review the applicant's efforts to develop

procedures that are technically accurate, comprehensive, explicit, easy to use, and validated using the

criteria provided in Section 9.4 below.

In addition, GTG and sample procedures should be available for review. The scope of the procedures

covered in the element are:

* GTG for emergency operating procedures (EOPs)

* plant and system operations (including startup, power, and shutdown operations)

* maintenance

* abnormal and emergency operations

* alarm response

9.4 Review Criteria

(1) Procedures should address applicable requirements of NUREG-0800, Section 13.5.

(2) The basis for procedure development should include:

* plant design bases

* system-based technical requirements and specifications

* task analyses results

* risk-important human actions identified in the HRA/PRA

45
* initiating events to be considered in the EOPs, including those events in the design bases

* GTG for EOPs

(3) A writers guide should be developed to establish the process for developing technical procedures

that are complete, accurate, consistent, and easy to understand and follow. The guide should

contain objective criteria so that procedures developed in accordance with it are consistent in

organization, style, and content. The guide should be used for all procedures within the scope of

this element. It should provide instructions for procedure content and format including the

writing of action steps and the specification of acceptable acronym lists and acceptable terms to

be used.

(4) The content of the procedures should incorporate the following elements:

* title and identifying information, such as number, revision, and date

* statement of applicability and purpose

* prerequisites

* precautions (including warnings, cautions, and notes)

* important human actions

* limitations and actions

* acceptance criteria

* checkoff lists

* reference material

(5) GTGs and EOPs should be symptom-based with clearly specified entry conditions.

(6) All procedures should be verified and validated, including:

* A review should be conducted to verify they are correct and can be carried out.

* Their final validation should be performed in a simulation of the integrated system as part

of the verification and validation activities described in the Human Factors Verification

and Validation element, see Section 11.

* When procedures are modified, they should be verified to verify their adequate content,

format, and integration. The procedures also should be assessed through validation if a

modification substantially changes personnel tasks that are significant to plant safety.

The validation should verify that the procedures correctly reflect the characteristics of the

modified plant and can be carried out effectively to restore the plant.

(7) An analysis should be conducted to determine the impact of providing computer-based

procedures (CBPs) and to specify where such an approach would improve procedure utilization

and reduce operating crew errors related to procedure use. The justifiable use of CBPs over paper

procedures should be documented. An analysis of alternatives in the event of loss of CBPs

should be performed and documented.

(8) A plan for procedure maintenance and control of updates should be developed. Procedure

modifications should be integrated across the full set of procedures; alterations in particular parts

of the procedures should not conflict nor be inconsistent with other parts.

(9) The physical means by which operators access and use procedures, especially during operational

events, should be evaluated as part of the HFE design process. This criterion generally applies to

46
both hard-copy and computer-based procedures, although the nature of the issues differs

somewhat depending on the implementation. For example, the process should address the storage

of procedures, ease of operator access to the correct procedures, and laydown of hard-copy

procedures for use in the control room, remote shutdown facility, and local control stations.

9.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

ANS 3.2-1994: Administrative Controls and QualityAssurancefor the Operational Phase of

NPPs (American Nuclear Society, 1994).

IP 42700: Plant Procedures. (NRC, periodically updated).

IP 42001: Emergency Operating Procedures. (NRC, periodically updated).

NRC Regulatory Guide 1.33 (Rev. 2): Quality Assurance Program Requirements (NRC, 1978).

NUREG/CR-6634: Computer-Based Procedure Systems: Technical Basis and Human Factors

Review Guidance (O'Hara, Higgins, Stubler, and Kramer, 2000).

NUREG/CR-5228: Techniques for Preparing Flowchart Format Emergency Operating

Procedures, Volumes I and 2 (Barnes et al., 1989).

NUREG-0800: Standard Review Plan (NRC, 2004).

NUREG-0899: Guidelines for the Preparation of Emergency Operating Procedures (NRC,

1982).

NUREG-1358: Lessons Learned From the Special Inspection Program for Emergency Operating

Procedures, Supplement I (NRC, 1992).

NUREG-1358: Lessons Learned From the Special Inspection Program for Emergency Operating

Procedures (NRC, 1989).

NUREG/CR-5228: Techniques for Preparing Flowchart-Format Emergency Operating

Procedures (Barnes et al., 1989).

47
10 TRAINING PROGRAM DEVELOPMENT

10.1 Background

Training of plant personnel is an important factor in ensuring safe and reliable operation of nuclear power

plants. Training programs help to provide reasonable assurance plant personnel have the knowledge,

skills, and abilities to properly perform their roles and responsibilities. Training design should be based

on the systematic analysis ofjob and task requirements. Therefore, training program development should

be coordinated with the other elements of the HFE design process.

10.2 Objective

The objective of this review is to verify that the applicant has a systematic approach for the development

of personnel training. The training development should include the following five activities:

* a systematic analysis of tasks and jobs to be performed

* development of learning objectives derived from an analysis of desired performance

following training

* design and implementation of training based on the learning objectives

* evaluation of trainee mastery of the objectives during training

* evaluation and revision of the training based on the performance of trained personnel in

the job setting

10.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for training program development. Upon completion of the applicant's efforts, a

results summary report should be submitted so that the staff can review the applicant's training program

using the criteria provided in Section 10.4 below.

10.4 Review Criteria

The review criteria are organized into the following sections: General Approach, Organization of

Training, Learning Objectives, Content of Training Program, Evaluation of Training, and Periodic

Re-training.

10.4.1 General Approach

(1) A systems approach to the training of plant personnel should be developed that address applicable

guidance in NUREG-0800 Section 13.2 ("Training"), as defined in 10 CFR 55.4, and as required

by 10 CFR 52.78 and 50.120.

(2) The overall scope of training should be defined including the following:

* categories of personnel (e.g., senior reactor operator) to be trained

* specific plant conditions (normal,'upset, and emergency)

* specific operational activities (e.g., operations, maintenance, testing and surveillance)

* lHSIs (e.g., in the main control room, emergency operations facility, remote shutdown

panel, local control stations)

49
(3) The training program should provide reasonable assurance that personnel have the qualifications

commensurate with the performance requirements of theirjobs. Training should address:

* the full range of positions of operational personnel including licensed and nonlicensed

personnel whose actions may affect plant safety

* the full range of plant functions and systems including those that may be different from

those in predecessor plants (e.g., passive systems and functions)

* the full range of relevant HSIs (e.g., main control room, remote shutdown panel, local

control stations) including characteristics that may be different from those in predecessor

plants (e.g., display space navigation, operation of "soft" controls)

* the full range of plant conditions

10.4.2 Organization of Training

(1) The roles of all organizations, especially the applicant and vendors, should be specifically defined

for the development of training requirements, development of training information sources,

development of training materials, and implementation of the training program. For example, the

role of the vendor may range from merely providing input materials (e.g., EPG) to conducting

portions of specific training programs.

(2) The qualifications of organizations and personnel involved in the development and conduct of

training should be defined.

(3) Facilities and resources such as plant-referenced simulator and part-task training simulators

needed to satisfy training design requirements and the guidance contained in ANSI 3.5 and

Regulatory Guide 1.149 should be defined.

10.4.3 Learning Objectives

(1) Learning objectives should be derived from the analysis that describes desired performance after

training. This analysis should include but not be limited to training needs identified in the

following:

* Licensing Basis - Final Safety Analysis Report, system description manuals and operating

procedures, facility license and license amendments, licensee event reports, and other

documents identified by the staff as being important to training

* Operating Experience Review - previous training deficiencies and operational problems

that may be corrected through additional and enhanced training, and positive characteristics of previous training programs

* Function Analysis and Allocation - functions identified as new or modified

* TaskAnalysis - tasks identified during task analysis as posing unusual demands including

new or different tasks, and tasks requiring a high degree if coordination, high workload,

or special skills

* Human Reliability Analysis - coordinating individual roles to reduce the likelihood and/or

consequences of human error associated with risk-important HAs and the use of

advanced technology

* HSIDesign - design features whose purpose or operation may be different from the past

experience or expectations of personnel

50
* Plant Procedures - tasks that have been identified during procedure development as

being problematic (e.g., procedure steps that have undergone extensive revision as a

result of plant safety concerns)

* . Verification and Validation (V&V) - training concerns identified during V&V, including

HSI usability concerns identified during validation or suitability verification and operator

performance concerns (e.g., misdiagnoses of plant event) identified during validation

trials

(2) Learning objectives for personnel training should address the knowledge and skill attributes

associated with all relevant dimensions of the trainee's job, such as interactions with the plant, the

HSIs, and other personnel. Table 10.1, below, shows these dimensions.

Table 10.1 Some knowledge and skill dimensions for learning objectives identification

Topic Knowledge | Skill

Plant Understanding of plant processes, Skills associated with monitoring

systems, operational constraints, and detection, situation awareness, Interactionsand failure modes., response planning and

implementation.

HSI and Procedure Understanding of procedures and Skills associated with interfaceHSI structure, functions, failure management tasks. Interactions modes, and interface management

tasks (actions, errors, and recovery

strategies).

Personnel Understanding information Skills associated with crew's

.e orequirements of others, how actions interactions (i.e., teamwork) Interactionsshould be coordinated with others,

(In the CR and in the plant) policies and constraints on crews'

interaction.

10.4.4 Content of Training Program

(1) 'The design of the training program should be defined to specify how learning objectives will be

conveyed to the trainee. The definition should include:

* -The use of lecture, simulator, and on-the-job training to convey particular categories of

learning objectives should be defined.

* Specific plant conditions and scenarios to be used in training programs should be defined.

* Training implementation considerations such as the temporal order and schedule of

training segments should be defined.

(2) Factual knowledge should be taught within the context of actual tasks so that personnel learn to

apply it in the work environment. The context of the job should be defined, and it should be

51
represented meaningfully to help trainees to link the knowledge to the job's requirements.

Training that addresses theory should be integrated with training in using procedures.

(3) Training programs for developing skills should be structured so that the training environment is

consistent with the level of skill being taught. It should support skill acquisition by allowing

trainees to manage cognitive demands. For example, trainees should not be placed in

environments teaching high-level skills, such as coordinating control actions among crew

members, before they have mastered requisite, low-level skills, such as how to manipulate control

devices.

(4) Training should address rules for decision-making related to plant systems, HSIs, and procedures.

It should include rules for accessing and interpreting information and rules for interpreting

symptoms of failures of systems, HSIs, and procedures. This training should cover acquiring new

decision-making rules and eliminating existing ones that are not appropriate to the design.

10.4.5 Evaluation and Modification of Training

(1) Methods for evaluating the overall effectiveness of the training programs and trainee mastery of

training objectives should be defined, including written and oral tests and review of personnel

performance during walkthrough, simulator exercises, and on-the-job. Evaluation criteria for

training objectives should be defined for individual training modules. Methods for assessing

overall proficiency should be defined and coordinated with regulations, where applicable.

(2) Methods for verifying the accuracy and completeness of training course materials should be

defined.

(3) Procedures for refining and updating the content and conduct of training should be established,

including procedures for tracking training course modifications.

10.4.6 Periodic Retraining

(I) Personnel should undergo periodic retraining.

(2) The applicant should evaluate whether any changes or increases in retraining are warranted

following plant modernization programs.

10.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

10 CFR Part 55: U.S. Code of Federal Regulations, Part 55, "Operators' Licenses," Title 10,

"Energy."

10 CFR 50.120: US. Code of Federal Regulations, Part 50, "Training and Qualification of

Nuclear Power Plant Personnel," Title 10, "Energy."

ANSI/ANS 3.1-1993; RI999: Selection, Qualiflcation, and Training ofPersonnelfor Nuclear

Power Plants (American Nuclear Society, 1999).

ANSI/ANS 3.5-1998: Nuclear Power Plant Simulatorsfor Use in Operator Training (American

Nuclear Society, 1998).

52
IP 41500: Training and Qualification Effectiveness. (NRC, periodically updated).

NUREG-0800: Standard Review Plan, Section 13.2, Training, (NRC, 1999)

NUREG-1220: Training Review Criteria and Procedures (NRC, Revised periodically).

Regulatory Guide 1.149: Nuclear Power Plant Simulators for Use in Operator Training (NRC,

2001).

Regulatory Guide 1.8: Personnel Selection and Training (NRC, 2000).

53
11 HUMAN FACTORS VERIFICATION AND VALIDATION

11.1 Background

Verification and validation (V&V) evaluations comprehensively determine that the design conforms to

HFE design principles and that it enables plant personnel to successfully perform their tasks to achieve

plant safety and other operational goals. This section describes four major activities: Operational

Condition Sampling, Design Verification, Integrated System Validation, and Human Engineering

Discrepancies (HEDs) Resolution (see Figure 11.1). The sampling of operational conditions to support

V&V tests is important because reviews of new plants and significant HSI modifications can involve

hundreds or thousands of individual HSI components. It would be impractical and unnecessary to review

all of them. Therefore, the applicant can employ a sampling strategy to guide the selection of HSIs to

review.

It should be noted that with the exception of Integrated System Validation, the majority of this section

mainly addresses verification of HSIs. There are separate NRC reviews to validate procedures and

training programs.

The review involves two types of Design Verification: HSI Task Support Verification and HFE Design

Verification. HSI Task Support Verification is an evaluation to verify that the HSI supports personnel

task requirements as defined by task analyses. HEDs are identified for: (1) personnel task requirements

that are not fully supported by the HSI, and (2) the presence of HSI components which may not be needed

to support personnel tasks. HFE Design Verification is an evaluation to verify that the HSI is designed to

accommodate human capabilities and limitations as reflected in HFE guidelines such as those provided in

NUREG-0700. HEDs are identified if the design is inconsistent with HFE guidelines.

Integrated System Validation is an evaluation using performance-based tests to determine whether an

integrated system design (i.e., hardware, software, and personnel elements) meets performance

requirements and acceptably supports safe operation of the plant. HEDs are identified if performance

criteria are not met.

HED Resolution is an evaluation to provide reasonable assurance that the HEDs identified during the

V&V activities have been acceptably assessed and resolved. HED Resolution is an activity that should be

performed iteratively with V&V. That is, the applicant may address and resolve issues identified during a

V&V activity prior to conducting other V&V activities. The preferred order is HSI Task Support

Verification, HFE Design Verification, and Integrated System Validation, although iteration may be

necessary.

Many design documents (e.g., ISO 11064) recommend conducting V&V throughout the design process.

This document agrees with that recommendation, with these activities called "HSI Tests and Evaluations"

(see the HSI Design element, Section 8.4.6). Such tests are distinguished from V&V since they are

activities whereby HSI subsystem design issues (such as the coding techniques used in the alarm system)

are explored and evaluated. V&V is considered a test that final design requirements are met.

55
Design Verification

.| HSI Task

I Support

verification

Inventory and

Characteftation

II HIFE Design|

Verification I

Operational

Conditions - Sampling l

Integrated

System

Validation

HED

* Resolution

Figure 11.1 Overview of verification and validation activities

11.2 Objective

Detailed review objectives for the various aspects of V&V are provided for each subsection of Section

11.4 below.

11.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for HFE V&V. Upon completion of the applicant's efforts, a results summary report

should be submitted so that the staff can review the applicant's V&V evaluations using the criteria

provided in Section 11.4 below.

In addition to the review of the applicant's documentation, the NRC staff may also verify a sample of

V&V activities to confirm the results and observe the integrated system validation trials as part of the

review.

11.4 Review Criteria

11.4.1 Operational Conditions Sampling

The sampling methodology will identify a range of operational conditions to guide V&V activities. The

review of operational conditions sampling considers the dimensions to be used to identify and select

conditions and their integration into scenarios.

11.4.1.1 Operational Conditions Sampling Review Objectives

The review should verify that the applicant has identified a sample of operational conditions that (1)

includes conditions that are representative of the range of events that could be encountered during

56
operation of the plant, (2) reflects the characteristics that are expected to contribute to system

performance variation, and (3) considers the safety significance of HSI components. These sample

characteristics are best identified through the use of a multidimensional sampling strategy to provide

reasonable assurance that variation along important dimensions is included in the V&V evaluations. The

review criteria, therefore, address the sampling dimensions used and the identification of scenarios based

on those dimensions. In addition, special considerations for plant modernization and modification

programs are identified.

11.4.1.2 Operational Conditions Sampling Review Criteria

11.4.1.2.1 Sampling Dimensions

The following sampling dimensions are addressed below: plant conditions, personnel tasks, and

situational factors known to challenge personnel performance.

(1) The following plant conditions should be included:

normal operational events including plant startup, plant shutdown or refueling, and

significant changes in operating power

* failure events, e.g.,

- instrument failures [e.g., safety-related system logic and control unit, fault

tolerant controller, local "field unit" for multiplexer (MUX) system, MUX

controller, and break in MUX line] including I&C failures that exceed the design

basis, such as a common mode I&C failure during an accident

- HSI failures (e.g., loss of processing and/or display capabilities for alarms,

displays, controls, and computer-based procedures)

* transients and accidents, e.g.,

- transients (e.g., turbine trip, loss of off-site power, station blackout, loss of all

feedwater, loss of service water, loss of power to selected buses or main control

room (MCR) power supplies, and safety and relief valve transients)

- accidents (e.g., main steam line break, positive reactivity addition, control rod

insertion at power, anticipated transient without scram, and various-sized loss-ofcoolant accidents)

- -reactor shutdown and cooldown using the remote shutdown system

* reasonable, risk-significant, beyond-design-basis events, which should be determined

from the plant specific PRA

* consideration of the role of the equipment in achieving plant safety functions [as

described in the plant safety analysis report (SAR)] and the degree of interconnection

with other plant systems. A system that is interconnected with other systems could cause

the failure of other systems because the initial failure could propagate over the

connections. This consideration is especially important when assessing non-class IE

electrical systems.

(2) The following types of personnel tasks should be included:

* Risk-signijlcant HAs, systems, and accident sequences - All risk-important HAs should

be included in the sample. These include identified in the PRA and those identified as

57
risk-important in the SAR and NRC's safety evaluation report (SER) should be included.

Situations where human monitoring of an automatic system is risk-important should be

considered. Additional factors should be sampled that contribute highly to risk, as

defined by the PRA, including:

- dominant human actions (selected via sensitivity analyses)

- dominant accident sequences

- dominant systems (selected via PRA importance measures such as Risk

Achievement Worth or Risk Reduction Worth)

* OER-identifled dijlcult tasks - The sample should include all personnel tasks identified

as problematic during the applicant's review of operating experience.

* Range ofprocedure guided tasks - These are tasks that are well defined by normal,

abnormal, emergency, alarm response, and test procedures. The operator should be able

to, as part of rule-based decision-making, understand and execute the specified steps.

Regulatory Guide 1.33, Appendix A, contains several categories of "typical safety-related

activities that should be covered by written procedures." The sample should include

appropriate procedures in each relevant category:

- administrative procedures

- general plant operating procedures

- procedures for startup, operation, and shutdown of safety-related systems

- procedures for abnormal, off normal, and alarm conditions

- procedures for combating emergencies and other significant events

- procedures for control of radioactivity

- procedures for control of measuring and test equipment and for surveillance tests,

procedures, and calibration

- procedures for performing maintenance

- chemistry and radiochemical control procedures

* Range of knowledge-based tasks - these are tasks that are not as well defined by detailed

procedures. Knowledge-based decision-making involves greater reasoning about safety

and operating goals and the various means of achieving them. A situation may require

knowledge-based decision-making if the rules do not fully address the problem, or the

selection of appropriate rule is not clear. An example in a pressurized water reactor plant

may be the difficulty in diagnosing a steam generator tube rupture (SGTR) with a failure

of radiation monitors on the secondary side of the plant because (1) there is no main

indication of the rupture (the presence of radiation in secondary side), and (2) the other

effects of the rupture (i.e., slight changes in pressures and levels on the primary and

secondary sides) may be attributed to other causes. While the operators may use

procedures to treat the symptoms of the event, the determination that the cause is a SGTR

may require situation assessment based on an understanding of the plant's design and the

possible combinations of failures that could result in the observed symptoms. Errors in

rule-based decision-making result from selecting the wrong rule or incorrectly applying a

rule. Errors in knowledge-based decision-making result from mistakes in higher-level

cognitive functions such as judgment, planning, and analysis. The latter are more likely

to occur in complex failure events where the symptoms do not resemble the typical case,

and thus, are not amenable to pre-established rules.

58
* Range ofhuman cognitive activities -The sample should include the range of cognitive

activities performed by personnel, including:

- detection and monitoring (e.g.,'of critical safety-function threats)

- situation assessment (e.g., interpretation of alarms and displays for diagnosis of

faults in plant processes and automated control and safety systems)

- response planning (e.g., evaluating alternatives for recovery from plant failures)

- response implementation (e.g., in-the-loop control of plant systems, assuming

manual control from automatic control systems, and carrying out complicated

control actions)

- obtaining feedback (e.g., of the success of actions taken)

* Range ofhuman interactions - The sample should reflect the range of interactions among

plant personnel, including tasks that are performed independently by individual crew

members and tasks that are performed by crew members acting as a team. These

interactions among plant personnel should include interactions between:

- main control room operators (e.g., operations, shift turnover walkdowns)

- main control room operators and auxiliary operators

- main control room operators and support centers (e.g., the technical support

center and the emergency offsite facility)

- main control room operators with plant management, NRC, and other outside

organizations

* Tasks that are performed with high frequency.

(3) The sample should reflect a range of situational factors that are known to challenge human

performance, such as:

* Operationally difficult tasks - The sample should address tasks that have been found to be

problematic in the operation of NPPs, e.g., procedure versus situation assessment

conflicts. The specific tasks selected should reflect the operating history of the type of

plant being validated (or the plant's predecessor).

Error-forcing contexts - Situations specifically designed to create human errors should be

included to assess the error tolerance of th'e system and the capability of operators to

recover from errors should they occur.

* High-workload conditions - The sample should include situations where human'

performance variation due to high workload and multitasking situations can be assessed.

* Varying-workload situations - The sample should include situations where human

performance variation due to workload transitions can be assessed. These include

conditions that exhibit (1) a sudden increase in the number of signals that must be

detected and processed following a period in which signals were infrequent and

(2) a rapid reduction in signal detection and processing demands following a period of

sustained high task demand.

* Fatigue and circadian factors - The sample should include situations where human

performance variation due to personnel fatigue and circadian factors can be assessed.

59
Environmentalfactors - The sample should include situations where human performance

variation due to environmental conditions such as poor lighting, extreme temperatures,

high noise, and simulated radiological contamination can be assessed.

11.4.1.2.2 Identification of Scenarios

(1) The results of the sampling should be combined to identify a set of scenarios to guide subsequent

analyses. A given scenario may combine many of the characteristics identified by the operational

event sampling.

(2) The scenarios should not be biased in the direction of over representation of the following:

* scenarios for which only positive outcomes can be expected

* scenarios that for integrated system validation are relatively easy to conduct

administratively (scenarios that place high demands, data collection or analysis are

avoided)

* scenarios that for integrated system validation are familiar and well structured

(e.g., which address familiar systems and failure modes that are highly compatible with

plant procedures such as "textbook" design-basis accidents)

11.4.1.2.3 Special Considerations for Plant Modernization Programs

When evaluating plant modifications, the following factors should be addressed when identifying

operational conditions:

(1) The operational conditions should reflect tasks that involve the modification, rather than the

entire range of topics discussed above for Personnel Tasks.

(2) For integrated system validation, the operational conditions should address the transfer of

learning effects on personnel performance when a modification replaces an old HSI or procedure.

(Negative transfer of learning effects may occur when the new and old components are different

and impose different demands on personnel.)

(3) For integrated system validation, when both old and new versions of the same HSI components

with different means of presentation and methods of operation are permanently present in the

HSI, evaluations should provide reasonable assurance that personnel can alternate their use of

these HSI components without degrading their performance.

(4) Where old HSI components that are to be deactivated and left in place in the HSI, conditions

should be identified for integrated system validation that would test the potential for task

interference. For example, the presence of deactivated HSI components may cause visual clutter

that interferes with the ability of operators to locate and use other HSI components.

60
11.4.2 Design Verificition

11.4.2.1 Inventory and Characterization

11.4.2.1.1 Inventory and Characterization Review Objectives

The objective of this review is to verify that the applicant's HSI inventory and characterization accurately

describes all HSI displays, controls, and related equipment that are within the defined scope of the HSI

design review.

11.4.2.1.2 Inventory and Characterization Review Criteria

(1) Scope - The applicant should develop an inventory of all HSI components associated with the

personnel tasks based on the identified operational conditions. The inventory should include

aspects of the HSI that are used for interface management such as navigation and display retrieval

in addition to those that control the plant.

(2) HSI Characterization -.The inventory should describe the characteristics of each HSI component

within the scope of the review. The following is a minimal set of information for the

characterization:

* a unique identification code number or name

* associated plant system and subsystem

* associated personnel functions/subfunction

* type of HSI component

- computer-based control (e.g., touch screen or cursor-operated button and

keyboard input)

- hardwired control (e.g., J-handle controller, button, and automatic controller)

- computer-based display (e.g., digital value and analog representation)

- hardwired display (e.g., dial, gauge, and strip chart recorder)

* display characteristics and functionality [e.g., plant variables/parameters, units of

measure, accuracy of variable/parameter, precision of display, dynamic response, and

display format (bar chart, and trend plot)]

* control characteristics and functionality [e.g., continuous versus discrete settings, number

and type of control modes, accuracy, precision, dynamic response, and control format

(method of input)] --

*i user-system interaction and dialog types (e.g., navigation aids and menus)

* location in data management system (e.g., identification code for information display

screen)

* physical location in the HSI (e.g., control panel section), if applicable

Photographs, copies of VDU screens, and similar samples of HSI components should be included

in the HSI inventory and characterization.

61
(3) Information Sources - The inventory should be based on the best available information sources.

Equipment lists, design specifications, and drawings describe HSI components. These

descriptions should be compared by directly observing the components, both hardwired and

computer-generated, to verify that the inventory accurately reflects their current state.

11.4.2.2 HSI Task Support Verification

11.4.2.2.1 HSI Task Support Verification Review Objectives

The objective of this review is to verify that the applicant has verified that the HSI provides all alarms,

information, and control capabilities required for personnel tasks.

11.4.2.2.2 HSI Task Support Verification Review Criteria

(1) Criteria Identification - The criteria for Task Support Verification come from task analyses of

HSI requirements for performance of personnel tasks that are selected operational conditions

should be defined.

(2) General Methodology - The HSIs and their characteristics (as defined in the HSI inventory and

characterization) should be compared to the personnel task requirements identified in the task

analysis.

(3) Task Requirements Deficiencies - HEDs should be identified when:

* an HSI needed for task performance (e.g., a required control or display) is not available

* HSI characteristics do not match the personnel task requirements, e.g., a display shows

the necessary plant parameter but not the range or precision needed for the task

(4) Unnecessary HSI Components - An HED should be identified for HSIs that are available in the

HSI but are not needed for any task. Unnecessary HSIs introduce clutter and can distract

personnel for the selection of appropriate HSIs. It is important to verify that the HSI is actually

unnecessary. Appropriate HSI components may not appear to be associated with personnel tasks

for the following reasons:

* The HSI component is needed for a task that was not addressed by the task analysis (e.g.,

it was not within the scope of the design review).

* The task analysis was incomplete, and thus overlooked the need for the HSI component.

* The HSI component only partially meets the personnel task requirements that were

established.

If an HSI component has no associated personnel tasks because the function and task analysis

was incomplete, then the applicant should identify and resolve any shortcomings in that analysis.

(5) Additional Methodology Considerations for Plant Modifications - the following considerations

should be addressed:

* HSI Task Support Verification should address all aspects of HSIs described above that

are relevant to the modification. For modifications to plant systems that do not include

62
modifications of the HSIs, task-support verification should identify any new demands for

monitoring and control, and determine whether they ire adequately addressed by the

existing HSI design.

HSI Task Support Verification should address modification configurations in which old

HSIs are permanently deactivated, but not removed (e.g., abandoned in place). Criterion

4, above, states that the HSIs should not contain any information, displays, or controls

that do not support personnel tasks. This verification should identify deactivated HSIs

that may have potentially negative effects on personnel performance, such as obstructing

the view of important information or adding visual clutter which may interfere with

monitoring. Deactivated HSIs requiring further evaluation through HFE design

verification or integrated system validation should be identified.

HSI Task Support Verification should address temporary configurations of the HSIs and

plant systems that may be created during implementation of the modification, and used

by operations and maintenance personnel when the plant is not shutdown. These

configurations may include:

- the use of HSIs that differ from the intended final design

- combinations of HSIs and system configurations that differ from both the

original and the intended final designs.

For each temporary HSI configuration, the task requirements of personnel should be

identified and compared to the information and control capabilities provided. For

example, if a temporary configuration of plant systems introduces special monitoring

requirements, then the HSIs should provide the necessary information.

(6) HED Documentation - HEDs should be documented to identify the HSI, the relevant task

criterion, and basis for the deficiency (what aspect of the HSI has been identified as not meeting

task requirements).

11.4.2.3 HFE Design Verification

11.4.2.3.1 HFE Design Verification Review Objective

The objective of this review is to verify that the applicant has verified that the characteristics of the HSI

and the environment in which it is used conform to HFE guidelines. The aspects of the applicant's HFE

Design Verification that are addressed in the staffs evaluation are discussed below.

11.4.2.3.2 HFE Design Verification Review Criteria

(1) Criteria IdentifIcation - The criteria for this verification are the HFE guidelines. The selection

of guidelines used in the review depends upon the characteristics of the HSI components included

in the scope of the review, as defined in the HSI characterization. It also depends upon whether

the applicant has developed a style guide (design-specific HFE guideline document). When a

style guide is used by the applicant, its acceptability should be reviewed by the staff. The

procedures involved are described in Section 8.4.5. The HFE guidelines contained in NUREG0700 may be used to support the staffs review of the guidance contained in an applicant's style

guide. When an NRC reviewed style guide has been used, it can provide the criteria for HFE

design verification.,

63
When no style guide is available, the guidelines in NUREG-0700 can be used for the HFE design

verification. However, since not all of these guidelines will be applicable to each review, the

selection of guidelines should be based on the characteristics of the HSI components being

evaluated. A subset of guidelines appropriate to the specific design implementation should be

identified based on the HSI characterization.

(2) General Methodology - The characteristics of the HSI components should be compared with HFE

guidelines. These guidelines are applicable to different aspects of the design: task-independent

features (e.g., font size), task-specific features (e.g., scale units), and task-integration features

(e.g., proximity of control-display).

A single guideline may apply to many identical HSI components, especially in the case of

significant HSI modifications and HSIs for new plants. In addition, some environmental

considerations (e.g., lighting) may be applicable. To simplify the application of guidelines and

reduce redundancy when reporting findings, the guidelines may be applied to features of the HSI

as follows:

* Globalfeatures - global HSI features are those relating to the configurational and

environmental aspects of the HSI, such as MCR layout, general workstation

configuration, lighting, noise, heating, and ventilation. These aspects of the review,

e.g., MCR lighting, tend to be evaluated only once.

* Standardizedfeatures - standardized features are those that were designed using HFE

guidelines applied across individual controls and displays (e.g., display screen

organization, display format conventions, and coding conventions). Therefore, their

implementation should be more consistent across the interface than features that were not

designed with guidelines. Thus, for example, if display labeling is standardized by the

applicant's HFE guidelines (style guide), which have been accepted by the NRC, then

display labels can be spot-checked rather than being verified individually.

* Detailedfeatures - detailed features are the aspects of individual HSIs that are not

addressed by general HFE guidelines. The latter can be expected to be more variable

than the standardized design features.

For each guideline, it should be determined whether the HSI is "acceptable" or "discrepant" from

the guideline (therefore, potentially unacceptable), an HED. "Acceptable" should be indicated

only if there is total compliance, i.e., only if every instance of the item is fully consistent with the

criteria established by the HFE guidelines. If there is any instance of noncompliance, full or

partial, then an evaluation of discrepant should be given, and a notation made as to where

noncompliance occurs.

Discrepancies should be evaluated as potential indicators of additional issues. For example,

identifying an inappropriate format for presenting data on an individual display should be

considered a potential sign that other display formats could be incorrectly used or that the

observed format is inappropriately used elsewhere. As a result, the sampling strategy could be

modified to encompass other display formats. In some cases, discovering these discrepancies

could warrant further review in the identified areas of concern.

(3) Additional Methodology Considerations for Plant Modifications - the following considerations

should be addressed:

64
;* The scope of HFE design verification may be restricted to the modified HSIs and their

interactions with the rest of the HSIs.

* When both old and new versions of similar HSIs are permanently present in design, this

verification should provide reasonable assurance that their means of presentation and

methods of operation are compatible, such that personnel performance will not be

impaired when the use of old and new components is alternated.

* HEDs should be identified for the following:

- failure to meet "crew-identified" functionality in addition to that specified by

system designers. When a digital system replaces an existing system, it is

important to make sure that all operational uses of the former system have been

addressed, even those that were not intended in the original design. The

replacement system's design should consider the actual usage of the former

system

:- poor integration with the rest of the HSI

- poor integration with procedures and training.

* Temporary configurations of the HSls and plant systems, which may be used by

operations and maintenance personnel when the plant is not shutdown, should be

reviewed to verify that their design is consistent with the principles of good HFE design,

including consistency with the rest of the HSIs.

(4) HED Documentation - HEDs, should be documented by the applicant in terms of the HSI

component involved and how its characteristics depart from a particular guideline.

11.4.3 Integrated System Validation

11.4.3.1 Integrated System Validation Review Objective

Integrated system validation is the process by which an integrated system design (i.e., hardware, software,

and personnel elements) is evaluated using performance-based tests to determine whether it acceptably

supports safe operation of the plant. It is intended to evaluate the acceptability of those aspects of the

design that cannot be determined though such analytical means as HSI task-support verification and HFE

design verification.

Plant personnel should perform operational events using a simulator or other suitable representation of the

system to determine its adequacy to support safety operations. This should be undertaken after significant

HEDs that were identified in verification reviews have been resolved, since these will negatively effect

performance and, therefore, the results of validation. (See O'Hara, et al., 1997 for a more detailed

discussion of integrated system validation methodology.)

For the case of plant modifications, the applicability and scope of integrated system validation may vary.

An integrated system validation should be reviewed for all modifications that may (1) change personnel

tasks; (2) change tasks demands, such as changing task dynamics, complexity, or workload; or (3) interact

with or affect HSIs and procedures in ways that mray degrade performance. Integrated system validation

may not be needed when a modification results in minor changes to personnel tasks such that they may

reasonably be expected to have little or no overall effect on workload and the likelihood of error. The

aspects of the validation that are addressed in the staffs evaluation are discussed below.

65
11.4.3.2 Integrated System Validation Review Criteria

11.4.3.2.1 Test Objectives

(1) Detailed objectives should be developed to provide evidence that the integrated system

adequately supports plant personnel in the safe operation of the plant. The test objectives and

scenarios should be developed to address aspects of performance that are affected by the

modification design, including personnel functions and tasks affected by the modification. The

objectives should be to:

* Validate the role of plant personnel.

* Validate that the shift staffing, assignment of tasks to crew members, and crew

coordination (both within the control room as well as between the control room and local

control stations and support centers) is acceptable. This should include validation of the

nominal shift levels, minimal shift levels, and shift turnover.

* Validate that for each human function, the design provides adequate alerting, information, control, and feedback capability for human functions to be performed under normal

plant evolutions, transients, design-basis accidents, and selected, risk-significant events

that are beyond-design basis.

Validate that specific personnel tasks can be accomplished within time and performance

criteria, with a high degree of operating crew situation awareness, and with acceptable

workload levels that provide a balance between a minimum level of vigilance and

operator burden. Validate that the operator interfaces minimize operator error and

provide for error detection and recovery capability when errors occur.

* Validate that the crew can make effective transitions between the HSIs and procedures in

the accomplishment of their tasks and that interface management tasks such as display

configuration and navigation are not a distraction or undue burden.

* Validate that the integrated system performance is tolerant of failures of individual HSI

features.

* Identify aspects of the integrated system that may negatively affect integrated system

performance.

* For modifications that change plant systems but do not modify the HSI, validation can

provide evidence about the adequacy of the existing HSIs, procedures, and training for

supporting personnel performance. The staff should verify that the applicant validates

that the functions and tasks allocated to plant personnel can be accomplished effectively

when the integrated design is implemented.

11.4.3.2.2 Validation Testbeds

A testbed is the HSI representation used to perform validation evaluations. One approach to identifying a

validation testbed that is consistent with the following review criteria, is to use the American National

Standard "Nuclear power plant simulators for use in operator training," (ANSUANS 3.5-1998) as a guide.

(1) Interface Completeness - The testbed should completely represent the integrated system. This

should include HSls and procedures not specifically required in the test scenarios. For example,

66
adjacent controls and displays may affect the ways in which personnel use those that are

addressed by a particular validation scenario.

(2) Interface Physical Fidelity - A high degree of physical fidelity in the HSls and procedures should

be represented, including presentation of alarms, displays, controls, job aids, procedures,

communications, interface management tools, layout and spatial relationships.

(3) Interface Functional Fidelity - A high degree of functional fidelity in the HSls and procedures

should be represented. All HSI functions should be available. High functional fidelity includes

HSI component modes of operation, i.e., the changes in functionality that can be invoked on the

basis of personnel selection and/or plant states.

(4) Environment Fidelity - A high degree of environment fidelity should be represented. The

lighting, noise, temperature, and humidity characteristics should reasonably reflect that expected.

Thus, noise contributed by equipment, such as air handling units and computers should be

represented in validation tests.

(5) Data Completeness Fidelity - Information and data provided to personnel should completely

represent the plant systems monitored and controlled from that facility.

(6) Data Content Fidelity - A high degree of data content fidelity should be represented. The

information and controls presented should be based on an underlying model that accurately

reflects the reference plant. The model should provide input to the HSI in a manner such that

information accurately matches that which will actually be presented.

(7) Data Dynamics Fidelity - A high degree of data dynamics fidelity should be represented. The

process model should be capable of providing input to the HSI in a manner such that information

flow and control responses occur accurately and in a correct response time; e.g., information

should be provided to personnel with the same delays as would occur in the plant.

(8) For important actions at complex HSIs remote from the main control room, where timely and

precise human actions are required, the use of a simulation or mockup should be considered to

verify that human performance requirements can be achieved. (For less risk-important HAs or

where the HSIs are not complex, human performance may be assessed based on analysis such as

task analysis rather than simulation.)

(9) The testbeds should be verified for conformance to the testbed characteristics identified above

before validations are conducted.

11.4.3.2.3 Plant Personnel

(1) Participants in the validation tests should be representative of actual plant personnel who will

interact with the HSI, e.g., licensed operators rather than training or engineering personnel.

(2) To properly account for human variability, a sample of participants should be used. The sample

should reflect the characteristics of the population from which the sample is drawn. Those

characteristics that are expected to contribute to system performance variation should be

specifically identified and the sampling process should provide reasonable assurance that

variation along that dimension is included in the validation. Several factors that should be con67
sidered in determining representativeness include: license and qualifications, skill/experience,

age, and general demographics.

(3) In selection of personnel, consideration should be given to the assembly of minimum and normal

crew configurations, including shift supervisors, reactor operators, shift technical advisors, etc.,

that will participate in the tests.

(4) To prevent bias in the sample, the following participant characteristics and selection practices

should be avoided:

* participants who are part of the design organization

* participants in prior evaluations

* participants who are selected for some specific characteristic, such as using crews that are

identified as good or experienced.

11.4.3.2.4 Scenario Definition

(1) The operational conditions selected for inclusion in the validation tests should be developed in

detail so they can be performed on a simulator. The following information should be defined to

provide reasonable assurance that important performance dimensions are addressed and to allow

scenarios to be accurately and consistently presented for repeated trials:

* description of the scenario and any pertinent "prior history" necessary for personnel to

understand the state of the plant upon scenario start-up

* specific initial conditions (precise definition provided for plant functions, processes,

systems, component conditions and performance parameters, e.g., similar to plant shift

turnover)

* events (e.g., failures) to occur and their initiating conditions, e.g., time, parameter values,

or events

* precise definition of workplace factors, such as environmental conditions

* task support needs (e.g., procedures and technical specifications)

* staffing objectives

* communication requirements with remote personnel (e.g., load dispatcher via telephone)

* the precise specification of what, when and how data are to be collected and stored

(including videotaping requirements, questionnaire and rating scale administrations)

* specific criteria for terminating the scenario.

(2) Scenarios should have appropriate task fidelity so that realistic task performance will be observed

in the tests and so that test results can be generalized to actual operation of the real plant.

(3) When evaluating performance associated with operations remote from the main control room, the

effects on crew performance due to potentially harsh environments (i.e., high radiation) should be

realistically simulated (i.e., additional time to don protective clothing and access radiologically

controlled areas).

68
11.4.3.2.5 Performance Measurement

The review criteria for performance measurement are divided into three sections. Section 11.4.3.2.5.1

addresses the measurement characteristics that effect the quality of the performance measures, Section

11.4.3.2.5.2 addresses the identification and selection of variables to represent measures of performance,

and Section 11.4.3.2.5.3 addresses the development of performance criteria.

11.4.3.2.5.1 Measurement Characteristics

(1) Performance Measurement Characteristics - Performance measures should acceptably exhibit the

following measurement characteristics to provide reasonable assurance that the measures are of

good quality (it should be noted that some of the characteristics identified below may not apply to

every performance measure):

* Construct Validity - A measure should accurately represent the aspect of performance to

be measured..

* Diagnosticity - A measure should provide information that can be used to identify the

cause of acceptable or unacceptable performance.

* Impartiality - A measure should be equally capable of reflecting good as well as bad

performance.

* Objectivity - A measure should be based on phenomena that are easily observed.

* Reliability - A measure should be repeatable; i.e., if the same behavior is measured in

exactly the same way under identical circumstances, the same measurement result should

be obtained.

* Resolution - A measure should reflect the performance at an appropriate level of

resolution, i.e., with sufficient detail to permit a meaningful analysis.

* Sensitivity - A measure's range (scale) and the frequency of measurement (how often data

are collected) should be appropriate to the aspect of performance being assessed.

* Simplicity - A measure should be simple both from the standpoint of executing the tests

and from the standpoint of communicating and comprehending the meaning of the

measures.

* Unintrusiveness - A measure should not significantly alter the psychological or physical

processes that are being investigated.

11.4.3.2.5.2 Performance Measure Selection

(1) A hierarchal set of performance measures should be used which includes measures of the

performance of the plant and personnel (i.e., personnel tasks, situation awareness, cognitive

workload, and anthropometric/physiological factors). 'Some of these measures could be used as

"pass/fail" criteria for validation and the others to better understand personnel performance and to

facilitate the analysis of performance errors. The applicant should identify which are in each

category.

(2) Plant Performance Measurement - Plant performance measures representing functions, systems,

components, and HSI use should be obtained.:

69
(3) Personnel Task Measurement - For each specific scenario, the tasks that personnel are required to

perform should be identified and assessed. Two types of personnel tasks should be measured:

primary (e.g., start a pump), and secondary (e.g., access the pump status display). Primary tasks

are those involved in performing the functional role of the operator to supervise the plant; i.e.,

monitoring, detection, situation assessment, response planning, and response implementation.

Secondary tasks are those personnel must perform when interfacing with the plant, but which are

not directed to the primary task, such as navigation and HSI configuration. This analysis should

be used for the identification of potential errors of omission.

Primary tasks should be assessed at a level of detail appropriate to the task demands. For

example, for some simple scenarios, measuring the time to complete a task may be

sufficient. For more complicated tasks, especially those that may be described as

knowledge-based, it may be appropriate to perform a more fine-grained analysis such as

identifying task components: seeking specific data, making decisions, taking actions, and

obtaining feedback. Tasks that are important to successful integrated system

performance and are knowledge-based should be measured in a more fine-grained

approach.

* The measurement of secondary tasks should reflect the demands of the detailed HSI

implementation, e.g., time to configure a workstation, navigate between displays, and

manipulate displays (e.g., changing display type and setting scale).

* The tasks that are actually performed by personnel during simulated scenarios should be

identified and quantified. (Note that the actual tasks may be somewhat different from

those that should be performed). Analysis of tasks performed should be used for the

identification of errors of commission.

* The measures used to quantify tasks should be chosen to reflect the important aspects of

the task with respect to system performance, such as:

- time

- accuracy

- frequency

- errors (omission and commission)

- amount achieved or accomplished

- consumption or quantity used

- subjective reports of participants

- behavior categorization by observers

(3) Situation Awareness - Personnel situation awareness should be assessed. The approach to

situation awareness measurement should reflect the current state-of-the-art.

(4) Cognitive Workload - Personnel workload should be assessed. The approach to workload

measurement should reflect the current state-of-the-art.

(5) Anthropometric and Physiological Factors - Anthropometric and physiological factors include

such concerns as visibility of indications, accessibility of control devices, and ease of control

device manipulation that should be measured where appropriate. Attention should be focused on

those aspects of the design that can only be addressed during testing of the integrated system,

e.g., the ability of personnel to effectively use the various controls, displays, workstations, or

consoles in an integrated manner.

70
11.4.3.2.5.3 Performance Criteria

(1) Criteria should be established for the performance measures used in the evaluations. The specific

-criteria that are used for decisions as to whether the design is validated or not should be specified

and distinguished from those being used to better understand the results.

(2) The basis for criteria should be defined, e.g., requirement-referenced, benchmark referenced,

normative referenced, and expert-judgement referenced.

11.4.3.2.6 Test Design

The review criteria for test design are divided into five sections. Section 1 1.4.3.2.6.1 addresses coupling

crews and scenarios, Section 11.4.3.2.6.2 addresses test procedures, Section 11.4.3.2.6.3 addresses the

training of test conductors, Section 11.4.3.2.6.4 addresses the training of test participants, and Section

11.4.3.2.6.5 addresses the conduct of pilot studies.

11.4.3.2.6.1 Coupling Crews and Scenarios

(1) Scenario Assignment - Important characteristics of scenarios should be balanced across crews.

Random assignment of scenarios to crews is not recommended. The value of using random

assignment to control bias is only effective when the number of crews is quite large. Instead, the

validation team should attempt to provide each crew with a similar and representative range of

scenarios.

(2) Scenario Sequencing - The order of presentation of scenario types to crews should be carefully

balanced to provide reasonable assurance that the same types of scenarios are not always being

presented in the same linear position, e.g., the easy scenarios are not always presented first.

11.4.3.2.6.2 Test Procedures

(1) Detailed, clear, and objective procedures should be available to govern the conduct of the tests.

These procedures should include:

* The identification of which crews receive which scenarios and the order that the

scenarios should be presented.

* Detailed and standardized instructions for briefing the participants. The type of

instructions given to participants can affect their performance on a task. This source of

bias can be minimized by developing standard instructions.

* Specific criteria for the conduct of specific scenarios, such as when to start and stop

scenarios, when events such as faults are introduced, and other information discussed in

Section 11.4.3.2.4, Scenario Definition.

* Scripted responses for test personnel who will be acting as plant personnel during test

scenarios. To the greatest extent possible, responses to communications from operator

participants to test personnel (serving as surrogate for personnel outside the control room

personnel) should be prepared. -There are limits to the ability to preplan communications

since personnel may ask questions or make requests that were not anticipated. However,

efforts should be made to detail what information personnel outside the control room can

provide, and script the responses to likely questions.

71
* Guidance on when and how to interact with participants when simulator or testing

difficulties occur. Even when a high-fidelity simulator is used, the participants may

encounter artifacts of the test environment that detract from the performance for tasks that

are the focus of the evaluation. Guidance should be available to the test conductors to

help resolve such conditions.

* Instructions regarding when and how to collect and store data. These instructions should

identify which data are to be recorded by:

- simulation computers

- special purpose data collection devices (such as situation awareness data

collection, workload measurement, or physiological measures)

- video recorders (locations and views)

- test personnel (such as observation checklists)

- subjective rating scales and questionnaires.

* Procedures for documentation, i.e., identifying and maintaining test record files including

crew and scenario details, data collected, and test conductor logs. These instructions

should detail the types of information that should be logged (e.g., when tests were

performed, deviations from test procedures, and any unusual events that may be of

importance to understanding how a test was run or interpreting test results) and when it

should be recorded.

(2) Where possible, test procedures should minimize the opportunity of tester expectancy bias or

participant response bias.

11.4.3.2.6.3 Test Personnel Training

(1) Test administration personnel should receive training on:

* the use and importance of test procedures

* experimenter bias and the types of errors that may be introduced into test data through the

failure of test conductors to accurately follow test procedures or interact properly with

participants

* the importance of accurately documenting problems that arise in the course of testing,

even if due to test conductor oversight or error.

11.4.3.2.6.4 Participant Training

(1) Participant training should be of high fidelity; i.e., highly similar to that which plant personnel

will receive in an actual plant. The participants should be trained to provide reasonable assurance

that their knowledge of plant design, plant operations, and use of the HSIs and procedures is

representative of experienced plant personnel. Participants should not be trained specifically to

perform the validation scenarios.

(2) Participants should be trained to near asymptotic performance (i.e., stable, not significantly

changing from trial to trial) and tested prior to conducting actual validation trials. Performance

criteria should be similar to that which will be applied to actual plant personnel.

72
11.43.2.6.5 Pilot Testing

(I) A pilot study should be conducted prior to conducting the integrated validation tests to provide an

opportunity to assess the adequacy of the test design, performance measures, and data collection

methods.

(2) If possible, participants who will operate the integrated system in the validation tests should not

be used in the pilot study. If the pilot study must be conducted using the validation test

participants, then:

* the scenarios used for the pilot study should be different from those used in the validation

tests, and

* care should be given to provide reasonable assurance that the participants do not become

so familiar with the data collection process that it may result in response bias.

11.4.3.2.7 Data Analysis and Interpretation,

(1) Validation test data should be analyzed through a combination of quantitative and qualitative

methods. The relationship between observed performance data and the established performance

criteria should be clearly established and justified based upon the analyses performed.

(2) For performance measures used as pass/fail indicators, failed indicators must be resolved before

the design can be validated. Where performance does not meet criteria for the other performance

measures, the results should be evaluated using the HED evaluation process.

(3) The degree of convergent validity should be evaluated, i.e., the convergence or consistency of the

measures of performance.

(4) The data analyses should be independently verified for correctness of analysis.

(5) The inference from observed performance to estimated real-world performance should allow for

margin of error; i.e., some allowance should be made to reflect the fact that actual performance

may be slightly more variable than observed validation test performance.

11.4.3.2.8 Validation Conclusions

(1) The statistical and logical bases for determining that performance of the integrated system is and

will be acceptable should be clearly documented.

(2) Validation limitations should be considered in terms of identifying their possible effects on

validation conclusions and impact on design implermentation. These include:

* aspects of the tests that were not well controlled

* potential differences between the test situation and actual operations, such as absence of

productivity-safety conflicts

* potential differences between the validated design and plant as built (if validation is

directed to an actual plant under construction where such information is available or a

new design using validation results of a predecessor)

73
11.4.4 Human Engineering Discrepancy Resolution

HED Resolution is an activity that can be performed iteratively with V&V. That is, the applicant may

integrate these activities so that issues identified during a V&V activity are addressed and resolved prior

to conducting other V&V activities.

The purpose of the staffs review of the HED Resolution is to verify that the applicant has adequately

completed the following tasks:

* evaluated HEDs to determine the need for their correction

* identified design solutions to address significant HEDs

* verified the implementation of the design solutions resolving HEDs.

HEDs should not considered in isolation and, to the extent possible, their potential interactions should be

considered when developing and implementing solutions. For example, if the HSI for a single plant

system is associated with many HEDs, then the set of design solutions should be coordinated to enhance

overall performance and avoid incompatibilities between individual solutions. Approaches that develop

design solutions to some HEDs before all have been identified from a particular verification or validation

activity are acceptable provided that the potential interactions between HEDs are specifically considered

prior to implementing the design solutions.

11.4.4.1 Human Engineering Discrepancy Resolution Review Objective

The objectives of the review are to verify that:

* The applicant's HED evaluation acceptably prioritizes HEDs in terms of their need for

improvement. (An HED evaluation is required only if the applicant does not plan to

correct all HEDs. If all HEDs are to be corrected, design improvements should be

identified, see Review Criteria 4 to 6 below).

* The applicant develops design solutions and a realistic schedule for implementation to

address those HEDs selected for correction.

11.4.4.2 Human Engineering Discrepancy Resolution Review Criteria

(I) HED Justification - Discrepancies could be acceptable within the context of the fully integrated

design. If sufficient justification exists, a deviation from the guidelines may not constitute an

HED. The technical basis for such a determination could include an analysis of recent literature

or current practices, tradeoff studies, or design engineering evaluations and data. Unjustified

discrepancies should be identified as HEDs to be addressed by the HED resolution.

(2) HED Analysis - The following should be included in the HED evaluations:

* Plant system - the potential effects of all HEDs relevant to a single plant system should

be evaluated. The potential effects of these HEDs on plant safety and personnel

performance should be determined, in part, by the safety significance of the plant

system(s), their effect on SAR accident analyses, and their relationship to risk significant

sequences in the plant PRA.

* HED scope

74
- Global features HEDs - these are HEDs that relate to configurational and

environmental aspects of the design such as lighting, ventilation, and traffic flow.

They relate to general human performance issues.

- Standardized features HEDs - these are HEDs that relate to design features that

are governed by the applicant's design guidelines used across various controls

and displays of the HSI (e.g., display screen organization and conventions for

format, coding, and labeling). Because a single guideline may be used across

many aspects of the design, a single HED could be applicable to many personnel

tasks and plant systems.

- Detailed features HEDs - these are HEDs that relate to design features that are

not standardized, thus there generality has to be assessed.

- Other - this subcategory specifically pertains to HEDs identified from integrated

system validation that cannot be easily assigned to any of the three preceding

categories.

* Individual HSI or procedure - HEDs should be analyzed with respect to individual HSIs

and procedures. The potential effects of these HEDs on plant safety and personnel

performance are determined, in part, by the safety significance of the plant system(s) that

are related to the particular component.

* Personnel function - HEDs should be analyzed with respect to individual personnel

functions ' The potential effects of these HEDs is determined, in part, by the importance

of the personnel function to plant safety (e.g., consequences of failure) and their

cumulative effect on personnel performance (e.g., degree of impairment and types of

potential errors).

HEDs should also be analyzed with respect to the cumulative effects of multiple HEDs on plant

safety and personnel performance. While an individual HED might not be considered sufficiently

severe to require correction, the combined effect of several HEDs upon the single aspect of the

design could have significant consequences to plant safety and, therefore, necessitate corrective

action. Likewise, when a single plant system is associated with multiple HEDs that affect a

number of HSI components, then their possible combined effect on the operation of that plant

system should be considered

In addition to addressing the specific HEDs, the analysis should treat the HEDs as indications of

potentially broader problems. For example, identifying multiple HEDs associated with one

particular aspect of the HSI design,'such as the remote shutdown panel, could also indicate that

there are other problems with that aspect of the design, such as inconsistent use of procedures and

standards; In some cases, the evaluation of HEDs could warrant further review in the identified

areas of concern.

(3) HED Prioritization - Identification of HEDs for correction should be based upon a systematic

evaluation, such as that illustrated in Figure 11.2. Priority I HEDs should be those with direct

safety consequences and those with indirect or potential safety consequences. HEDs with

significant safety'consequences are those that affect personnel performance where the consequences of error could reduce the margin of plant safety below an acceptable level, as indicated

by such conditions as violations of Technical Specification safety limits, operating limits, or

limiting conditions for operations. They include deviations from personnel information

requirements or HFE guidelines for personnel tasks that 'are related to plant safety. These could

include the following:

75
* are required by personnel tasks but are not provided by the HSI

* do not satisfy all personnel information needs (e.g., information not presented with the

proper range or precision)

* contain deviations from HFE guidelines that are likely to lead to errors that would

prevent personnel from performing the task.

HEDs with indirect safety consequences include deviations from HFE guidelines that would

seriously affect the ability of personnel to perform the task. The severity of an HFE guideline

deviation should be assessed in terms of the degree to which it contributes to human performance

problems, such as workload and information overload.

Priority 2 HEDs should be those that do not have significant safety consequences, but do have

potential consequences to plant performance/operability, non-safety-related personnel

performance/efficiency, or other factors affecting overall plant operability. These include

deviations from personnel information requirements and HFE guidelines for tasks associated with

plant productivity, availability, and protection of investment. These HEDs should be considered

for correction.

The remaining HEDs are those that do not satisfy the criteria associated with the first and second

priorities. Resolution of these HEDs is not an NRC safety concern but may be resolved at the

discretion of the applicant.

(4) HED Evaluation Documentation - Each HED should be fully documented including assessment

category (priority for correction), associated plant system, associated personnel function, and

associated HSI or procedure. The documentation should clearly show whether the HED was

dismissed or identified as needing design modification, and the basis for this determination in

terms of consequence to plant safety or operation should be clearly described.

(5) Development of Design Solutions - Design solutions to correct HEDs should be identified. The

design solutions should be consistent with system and personnel requirements identified in the

Preparatory Analysis (i.e., Operating Experience Review, Function and Task Analysis, and HSI

Characterization).

Inter-relationships of individual HEDs should be evaluated. For example, if a single HSI

component is associated with multiple HEDs, then design solutions should be considered to

address these HEDs together. If a single plant system is associated with multiple HSI

components that are associated with HEDs, then the design of the individual solutions should be

coordinated so that their combined effect enhances rather than detracts from that system's

operation.

(6) Design Solution Evaluation - Designs should be evaluated by repeating the appropriate analyses

of the verification and validation. For example, the HSI Task Support Verification should be

conducted to provide reasonable assurance that the design satisfies personnel task requirements.

Portions of the HFE design verification analysis should be conducted to provide reasonable

assurance that the design is consistent with HFE guidelines, and integrated system validation

could be conducted to evaluate its usability. When the problems identified by an HED cannot be

fully corrected, justification should be given.

76
Vt )

Figure 11.2 The HED evaluation process

77
(7) Design Modification - There should be an implementation schedule for activities associated with

installing, testing, and HFE evaluation of the design solutions. All design solutions for Priority 1

HEDs should be scheduled for prompt implementation. The schedule should be developed to

minimize demands and disruptions for personnel. For operating plants, the schedule should

distinguish between solutions that can be implemented without interfering with the operation of

the plant, and improvements that can only be made when the plant is not operating. Installing

large groups of design solutions at discrete intervals should be considered to avoid subjecting

operating crews to a continually changing HSI. Procedures should be established to provide

reasonable assurance that information related to the design of the HSI such as plant procedures,

drawings, and training programs is updated to reflect the changes.

11.5 Sources of Additional Information

The following documents may be used as guidance (per Section 1.2.2):

ANSIIAIAA G-035-1992: Guide to Human Performance Measurements (American National

Standards Institute, 1993).

IEC 1771: Nuclear Power Plants Main Control Rooms-- Verification and Validation of Design

(International Electrotechnical Commission, 1995).

IEEE Std. 845-1999: IEEE Guide to the Evaluation of Human-System Performance in Nuclear

Power Generating Stations, 1999 (Institute of Electrical and Electronics Engineers, 1999).

NUREG/CR-6393: Integrated System Validation: Methodology and Review Criteria (O'Hara et

al., 1997)

NUREG-0700: Human-System Interface Design Review Guidelines (NRC, 2002).

Regulatory Guide 1.33: Quality Assurance Program Requirements (NRC, 1978).

In addition, documents listed for the following elements can be used to support V&V activities:

* HSI Design

* Procedure Development

* Training Program Development

78
12 DESIGN IMPLEMENTATION

12.1 Background

This section addresses the implementation of the HFE aspects of the plant design for both new plants and

plant modifications. For a new plant, the implementation phase is well defined and carefully monitored by

start-up procedures and testing; implementation of plant modifications is more complex.

Plant modifications can effect personnel in various ways. Changes to systems and components can

impact the role of personnel and the way their tasks are performed. Often such plant modifications lead

to changes in HSIs, procedures, and training as well. Modifications can also address the HFE aspects of

the plant even though the plant's systems and components are not changed.

There are many different ways that modifications can be implemented. Some of these approaches and

their advantages and disadvantages are illustrated in Table 12-1. The HFE considerations for each are

somewhat unique.

For both new and modified designs, it is important to determine that the design that is implemented (i.e.,

the "as-built" design) accurately reflects the verified and validated design.

Table 12.1 Typical advantages and disadvantages of

different methods of modernization program implementation

Many Small Modifications

Advantages

Minima

Disadvantages

I disruption to operations.

* Risk of unexpectedly affecting plant operation (such as through spurious actuation)

* Likelihood for inconsistency and lack of standardization of HSls as many new and different system are

separately added to the control room. As a result, operators are unsure precisely how each HSI functions.

* Overlapping functionality - many HSls do the same things and provide the same functions.

* Training on small modifications may be lacking, so personnel do not use the new systems effectively or at

all.

Large Modifications During a Single Outage -

Advantages

* The potential for negative effects on crew performance of interim configurations is eliminated, since all the

changes are made at once.

* Cost savings in comparison to multiple outages because (I) interim periods do not have to be analyzed, (2)

procedures and do not have to be temporarily modified, and (3) crews do not have to be trained for

temporary HSIs.

Disadvantages

Significant changes to the plant and HSIs can have significant effects on the way crews operate the plant.

Cost impact of loss of production during the extended outage.

Larce Modifications During Multiple Outages

Advantages

Disadvantages

iDrastic changes to operations can be minimized by breaking up modifications into smaller logical units.

Plant staff can gain experience with non-safety (less critical) systems so when safety (critical) systems are

modified the plant staff are in a more knowledgeable position.

Task performance can be hampered if the interim configuration requires parts of a task to be performed

using the old HSI and other parts with the new HSI.

79
* Interim stages between old and new systems can be especially error prone if not fully addressed in analyses

and by training and procedure modifications.

Both Old and New Equipment are Left in Place

Advantages

* Any problems with the new system can be identified and resolved while the old HSIs are in place and can

act as backtips-. ,'

* Operators can become familiar with the new HSls white the old HSIs are still available.

* Old HSIs are available ifian'eniergency (research has showri that operators often prefer the familiar HSIs

when in stressful conditions)

Disadvantages i.

* HSI conflicts between old and new systems (such as different values for the same process parameter)

Control room clutteraid potential distraction of two sets of HSIs

New Non-Functional HSIs in Place in Parallel with Old Functional HSIs

Advantages

Operators can become familiar with the new HSIs while the old HSIs are still available.

Disadvantages -

* Control room clutter and potential distraction of two sets of HSIs

* Crew may use the new HSIs inadvertently or because they do not realize that the HSIs are non-functional.

12.2 Objective

The objective of this review is to verify that:

* the applicant's implementation of plant changes considers the effect on personnel

performance and provides the necessary support to provide reasonable assurance safe

operations

* the applicant's as-built design conforms to the verified and validated design that resulted

from the HFE design process.

12.3 Applicant Submittals

As per Section 1.2.1, item (3) Applicant Submittals, the applicant should provide for staff review an

implementation plan for design implementation. Upon completion of the applicant's efforts, a results

summary report should be submitted so that the staff can review the applicant's design implementation

using the criteria provided in Section 12.4 below.

12.4 Review Criteria

The first five sections of review criteria are for the review of plant'ni6difications only. Section 12.4.6,

Final Plant HFE Design Verification, applies to both new and modified plant designs.

12.4.1 General Criteria

(1) The applicant should provide reasonable assurance that the reactorfiuel is safely monitored during

the shutdown time period while the physical modifications are being implemented in the control

room.

(2) Operations and maintenance crews should be fully trained and qualified to operate and maintainx

the plant with respect to all modifications prior to starting-up with the new systems and HSIs in

place.

80
(3) Modifications in plant procedures and training should reflect changes in plant systems, crew roles

and responsibilities, HSIs, and procedures resulting from the new systems and HSIs should be in

place prior to startup.

(4) The applicant should have a plan in place to monitor the initial phase of startup to provide

reasonable assurance that:

* operational and maintenance problems that arise with personnel interactions with the new

systems, HSIs, and procedures are identified and addressed

* personnel are sufficiently familiar with the new systems, HSIs, and procedures to support

safe operations and maintenance

* any negative transfer of training from the old removed HSIs to the corresponding new

HSIs is identified and corrected

* no new problems are created based on coordination of tasks between remaining old HSIs

and new HSIs

* no unanticipated negative effects on crew interaction and teamwork arise

12.4.2 Modernization Programs Consisting of Many Small Modifications

(I) Each modification should follow a HFE program that provides reasonable assurance of

standardization and consistency (I) between old and new equipment, and (2) across the new

systems being implemented.

(2) The applicant should verify that new modifications fulfill a clear operational need and do not

interfere with existing systems. For example, the auditory alerts in a new HSI should not distract

operators from addressing more important alarms in the main system.

12.4.3 Modernization Programs Consisting of Large Modifications During Multiple Outages

(1) Tlhe interim configurations should be carefully defined and evaluated to verify that they are

acceptable both from an engineering and operations perspective and meet regulatory

requirements. Evaluations should include:

* PRA

SAR

* Technical specifications

* Defense-in-depth

(3) Task analysis should be performed for each interim configuration to verify that the task demands

the are unique to interim configurations are known.

(4) HRA should address any unique tasks that may impact risk or any changes to existing tasks due

to the interim configuration.

(5) The HSIs needed to perform important tasks should be consistent and standardized. Task

performance should not require personnel to use both old and new HSIs for different aspects of

81
the same task. If the underlying I&C modifications necessitate this situation, consideration

should be given to creating temporary HSIs specifically designed for such tasks. !

(6) Procedures should be developed for temporary configurations of systems and HSIs that are used

by personnel when the plant is not shutdown.

(7) Training should be developed for temporary configurations of systems, HSIs, and procedures that

are used by personnel when the plant is not shutdown.

(8) Verification and Validation

* HFE Design Verification - Temporary configurations of the systems, HSIs, and

procedures, which may be used by operations and maintenance personnel when the plant

is not shutdown, should be reviewed to verify thattheir design is consistent with the

principles of good HFE design.

* HSI Task-Support Verification - Temporary configurations of the systems, HSIs, and

procedures, which may be used by operations and maintenance personnel when the plant

is notfshutdown, should be reviewed to verify that their design supports the tasks that will

be performed. For example, if a temporary configuration of plant systems introduces

special monitoring requirements, then the HSIs should provide the necessary information.

* Validation should be performed on interim configurations if warranted by the

risk-significance of the crew tasks affected by the temporary configuration.

12.4.4 Modernization Programs Where Both Old and New Equipment are Left in Place

(1) The potential for negative eects on personnel performance dde t.o control room or HSI clutter

arising from' having both old and new HSIs available in parallel should be evaluated. Where

safety concerns are identified, appropriate measures should be taken to improve the HSIs.

(2) The potential for negative effects on personnel performance due to the simultaneous presence of

parallel alarm systems should be evaluated. Where safety concerns are identified, appropriate

measures should be taken to improve the HSIs.

(3) The potential should be evaluated for negative effects on personnel performance due to the

differences between information from old and new systems for the same parameter or equipment.

Where safety concerns are identified, appropriate measures should be taken to improve the HSIs.

(4) An evaluation should be performed to identify any safety concerns from providing controls from

two different HSIs. Where a concern is identified, appropriate measures should be developed to

prevent the concern. For example, a switch may be added to select which HSI controls the

equipment thus preventing simultaneous control inputs.

12.4.5 Modernization Programs Where New Non-Functional ilSIs are in Place in Parallel with

Old Functional HSIs

(1) The potential for negative effects on personnel performance due to control room or HSI clutter

arising from having both old and new HSIs available in parallel should be evaluated. Where

- safety concerns are identified, appropriate measures should be taken to improve the HSIs..

82
(2) The non-functional state of the HSIs should be clearly indicated.

12.4.6 Final Plant HFE Design Verification

(1) Aspects of the design that were not addressed in V&V should be evaluated using an appropriate

V&V method. Aspects of the design addressed by this criterion may include design

characteristics such as new or modified displays for plant-specific design features and features

that cannot be evaluated in a simulator such as CR lighting and noise.

(2) The final (as-built in the plant) HSIs, procedures, and training should be compared with the

detailed design description to verify that they conform to the design that resulted from the HFE

design process and V&V activities. Any identified discrepancies should be corrected or justified.

(3) All HFE-related issues documented in the issue tracking system should be verified as adequately

addressed.

12.5 Sources of Additional Information

The following document may be used for additional information (per Section 1.2.2):

10 CFR Part 50.59: U.S. Code of Federal Regulations, Part 50.59 "Changes, Tests, and

Experiments," Title 10, "Energy."

IEC 62096 Nuclear Power Plants - Instrumentation and Control: Guidance for the Decision on

Modernization (IEC, 2001).

NEI 96-01, Rev. 1: Guidelinesfor JO CFR 50.59Implementation (Nuclear Energy Institute,

2000).

83
13 HUMAN PERFORMANCE MONITORING

13.1 Background

A human performance monitoring strategy will help to provide reasonable assurance that the confidence

developed by the completion of the integrated system validation is maintained over time. There is no

intent to periodically repeat the fill integrated system validation; however, there should be sufficient

evidence to provide reasonable confidence that plant personnel have maintained the skills necessary to

accomplish the assumed actions.

13.2 Objective

The objective of this review is to verify that the applicant has prepared a human performance monitoring

strategy for ensuring that no significant safety degradation occurs because of any changes that are made

in the plant and to provide adequate assurance that the conclusions that have been drawn from the

evaluation remain valid over time. The applicant may incorporate this monitoring strategy into their

problem identification and corrective action program

13.3 Applicant Submittals

Submittals for the staffs review of an applicant's human performance monitoring program should be

made on a case-by-case basis.

13.4 Review Criteria

(1) The scope of the performance monitoring strategy should provide reasonable assurance that:

* The design can be effectively used by personnel, including within the control room and

between the control room and local control stations and support centers.

* Changes made to the HSIs, procedures, and training do not have adverse effects on

personnel performance, e.g., a change interferes with previously trained skills.

* Human actions can be accomplished within time and performance criteria.

* The acceptable level of performance established during the integrated system validation

is maintained.

(2) A human performance monitoring strategy should be developed and documented. The strategy

should be capable of trending human performance after the changes have been implemented to

demonstrate that performance is consistent with that assumed in the various analyses that were

conducted to justify the change. Applicants may integrate, or coordinate, their performance

monitoring for risk-informed changes with existing programs for monitoring personnel

performnance, such as the licensed operator training program and the corrective action program. If

a plant change requires monitoring of actions that are not included in existing training programs,

it may be advantageous to adjust the existing training program rather than to develop additional

monitoring programs for risk-informed purposes.

85
(3) The program should be structured such that {I

* human actions are monitored commensurate with their safety importance

v feedback of information and corrective actions are accomplished in a timely manner

* degradation in pe formance can be detected and c66rected before plant safety is

compromised (erg., by use of the plant simulator during periodic training exercises)

(4) Plant or personnel perfoianice under actual design conditions may not be readily measurable.

When actual conditions cannot be simulated, monitored, or measured, the available information

that most closely approximates performance data in actual conditions should be used.

(5) As part of the monitoring program, it is important that provisions for specific cause

''deeniation,' trending'of performaince-degradation and failures; and corrective actions be

'included' The cause deterrm ination should identify' the cause ofthe failure'or degraded

' performahc-e t6 the extent thatco-rrective action '6an be identified that would preclude the problem

or provide 'dequate assurance that'it is anticipated prior to becoming a safety concern. The

program should address failure significance, the circumstances surrounding the failure or

degraded performance, the characteristics of the failure, and whether the failure is isolated or has

generic or common cause implications. The monitoring program should identify and establish

any corrective actions necessary to preclude the recurrence of unacceptable failures or degraded

performance.

13.5 Sources of Additional Information

The following documents may be used for additional information (per Section 1.2.2):

CN Number 02-001: NRC Inspection Manual: Chapter 2515, Light- Water Reactor Inspection

Program - Operations Phase (NRC, 2002);

CN Number 01-015: NRC Inspection Manual: Chapter 0609, Significance Determination

Process (NRC, 2001).

IP 71715: Sustained Control Room and Plant Observation. (NRC, periodically updated).

NUREG-1649: Reactor Oversight Process (NRC, 2000).

Regulatory Guide 1.174: An approach for using probabilistic risk assessment in risk-informed

decisions on plant-spec ic changes to the licensing basis (NRC, 1998).

86
14 REFERENCES

American Nuclear Society (1999). Selection, Qualification, and Training of Personnelfor Nuclear

Pover Plants (ANSI/ANS 3.1-1993,R1999). LaGrange Park, Illinois: American Nuclear Society.

American Nuclear Society (1994). Administrative Controls and QualityAssurancefor the Operational

Phase of NPPs (ANS 3.2-1994). LaGrange Park, IL: American Nuclear Society.

American National Standards Institute (1993). Guide to Human Performance Measurements

(ANSI/AIAA G-03 5-1992). Washington, DC: American National Standards Institute.

American National Standards Institute (1994). Time Response Design Criteria for Safety-Related

Operator Actions (ANSI/ANS 58.8-1994). La Grange Park, IL: American National Standards Institute.

American National Standards Institute (1988). American National Standardfor Human Factors

Engineering of Visual Display Terminal Workstations (ANSI HFS-100-1988). Santa Monica, California:

American National Standards Institute.

American National Standards Institute (1998). Nuclear Power Plant Simulatorsfor Use in Operator

Training (ANSIIANS-3.5-1998). La Grange Park, IL: American National Standards Institute.

Barnes, V., Moore, C., Wieringa, D., Isakson, C., Kon6, B., and Gruel, R. (1989). Techniquesfor

Preparing Flowchart Format Emergency Operating Procedures (NUREG/CR-5228, Volumes 1 and 2).

Washington, DC: U.S. Nuclear Regulatory Commission.

Billings, C. (1991). Human-Centered.AircraftAutomation: A Concept and Guidelines (NASA Tech.

Memo No. 103885) Moffett Field, CA: NASA-Ames Research Center.

Bongarra, J., Van Cott, H., Pain, R., Peterson, L., and Wallace, R. (1985). Human factors design

guidelines for maintainability ofDepartment of Energy Nuclear Facilities (Tech. Report UCRL- 15673).

Livermore, CA: Lawrence Livermore National Laboratory.

Brown, W., O'Hara, J., & Higgins, J. (2000). AdvanceAlarm Systems: Guidance Development and

Technical Basis (NUREG/CR-6684). Washington, DC: U.S. Nuclear Regulatory Commission.

Burgy, D., Lempges, C., Miller, A., Schroeder, L. Van Cott, H., Paramore, B. (1983). TaskAnalysis of

Nuclear Power Plant Control Room Crews (NUREG/CR-3371, Volumes 1 and 2). Washington, DC: U.S.

Nuclear Regulatory Commission.

Bye, A. and Hoffiman, M. (2001) FAME - A Function Allocation Design Tool (HWR-693). Halden,

Norway: OECD Halden Reactor Project.

Electric Power Research Institute (EPRI) (1992). Advanced Light Water Reactor Utility Requirements

Document, Volume II, Evolutionary Plant (Revision 4). Palo Alto, California: Electric Power Research

Institute.

Higgins, J. and Nasta, K. (1996). HFE Insights ForAdvanced Reactors Based Upon Operating

Experience (NUREG/CR-6400). Washington, DC: U.S. Nuclear Regulatory Commission.

87
Higgins, J. and O'Hara, J., (2000). Proposed Approach for Reviewing Changes to Risk-Important Human

Actions (NUREG/CR-6689). Washington, DC: U.S. Nuclear Regulatory Commission.

Institute of Electrical and Electronics Engineers (1999). IEEE Guide to the Evaluation of Human-System

Performance in Nuclear Power Generating Stations (IEEE Std. 845-1999). New York: Institute of

Electrical and Electronics Engineers.

Institute of Electrical and Electronics Engineers (1988). IEEE Guide to the Application of Human

Factors Engineering to Systems, Equipment, and Facilities of Nuclear Power Generating Stations (IEEE

Std. 1023-1988). New-York: Institute of Electrical and Electronics Engineers.

Institute of Electrical and ElectronicsEngineers (1997). A GuideforIncorporatingHuman Action

Reliability Analysisfor Nuclear Powuer Generating Stations (IEEE Std. 1082-1997). New York: Institute

of Electrical and Electronics Engineers.

International Atomic Energy Agency (1992). The Role ofAutomation and Humans in NuclearPower

Plants (IAEA-TECDOC-668). Vienna, Austria: International Atomic Energy Agency.

International Atomic Energy Agency (1988). Basic Safety Principles for Nuclear Power Plants (Safety

Series No. 75-4NSAG-3).- Vienna, Austria: International Nuclear Safety Advisory Group.

International Electrotechnical Commission (2001). Nuclear Power Plants - Instrumentation and Control:

Guidancefor the Decision on Modernization (EC 62096). Geneva, Switzerland: Bureau Central de la

Commission Electrotechnique Internationale. .

International Electrotechnical Commission (1995). Nuclear Power Plants Main Control Rooms--

Verification and Validation ofDesign (IEC Standard 1771). Geneva, Switzerland: Bureau Central de la

Commission Electrotechnique Internationale.

International Electrotechnical Commission (1989). Design for Control Rooms of Nuclear Power Plants

(IEC-964). Geneva, Switzerland: Bureau Central de la Commission Electrotechnique Internationale.

International Standards Organization (2000). Ergonomic Design of Control Centres -- Part 1: Principles

for the Design of Control Centres (ISO- 11064-1). Geneva, Switzerland: International Standards

Organization.

Kinkade, R.G., and Anderson, J. (1984). Human Factors Guidejfor Nuclear Power Plant Control Room

Development (EPRI NP-3659). Palo Alto, California: Electric Power Research Institute.

Kirwan, B. and Ainsworth L. (Eds). (1992). A Guide to Task Analysis. London: Taylor and Francis.

Nuclear Energy Institute (2000). Guidelines for 10 CFR 50.59 Implementation (NEI 96-01, Rev. 1).

Washington, DC: Nuclear Energy Institute.

NRC (periodically updated). Sustained Control Room and Plant Observation (IP 71715).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (periodically updated). Plant Procedures (IP 42700). Washington, DC: U.S. Nuclear Regulatory

Commission.

88
NRC (periodically updated). Emergency Operating Procedures (IP 42001). Washington, DC: U.S.

Nuclear Regulatory Commission.

NRC (periodically updated). Training and Qualification Effectiveness (IP 41500). Washington, DC: U.S.

Nuclear Regulatory Commission.

NRC (Revised periodically). TrainingReview Criteria and Procedures (NUREG-1220).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (2004). Standard Review Plan, Chapter 18 - Human Factors Engineering (NUREG-0800).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (2002). Human System Interface Design Review Guidelines (NUREG-0700). Washington, DC:

U.S. Nuclear Regulatory Commission.

NRC (2002). NRC Inspection Manual: Chapter 2515, Light-Water Reactor Inspection Program -

Operations Phase (CN Number 02-001). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (2002). Standard Review Plan: Chapter 19, Use of Probabilistic RiskAssessment in Plant-Specific,

Risk-Informed Decision Making: General Guidance (NUREG-0800). Washington, D.C.: U.S. Nuclear

Regulatory Commission.

NRC (2001). Nuclear Power Plant Simulatorsfor Use in Operator Training (Regulatory Guide 1.149,

Rev. 3). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (2001). NRC Inspection Manual: Chapter 0609, Significance Determination Process (CN Number

01-015). Washington, DC: U.S.NuclearRegulatory Commission.

NRC (2000). Reactor Oversight Process (NUREG- 1649). Washington, DC: U.S. Nuclear Regulatory

Commission.

NRC (2000). Personnel Selection and Training (Regulatory Guide 1.8, Rev 3). Washington, DC:

U.S. Nuclear Regulatory Commission.

NRC (2000). Guidancefor Implementation of 10 CFR 50.59, Changes. Tests, and Experiments

(Regulatory Guide 1.187). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (2000). Technical Basis and Implementation Guidelinesfor A Techniquefor Human Event Analysis

(ATHEANA) (NUREG-1624, Rev. 1). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1999). Standard Review Plan: Chapter 13, Conduct of Operations (NUREG-0800). Washington,

D.C.: U.S. Nuclear Regulatory Commission.

NRC (1999). Instrumentation Setpoints (Regulatory Guide 1.105, Rev. 3). Washington, DC: U.S.

Nuclear Regulatory Commission.

NRC (1998). AnApproachfor Plant-Specific, Risk-Informed Decisionmaking: Technical Specifications

(Regulatory Guide 1.177). Washington, DC: U.S. Nuclear Regulatory Commission.

89
NRC (1998). An approach for 'uiniipirobabilistic risk assessment in risk-informed decisions on plantspecific changes to the licensing basis (Regulatory Guide 1.174). Washington, DC: U.S. Nuclear

Regulatory Commission.

NRC (1997). Crediting of Operator Actions in Place ofAutomatic Actions and Modifications of

OperatorActions, IncludingResponse Times (Information Notice 97-78). Washington, DC: U.S. Nuclear

Regulatory Commission.

NRC (1997). Individual Plant Examination Program: Perspectives on Reactor Safety and Plant

Performance (NUREG-1560). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1995). Results Of Shift Staffing Study (Information Notice 95-48). Washington, DC: U.S.

Nuclear Regulatory Commission.

NRC (1992). Lessons Learned From the Special Inspection Program for Emergency Operating

Procedures, Supplement I (NUREG-1358): Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1992). Lessons Learned From the Special Inspection Program for Emergency Operating

Procedures (NUREG-1358, Supplement 1). Washington, DC: U.S. NuclearRegulatory Commission.

NRC (1989). Lessons Learned From the Special Inspection Program for Emergency Operating

Procedures (NUREG-1358). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1986). Guidance to Operators at the Controls and to Senior Operators in the Control Room of a

Nuclear Power Unit (Regulatory Guide 1.114, Rev. 2). Washington, DC: U.S. Nuclear Regulatory

Commission.

NRC (1983). Instrumentation for Light-Water-Cooled Nuclear Power Plants to Assess Plant and

Environmental Conditions During and Following an Accident (Regulatory Guide 1.97).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1982). Guidelinesfor the Preparation of Emergency Operating Procedures (NUREG-0899).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1982). Policy on Factors Causing Fatigue of Operating Personnel at Nuclear Reactors (Generic

Letter No. 82-12). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1980). Functional Criteriafor Emergency Response Facilities (NUREG-0696). Washington, DC:

U.S. Nuclear Regulatory Commission.

NRC '(1980). Criteria/for Preparation and Evaluation of Radiological Emergency Response Plans and

Preparedness in Support of Nuclear Power Plants (NUREG-0654). Washington, DC: U.S. Nuclear

Regulatory Commission.

NRC (1980). Clarification of TMIAction Plan Requirements (NUREG-0737 and supplements).

Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1978). Quality Assurance Program Requirements (Regulatory Guide 1.33, Revision 2).

Washington, DC: U.S. Nuclear Regulatory Commission.

90
NRC (1975). Shared Emergency and Shutdown Electrical Systems for Multi-Unit NPPs (Regulatory

Guide 1.81, Rev. 1). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (1973). Manual Initiation of Protective Actions (Regulatory Guide 1.62). Washington, DC:

U.S. Nuclear Regulatory Commission.

NRC (1973). Bypassed and Inoperable Status Indication for NPP Safety Systems (Regulatory Guide

1.47). Washington, DC: U.S. Nuclear Regulatory Commission.

NRC (I 972). Periodic Testing of Protection System Actuation Functions (Regulatory Guide 1.22).

Washington, DC: U.S. Nuclear Regulatory Commission.

O'Hara, J., Stubler, W., Brown, W. and Higgins, J. (1997). Integrated System Validation: Methodology

and Review Criteria (NUREG/CR-6393). Washington, DC: U.S. Nuclear Regulatory Commission.

O'Hara, J., Higgins, J., Stubler, W., and Kramer, J. (2000). Computer-Based Procedure Systems:

Technical Basis and Human Factors Review Guidance (NUREG/CR-6634). Washington, DC:

U.S. Nuclear Regulatory Commission.

O'Hara, J., Higgins, J., and Kramer, J. (2000). Advanced Information Systems: Technical Basis and

Human Factors Review Guidance (NUREG/CR-6633). Washington, DC: U.S. Nuclear Regulatory

Commission.

Pack, R., Seminara, J, Shewbridge, E., and Gonzalez, W. (1 95). Human Engineering Design Guidelines

for Maintainability (EPRI NP-4350). Palo Alto, California: Electric Power Research Institute.

Price, H. Maisano, R., and Van Cott, H. (1982). The Allocation of Functions in Man-Machine Systems:

A Perspective and Literature Review (NUREG/CR-2623). Washington, DC: U.S. Nuclear Regulatory

Commission.

Pulliam, R., Price, H., Bongarra, J., Sawyer, C., and Kisner, R. (1983). A Methodologyfor Allocation of

Nuclear Power Plant Control Functions to Human andAutomated Control (NUREG/CR-333 1).

Washington, DC: U.S. Nuclear Regulatory Commission.

Shraagen, J., Chipman, S., and Shalin, V. (2000). Cognitive TaskAnalysis. New Jersey: Lawrence

Erlbaum Associates.

Stubler, W., O'Hara, J., Higgins, J., and Kramer, J. (2000). Human-System Interface and Plant

Modernization Process: Technical Basis and Human Factors Review Guidance (NUREG/CR-663 7).

Washington, DC: U.S. Nuclear Regulatory Commission.

Stubler, W. and O'Hara, J. (1996). Group-View Displays: Functional Characteristics and Review

Criteria (BNL TR E2090-T4-4-12/94, Rev. 1). Upton, New York: Brookhaven National Laboratory.

Stubler, W., O'Hara, J., and Kramer, J. (2000). Soft Controls: Technical Basis and Human Factors

Review Guidance (NUREG/CR-6635). Washington, DC: U.S. Nuclear Regulatory Commission.

91
Stubler, W., Higgins, J., and Kramer, J. (2000). Maintenance of Digital Systems: Technical Basis and

Human Factors Review Guidance (NUREG/CR-6636). Washington, DC: U.S. Nuclear Regulatory

Commission.

Stubler, W. and O'Hara, J., (1996). Human-System Interface Design Process and Review Criteria

(BNL TR E2090-T4-1-9/96). Upton, New York: Brookhaven National Laboratory.

U.S. Code of Federal Regulations (revised periodically), Part 2.802, "Petition for Rulemaking" Title 10,

"Energy," Washington, DC: U.S;. G6v~ennent Printing Office..'

U.S. Code of Federal Regulations (revised periodically), Part 20, "Standards for Protection Against

Radiation" Title 10, "Energy," Washington, DC: U.S. Government Printing Office.

U.S. Code of Federal Regulations (revised periodically), Part 26, "Fitness for Duty Programs," Title 10,

"Energy," Washington, DC: U.S Government Printing Office.

U.S. Code of Federal Regulations (revised periodically), Part 50, "Domestic Licensing of Production and

Utilization Facilities," Title 10, "Energy," Washington, DC: U.S. Goveirnment Printing Office.>

U.S. Code of Federal Regulations (revised periodically), Part 50.59, "Changes, Tests, and Experiments,

"Title 10, Energy," Washington, DC: U.S. Government Printing Office.

US. Code of Federal Regulations (revised periodically), Part 50.120, "Training and Qualification of

Nuclear Power Plant Personnel," Title 10, "Energy," Washington, DC: U.S. Government Printing Office.

US. Code of Federal Regulations (revised periodically), Part 52, "Early Site Permits; Standard Design

Certifications; and Combined Licenses for Nuclear Power Plants," Title 10, "Energy," Washington, DC:

U.S. Government Printing Office.

US. Code of Federal Regulations (revised periodically), Part 55, "Operator's Licenses," Title 10,

"Energy," Washington; DC: U.S. Govemnment Printing Office.-

Vicente, K., (1999). Cognitive WorkAnalysis. Toward Safe, Productive, and Healthy Computer-Based

Work. New Jersey: Lawrence Erlbaum Associates.

92
GLOSSARY

Advanced control room - A control room that is primarily based on digital technology. It typically

provide the primary operator interaction with the plant via computer-based interfaces, such as video

display units. This is in contrast to "conventional" control rooms, which provide the primary operator

interaction with the plant via analog interfaces, such as gauges.

Applicant - An organization such as a nuclear plant vendor or utility that is applying to the U.S. Nuclear

Regulatory Commission for design certification or plant licensing.

Bias - Bias is an aspect of an evaluation methodology which systematically modifies performance or the

interpretation of performance.

Component - The meaning of the word component depends on its context. In context of the entire plant,

it is an individual piece of equipment such as a pump, valve, or vessel; usually part of a plant system. In a

human-system interface context, a component is one part of a larger unit, such as one meter in a control

board. In a maintenance context, a component is a subdivision of a unit of equipment that can be treated

as an object by the maintainer, but which can be further broken down into parts. A mounting board

together with its mounted parts is an example of a component.

Confound - The systematic coupling of one aspect of the test with another or with an extraneous variable.

Confounding makes important relationships ambiguous.

Construct validity - The extent to which a selected performance measure accurately represents the aspect

of performance to be measured.

Convergent validity - The degree to which consistent results are observed across different review,

evaluation, or measurement techniques.

Function - (1) A software supported capability provided to a user to aid in performing a task. (2) A

process or activity that is required to achieve a desired goal; see, e.g., "Safety function."

Function allocation -The process of assigning responsibility for function accomplishment to human or

machine resources, or to a combination of human and machine resources.

Functional requirements analysis - The examination of system goals to determine what functions are

needed to achieve them.

Functional requirements specification - A specification which identifies the functions and

characteristics that the human-system interface and its components accomplish or satisfy.

Human action - See "risk-important human actions."

Human-centered design goals - Human factors engineering design goals that address the cognitive and

physical support of personnel performance.

Human factors - A body of scientific facts about human characteristics. The term covers all biomedical,

psychological, and psycho-social considerations; it includes, but is not limited to, principles and

93
applications in the areas of human factors engineering, personnel selection, training, job performance

aids, and human performance evaluation (see "Human factors engineering").

Human factors engineering (HFE) 7The application of knowledge about human capabilities and

limitations to plant, system, and equipment design. HFE provides reasonable assurance that the desigi of

the plant, systems, equipment, human tasks, and the work environment are compatible with the sensory,

perceptual, cognitive, and physical attributes of the personnel who operate, maintain, and support the

plant (see "Human factors").;

Human-system interfaces (HSIs) - A human-system interface (HS1) is that part of the system through

which personnel interact to perform their functions and tasks. In this document, "system" refers to a

nuclear power plant. Major HSIs inciude alarms, information displays, controls, and proceduresi. Use of

HSIs can be influenced directly by factors such as, (1) the organization of HSIs into workstationis (e.g.,

consoles and panels); (2) the arrangement of workstations and supporting equipment into facilities such as

a main control room, remote shutdown station, local control station, technical support center, and

emergency operations facility; and (3) the environmental conditions inwhich the HSIs- are used, including

temperature, humidity,,ventilation, illumination, an"d noise. HSI use can also be affected indirectly by

other aspects of plant design and operation such as crew training, shift schedules, work practices, and

management and organizational factors.

Integrated system validation - Integrated System Validation is an evaluation using performance-based

tests to determine whether an integrated system design (i.e., hardware, software, and personnel elements)

meets performance requirements and acceptably supports safe operation of the plant.

Local control station (LCS) - An operator interface related to process control that is not located in the

main control room. This includes multifunction panels, as well as single-function LCSs such as controls

(e.g., valves, switches, and breakers) and displays (e.g., meters) that are operated or consulted during

normal, abnormal, or emergency operations.

Mockup - A static representation of an human-system interface (see "Simulator" and "Prototype").

Modification - Any type of change or modernization made to HSI components or plant systems that may

influence personnel performance.

Operating experience review - A review of relevant history from the plant's on-going collection,

analysis, and documentation of operating experiences and from interviews with plant staff.

Performance-based test - Tests that involve the measurement of behavior of personnel, the humansystem interface, or aspects of the plant to address design issues and design acceptability.

Performance shaping factors (PSFs) - Factors that influence human reliability through their effects on

performance. PSFs include factors such as environmental conditions, human-system interface design,

procedures, training, and supervision.

Personal safety - Relates to the prevention of individual accidents and injuries of the type regulated by

the Occupational Safety and Health Administration.

94
Plant - The operating unit of a nuclear power station including the nuclear steam-supply system, the

turbine, electrical generator, and all associated systems and components. In the case of a multi-unit plant,

the term plant refers to all systems and processes associated with the unit's ability to produce electrical

power, even though some systems or portions of systems may be shaiid 'with the other units.

Plant safety - Also called "safe operation of the plant." A general term used herein to denote the

technical safety objective as articulated by the International Nuclear Safety Advisory Group of the

International Atomic Energy Agency (IAEA) in the "Basic Safety Principles for Nuclear Power Plants"

(IAEA, 1988): "To prevent with high confidence accidents in nuclear plants; to verify that, for all

accidents taken into account in the design of the plant, even those of very low probability, radiological

consequences, if any, would be minor; and to provide reasonable assurance that the likelihood of severe

accidents with serious radiological consequences is extremely small."

Primary tasks - Those tasks performed by the operator to supervise the plant; i.e., monitoring, detection,

situation assessment, response planning, and response implementation.

Procedures - Written instructions providing guidance to plant personnel for operating and maintaining

the plant and for handling disturbances and emergency conditions.

Prototype - A dynamic representation of a human-system interface that is not linked to a process model

or simulator. A model of an interface which includes the functions and capabilities expected in the final

system, though not in a finished form. (See "Simulator" and "Mockup").

Requirement - The term "requirements" is used in two different ways in this document: (1) requirements

that are established as part of the design process; e.g., design requirements, functional requirements, task

requirements, etc.; and (2) regulatory requirements identified in 10 CFR. There are no regulatory

requirements established in this document.

Risk-important human actions - Actions that are performed by plant personnel to provide reasonable

assurance of plant safety. Actions may be made up of one or more tasks. There are both absolute and

relative criteria for defining risk important actions. -From an absolute standpoint; a risk important action

is any action whose successful performance is needed to provide reasonable assurance that predefined risk

criteria are met. From a relative standpoint, the risk important actions may be defined as those with the

greatest risk in comparison to all human actions. The identification can be done quantitatively from risk

analysis and qualitatively from various criteria such as task performance concerns based on the

consideration of performance shaping factors.

Safety - See "Personal safety," "Plant safety," "Safety evaluation," "Safety function," "Safety issue," and

"Safety-related."

Safety evaluation - The NRC process of reviewing an aspect of an NPP to verify that it meets

requirements and that it will perform as needed to provide reasonable assurance of plant safety.

Safety function - Safety functions are those functions that serve to verify higher-level objectives and are

often defined in terms of a boundary or entity that is important to plant integrity and the prevention of the

release of radioactive materials. A typical safety function is "reactivity control." A high-level objective,

such as preventing the release of radioactive material to the environment, is one that designers strive to

achieve through the design of the plant and that plant operators strive to achieve through proper operation

95
of the plant. The! function is often described without reference to specific plant systems and components

or the level of human and machine intervention that is needed to carry out this action. Functions are often

accomplished through some combination of lower-level functions' such as "reactor trip." The' process of

manipulating lower-level' functions to satisfy a higher-leVel function is defined here as a control function.

During function allocation the control function is assigned to human and machine elements.

Safety issue - An item identified during plant design, operation, or review that has the potential to affect

the safe operation of the plant.

Safety-related - A term applied to those NPP structures, systems, and components (SSCs) that prevent or

mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of

the public (see Appendix B to Part 50'of Title 10 ofthe U.S. Code of Federal Regulations). These are the

SSCs on which the design-basis analyses of the safety analysis report are performed. They also should be

part of a full quality assurance program in accordance with Appendix B of that document.

Secondary tasks - Those tasks that the operator perform when interfacing with the plant, but are not

directed to the primary task., Secondary tasks may include: navigating through and paging displays,

searching for data, choosing between multiple ways of accomplishing the same task, and making

decisions regarding how to configure the interface.

Simulator - A facility that physically represents the human-system interface configuration and that

dynamically represents the operating characteristics and responses of the plant in real time.

(see "Mockup" and "Prototype").

Situation awareness - The relationship between the operator's understanding of the plant's condition and

its actual condition at any given time. ;

State-of-the-art human factors principles - Those principles currently accepted by human factors

practitioners. "Current" is defined with reference to the time at which 'a program management or a

implementation plan is prepared. -"Accepted" is defined as a practice, method, or guide that

(1) is documented in the human factors literature within a standard or guidance document that has -

undergone a peer-review process or (2) can be justified through scientific research and/or industry

practices.

Style guide - A document that contains guidelines that have been tailored so they describe the

implementation of HFE guidance to a specific design, such as for a specific'plant control room.

Subsidiary tasks - Tasks used for'workload assessment.: These tasks are given to operators to perform.

while they are performing primary and secondary tasks. Their performance is theoretically tied to.

cognitive workload through measurement of spare mental capacity. Better performance on subsidiary

tasks reflects more spare mental capacity and,. therefore, lower primary/secondary task workload.

System - An integrated collection of plant components and control elements that operate alone or with

other plant systems to perform a function.

Task - A group of activities that have a common purpose, often occurring in close temporal proximity. = I. . .

96
Task Analysis - A method for describing what plant personnel must do to achieve the purposes or goal of

their tasks. The description can be in terms of cognitive activities, actions, and supporting equipment.

Testbed - The representation of the human-system interface and the process model used in testing.

Top-down review - A review approach that follows top - down design. In a top down design approach,

the design starts at the "top" with high-level plant mission goals that are broken down into functions that

are allocated to human and system resources and are further broken down into tasks performed to

accomplish function assignments. Tasks are arranged into meaningful jobs and the human-system

interface is designed to best support job task performance. The detailed design is the "bottom" of the topdown design process.

Trade-off evaluations - Comparisons performed between design options based on aspects of human

performance that are important to task performance.

Validation - (see "Integrated system validation")

Validity - The characteristics of the methods and tools used in the validation process.

Verification - The process by which the design is evaluated to determine whether it acceptably satisfies

personnel task needs and HFE design guidance.

Vigilance - The degree to which an operator is alert.

Workload - The physical and cognitive demands placed on plant personnel.

97
APPENDIX A

HFE DESIGN TEAM COMPOSITION

The term "HFE design team" is used in a generic sense to refer to the personnel who are responsible for

HFE within the scope of this report. There is no intent to prescribe any particular organizational structure

for an applicant, nor is it assumed that HFE is the responsibility of a single organization or that there is

necessarily an organizational unit called the HFE design team.

The following is a listing of the areas of expertise for the HFE design team. Associated with each area of

expertise is a listing of minimum qualifications and descriptions of typical contributions to the HFE

design and implementation process. The descriptions of typical contributions are provided as examples to

further describe the potential value of the various areas of expertise to the HFE design and

implementation process. This is not intended to define the total role of each area of expertise.

(1) Technical Project Management

* Minimum qualifications:

- Bachelor's degree

- 5 years of experience in nuclear power plant design or operations

- 3 years of management experience

* Typical contributions:

- develop and maintain the schedule for the HFE design process

- provide a central point of contact for management of the HFE design and

implementation process

(2) Systems Engineering

* Minimum qualifications:

- Bachelor of Science degree

- 4 years of cumulative experience in at least three of the following areas of

systems engineering; design, development, integration, operation, and test and

evaluation

* Typical contributions:

- provide knowledge of the purpose, operating characteristics, and technical

specifications of major plant systems

- provide input to HFE analyses, especially function analysis and task analysis

- participate in the development of procedures and scenarios-for task analysis,

validation, and other analyses

(3) Nuclealr Engineering

* Minimum qualifications:

- Bachelor of Science degree

- 4 years of nuclear design, development, test, or operations experience.

A-l
Typical contributions:

- provide knowledge of the processes involved in reactivity control and power

generation

- provide input to HFE analyses, especially function analysis and task analysis

- participate in the development of scenarios for task analysis, validation, and other

analyses

(4) Instrumentation and Control (I&C) Engineering

* Minimum qualifications:.

- Bachelor of Science degree

- 4 years of experience in design of hardware and software aspects of process

control systems

- experience in at least one of the following areas of I&C engineering: development, power plant operations, and test and evaluation -

- familiarity with the theory and practice of software quality assurance and control

* Typical contributions:

- provide detailed knowledge of the human-system interface (HSI) design,

including control and display hardware selection, design, functionality, and

installation

- provide knowledge of information display design, content, and functionality

- participate in the design, development, test, and evaluation of the HSIs

- participate in the development of scenarios for human reliability analysis (HRA),

validation, and other analyses involving failures of the HSI data processing

systems

- provide input to software quality assurance programs

(5) Architect Engineering

* Minimum qualifications:

- . Bachelor of Science degree

- 4 years of experience in design of power plant control rooms

* Typical contributions:

- provide knowledge of the overall structure of the plant including performance

requirements, design constraints, and design characteristics of the following:

containment building, control room, remote shutdown area, and local control

stations

- provide knowledge of the configuration of plant components within the plant

- provide input to plant analyses, especially function analysis, task analysis, and

the development of scenarios for task analysis and validation

A-2
(6) Human Factors Engineering

Minimum qualifications:

- Bachelor's'degree in Human Factors Engineering, Engineering Psychology, or

related science

- 4 years of cumulative experience related to the human factors aspects of

human-computer interfaces. Qualifying experience should include at least the

following activities within the context of large-scale human-machine systems

(e.g., process control): design, development, and test and evaluation

- 4 years of cumulative experience related to the human factors aspects of

workplace design. Qualifying experience should include at least two of the

following activities: design, development, and test and evaluation.

* Typical contributions:

- provide knowledge of human performance capabilities and limitations, applicable

human factors design and evaluation practices, and human factors principles,

guidelines, and standards

- develop and perform human factors analyses and participate in the resolution of

identified human factors problems

(7) Plant Operations

* Minimum qualifications:

- has or has held a senior reactor operator license

- 2 years of experience in relevant nuclear power plant operations

* Typical contributions:

- provide knowledge of operational activities including task characteristics, HSI

characteristics, environmental characteristics, and technical requirements related

to operational activities

- provide knowledge of operational activities in support of HSI activities such as

development of HSIs, procedures, and training programs

- participate in the development of scenarios for HRA evaluations, task analyses,

HSI tests and evaluations, validation, and other evaluations

(8) Computer System Engineering

* Minimum qualifications:

- Bachelor's degree in Electrical Engineering or Computer Science, or graduate

degree in other engineering discipline (e.g., Mechanical Engineering or Chemical

Engineering)

- 4 years of experience in the design of digital computer systems and real-time

systems applications

- familiarity with the theory and practice of software quality assurance and control

* Typical contributions:

- provide knowledge of data processing associated with displays and controls

A-3
- participate in the design and selection of computer-based equipment such as

controls and displays

- . participate in the development of scenarios for HRA, validation, and other

analyses involving failures of the HSI data processing systems

(9)- Plant Procedure Development

* Minimum qualifications:

- Bachelor's degree

- 4 years of experience in developing nucleartpower plant operating procedures

v Typical contributions: ..

- provide knowledge of operational tasks and procedure formats, especially as

presented in emergency procedure guidelines and operational procedures of

current a'nd priedecessor plants

T - participate in the development of scenarios for HRA evaluations, task analyses,

HSI tests and evaluations, validation, and other evaiuations

- provide input for the development of emergency operating procedures, procedure

aids, computer-based procedures, and training systems

(10) Personnel Training

* Minimum qualifications:.

I - Bachelor's degree

- 4 years of experience in the development of personnel training programs for

power plants

- experience in the application of systematic training development methods

* Typical contributions:

-- develop content and format of personnel training programs for licensed and nonlicensed plant personnel

- coordinate training issues arising from activities such as HRA, HSI design, and

procedure design with the training program -

- participate in the development of scenarios for HRA evaluations, task analyses,

HSI tests and evaluations, validation, and other evaluations

(11) Systems Safety Engineering

Minimum qualifications:

- Bachelor's degree in Science

- '4 years of experience in system safety engineering

: * Typical contributions:

- identify safety concerns and perform a system safety hazard analysis

- -provide results of system safety hazard analysis to probabilistic risk assessment/HRA and human factors analyses

A-4
(12) Maintainability/lnspectability Engineering

Minimum qualifications:

- Bachelor's degree in Science

- 4 years of cumulative experience in at least two of the' following areas of power

plant maintainability and inspectability engineering activity: design, development, integration, and test and evaluation

- ' experience in analyzing and resolving plant system and/or equipment-related

maintenance problems

Typical contributions:

- provide knowledge of maintenance, inspection, and surveillance activities

including task characteristics, HSI characteristics, human performance demands,

environmental characteristics, and technical requirements related to the conduct

of these activities

- support the design, development, and evaluation of the control room and other

HSIs throughout the plant to provide reasonable assurance that they can be

inspected and maintained to the specified level of reliability

- - provide input in the areas of maintainability and inspectability to the development of procedures and training

- participate in the development of scenarios for HSI evaluations including task

analyses, HSI design tests and evaluations, and validation

(13) Reliability/Availability Engineering

* Minimum qualifications:

- -Bachelor's degree

- 4 years of cumulative experience in at least two of the following areas of power

plant reliability engineering activity: design, development, integration, and test

and evaluation

- knowledge of computer-based, human-interface systems

* Typical contributions:

- provide knowledge of plant component and system reliability and availability

and assessment methodologies to the HSI development activities

- participate in human reliability analyses

- participate in the development of scenarios for HSI evaluations, especially

validation

- provide input to the design of HSI equipment to provide reasonable assurance

that it meets reliability goals during operation and maintains the specified level of

availability

The education and related professional experience of the HFE design team personnel should satisfy the

minimum qualification specified above for each of the areas of expertise. Qualifying professional

experience (e.g., design, development, analysis) for each area of expertise should be directly related to

those technologies and techniques that will be used in the HFE design and implementation process.

A-5
The professional experience is to be satisfied by the HFE design team as a collective whole. Therefore,

satisfaction of the professional experience requirements associated with a particular skill area may be

realized through the combination of the professional experience of two or more' members of the HFE

design team who each, individually, satisfy the other defined credentials:of the particular skill area but

who do not possess all of the specified professional experience.: It is recognized that one person may

possess multiple skills and that people may have additional responsibilities beyond the HFE design team.

Alternative personal credentials maybe accepted as the basis for satisfying the minimum personal

qualification specified above. A'cceptanlce'of sfuch alterfiative person'al crede'ntials should be evaluated on

a case-by-case basis and approved, documented, and retained in auditable piant files by the combined

operating license applicant. The following factors are examples of alteimative credentials that may be

considered acceptable:

* A professional Engineer's license in the skill area may be substituted for the Bachelor's

degree.

* Successful completion of all technical portions of an engineering, technology or related

science baccalaureate program may be substituted for the Bachelor's degree. The

successful completion will be determined by a transcript or other certification by an

accredited institution.. For example, completion of 80 semester credit hours may be

substituted for the baccalaureate requirement. The courses should be in appropriate

technical subjects relevant to the skill areas of the HFE design team for which the

individual will be reponsible.

* Related experience may substitute for education at the rate of 6 semester credit hours for

each year of experience up to a maximum of 60 credit hours.

* Where course work is related to job assignments, post-secondary education may be

substituted for experience at the rate of 2 years of education for 1 year of experience.

Total credit for post-secondary education should not exceed 2 years experience credit.

A-6
NRC FORM 335 U.S. NUCLEAR REGULATORY COMMISSION 1. REPORT NUMBER

(2-89) (AssIgned by NRC, Add Vol, Supp.. Rev..

NRCM 1102. BIBLIOGRAPHIC DATA SHEET and Addendum Numbers, If any.)

3201,320ILIGAHI2AA HE

(See stnfhwions on the reverse)

2 TITLE AND SUBTITLE NUREG-0711, Rev. 2

Human Factors Engineering Program Review Model

3. DATE REPORT PUBUSHED

MONTH | YEAR

February 2004

4. FIN OR GRANT NUMBER

Y6022

5. AUTHOR(S) 6. TYPE OF REPORT

John M. O'Hara, James C. Higgins%, J. J. Persensky^, Paul M. Lewis*, James P. Bongarra* Technical

7. PERIOD COVERED (Indushe Dates)

6.PERFORMINGORGANIZATION -NAMEANDADDRESS (ItNRC provideDvtsoOffice orRegion U.S. NudearRegufatoryCommission admaitirtgddressrcontractor,

provie name and maliig addres&)

*Energy Sciences and Technology Department **U.S. Nuclear Regulatory Commission

Brookhaven National Laboratory Washington DC 20555-0001

Upton NY 11973-5000

9. SPONSORING ORGANIZATION-NAMEANDADDRESS(NNRC, tye Sameasaboe' w'ontractor~pmdeNRCDhis5 OtffeorRegin U.S. NuLearRftoraloryComminsi

andmaDigaddress.)

Division of Systems Analysis and Regulatory Effectiveness

Office of Nuclear Regulatory Research

U.S. Nuclear Regulatory Commission

Washington DC 20555-0001

10. SUPPLEMENTARY NOTES

Paul M. Lewis, NRC Project Manager

11. ABSTRACT rz2w wonds or less)

This document Is used by the staff of the Nuclear Regulatory Commission to review the human factors engineering (HFE)

programs of applicants for construction permits, operating licenses, standard design certifications, combined operating

licenses, and for license amendments. The purpose of these reviews Is to verify that accepted HFE practices and guidelines

are Incorporated Into the applicant's HFE program. The review methodology provides a basis for performing reviews that

address the twelve elements of an HFE program: HFE Program Management, Operating Experience Review, Functional

Requirements Analysis and Function Allocation, Task Analysis, Staffing, Human Reliability Analysis, Human-System Interface

Design, Procedure Development, Training Program Development, Human Factors Verification and Validation, Design

Implementation, and Human Performance Monitoring. Each review element Is divided Into four sections: Background,

Objective, Applicant Submittals, and Review Criteria. References to sources of additional Information are also provided for

each element.

12. KEY WORDS/DESCRIPTORS (Lst wods or phrases that wI? assist researchers in bcxat the report) 13. AVAILABILITY STATEMENT

Human factors, human factors engineering, human factors evaluation, human factors review criteria, unlimited

nuclear safety, safety review, design certification, design review, design process, human-system 14. SECURITY CLASSIFICATION

interface, man-machine Interface, verification and validation (This Page)

unclassified

(This Repo")

unclassified

15. NUMBER OF PAGES

16. PRICE

NFIC FORM 335 (2 89) This form was electronically prodcd by Elite Federal Forrns, tnc.
Federal Recycling Program
UNITED STATES

NUCLEAR REGULATORY COMMISSION

WASHINGTON, DC 20555-0001

OFFICIAL BUSINESS