NUREG/CR-6637 

BNL-NUREG-52567

Human Systems 

Interface and 

Plant Modernization 

Process: Technical Basis and 

Human Factors 

Review Guidance 

Brookhaven National Laboratory 

U.S. Nuclear Regulatory Commission 

Office of Nuclear Regulatory Research Ai 

Washington, DC 20555-0001
AVAILABILITY NOTICE 

Availability of Reference Materials Cited in NRC Publications

NRC publications in the NUREG series, NRC regu

lations, and Title 10, Energy, of the Code of Federal 

Regulations, may be purchased from one of the fol

lowing sources: 

1. The Superintendent of Documents 

U.S. Government Printing Office 

PO. Box 37082 

Washington, DC 20402-9328 

<http://www.access.gpo.gov/su_docs> 

202-512-1800 

2. The National Technical Information Service 

Springfield, VA 22161-0002 

<http://www.ntis.gov> 

1-800-553-6847 or locally 703-605-6000 

The NUREG series comprises (1) brochures 

(NUREG/BR-XXXX), (2) proceedings of confer

ences (NUREG/CP-)0000, (3) reports resulting 

from international agreements (NUREG/IA-XXXX), 

(4) technical and administrative reports and books 

[(NUREG-)000) or (NUREG/CR-)000g], and (5) 

compilations of legal decisions and orders of the 

Commission and Atomic and Safety Licensing 

Boards and of Office Directors' decisions under 

Section 2.206 of NRC's regulations (NUREG

xxOON. 

A single copy of each NRC draft report for com

ment is available free, to the extent of supply, upon 

written request as follows: 

Address: Office of the Chief Information Officer 

Reproduction and Distribution 

Services Section 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

E-mail: <DISTRIBUTION@nrc.gov> 

Facsimile: 301-415-2289 

A portion of NRC regulatory and technical informa

tion is available at NRC's World Wide Web site:

<http://www.nrc.gov> 

After January 1, 2000, the public may electronically 

access NUREG-series publications and other NRC 

records in NRC's Agencywide Document Access 

and Management System (ADAMS), through the 

Public Electronic Reading Room (PERR), link 

<http://www.nrc.gov/NRC/ADAMS/index.html>. 

Publicly released documents include, to name a 

few, NUREG-series reports; Federal Register no

tices; applicant, licensee, and vendor documents 

and correspondence; NRC correspondence and 

internal memoranda; bulletins and information no

tices; inspection and investigation reports; licens

ee event reports; and Commission papers and 

their attachments. 

Documents available from public and special tech

nical libraries include all open literature items, such 

as books, journal articles, and transactions, Feder

al Register notices, Federal and State legislation, 

and congressional reports. Such documents as 

theses, dissertations, foreign reports and transla

tions, and non-NRC conference proceedings may 

be purchased from their sponsoring organization. 

Copies of industry codes and standards used in a 

substantive manner in the NRC regulatory process 

are maintained at the NRC Library, Two White Flint 

North, 11545 Rockville Pike, Rockville, MD 

20852-2738. These standards are available in the 

library for reference use by the public. Codes and 

standards are usually copyrighted and may be 

purchased from the originating organization or, if 

they are American National Standards, from

American National Standards Institute 

11 West 42nd Street 

New York, NY 10036-8002 

<http://www.ansi.org> 

212-642-4900

DISCLAIMER

This report was prepared as an account of work sponsored by 

an agency of the United States Government. Neither the United 

States Government nor any agency thereof, nor any of their em

ployees, makes any warranty, expressed or implied, or assumes

any legal liability or responsibility for any third party's use, or the 

results of such use, of any information, apparatus, product, or 

process disclosed in this report, or represents that its use by 

such third party would not infringe privately owned rights.

I I
NUREG/CR-6637 

BNL-NUREG-52567 

Human Systems 

Interface and 

Plant Modernization 

Process: Technical Basis and 

Human Factors 

Review Guidance 

Manuscript Completed: January 2000 

Date Published: March 2000 

Prepared by 

W. F. Stubler, J. M. O'Hara, J. C. Higgins/BNL 

J. Kramer/NRC 

Brookhaven National Laboratory 

Upton, NY 11973 

J. Kramer, NRC Project Manager 

Prepared for 

Division of Systems Analysis and Regulatory Efectiveness 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

NRC Job Code J6012
ABSTRACT 

Hybrid human-system interfaces (HSIsy result from the combination of new (e.g., digital) and traditional technologies. New 

demands may be imposed on personnel in operating and maintaining these systems. These demands may result from many 

factors including the characteristics of the new technologies, the characteristics of the mixture of new and traditional 

technologies, the process by which the hybrid HSI is developed and used, and the way in which personnel are prepared to 

use the hybrid HSI. The objective of this study was to develop human factors review guidance on the processes by which 

hybrid HSIs are developed, implemented, and integrated into plant operations. A characterization framework was 

developed for describing the key characteristics of hybrid HSIs that are important to human factors engineering (HFE) 

reviews. Then, the research studies, HFE processes, and guidance related to system development and modernization were 

reviewed. This information was used as the technical basis upon which we developed the design review guidelines. This 

guidance applies to general work that should be undertaken and factors that should be considered in designing and 

implementing hybrid HSIs, particularly in upgrading existing HSIs. The establishment and analysis of design 

requirements, interface design, and the evaluation of the final system are addressed. Issues for further research were 

identified for process consideration for which the technical basis was insufficient to support guidance development. 

iii NUREG/CR-6637
CONTENTS 

PaA.e 

A BSTRA CT ... .. ............................................................................... ix 

LIST O F FIG U RES ........ ......................................................................... x 

LIST OF TABLESUM M AR ......................................................................... xi 

EXECUTIVE SUM M ARY........................................................................... xii 

PREFACE D.. .. EN T ........................................................................... x ix 

ACKNOW LEDGM ENTS ....... .................................................................... xix 

ACRONYMS................................................................................. M 

PART 1: GUIDANCE DEVELOPMENT AND ITS TECHNICAL BASIS 

I IN TRO D U CTION ............................................................................ 1-! 

1.1 B ackground ....................................................................... - I 

1.2 HSI and Plant Modernization Processes ................................................. I - 1 

1.2.1 Standard Review Plan (NUREG-0800) ........................................... 1 -2 

1.2.2 Human Factors Engineering Program Review Model (NUREG-071 1) .................. - 2 

1.2.3 Human-System Interface Design Review Guideline (NUREG-0700) ................... - 8 

1.2.4 Nuclear Regulatory Commission Inspection Manual .............................. - 9 

1.2.5 Sum m ary .................... ............................................ 1- 11 

1.3 Earlier Research on HSI Upgrades .................................................... I - 11 

1.4 Organization of the Report .......................................................... 1 - Ii 

2 O BJECTIV E ................................................................................. 2 - 1 

3 M ETHODOLOGY ............................................................................ 3 -1 

3.1 O verview ......................................................................... 3 -I 

3.2 Characterization .................................................................... 3 -2 

3.3 Development of the Technical Basis .................................................... 3 -2 

3.4 Development and Documentation of Guidance ............................................ 3 -4 

3.5 Identification of the Issues ........................................................... 3 -4 

3.6 Peer Review ....................................................................... 3 -4 

4 TECHNICAL BASIS DEVELOPMENT: HUMAN PERFORMANCE 

IN COMPLEX HUMAN-MACHINE SYSTEMS ................................................ 4-1 

4.1 The Effects of HSI Design on Personnel Performance ...................................... 4 - 1 

4.2 Primary and Secondary Tasks ......................................................... 4-2 

4.3 Basic Properties of Human Information Processing and Skilled Performance .................... 4 - 3 

v NUREG/CR-6637
CONTENTS (Continued) 

Pag'e 

4.3.1 M emory and Attention ........................................................ 4 - 3 

4.3.2 Cognitive Skills for Overcoming Limitations of Memory and Attention ................. 4 -4 

4.3.3 Cognitive Skills for Problem Solving ................. ... ....................... 4 - 6 

4.3.4 H um an Error ............................................................... 4 -6 

4.4 Generic Cognitive Tasks ............................................................ 4- 7 

4.4.1 Individual Performance ........................................ .............. 4 - 8 

4.4.2 Crew Performance and Team Skills ............................................ 4 - 13 

5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS CONSIDERATIONS FOR UPGRADES ........ 5-1 

5.1 Planning Phase .................................................................... 5 - 1 

5.1.1 General Considerations ...................................................... 5 -1 

5.1.2 HFE Program M anagement ................................................... 5 -3 

5.2 A nalysis Phase ..................................................................... 5 - 6 

5.2.1 Functional Requirements Analysis and Function Allocation .......................... 5 - 6 

5.2.2 Task Analysis .... ..................................................... 5 - 7 

5.2.3 Staffing A nalysis ........................................................... 5 -15 

5.2.4 Human Reliability and Design Basis Considerations ............................... 5 - 16 

5.2.5 Summary of Analysis Phase .................................................. 5 - 17 

5.3 Interface Design Phase ............................................................. 5 - 18 

5.3.1 H SI D esign ............................................................... 5 - 18 

5.3.2 Procedure Development ..................................................... 5 - 24 

5.3.3 Training Development ....................................................... 5 - 25 

5.3.3.1 Determining Relevant Knowledge and Skills .............................. 5 - 26 

5.3.3.2 Approaches to Training Cognitive Skills .................................. 5 - 29 

5.3.3.3 Training for Conceptual Knowledge ..................................... 5 - 32 

5.3.3.4 Training for Skill Automaticity ......................................... 5 - 33 

5.3.3.5 Team Training ...................................................... 5 - 34 

5.3.3.6 Evaluation of Training Systems ......................................... 5 - 37 

5.3.4 Summary of Interface Design Phase ............................................ 5 - 38 

5.4 Verification and Validation Phase ..................................................... 5 - 38 

5.4.1 HSI Task Support Verification ................................................ 5 - 39 

5.4.2 HFE Design Verification ..................................................... 5 - 39 

5.4.3 Integrated System Validation ................................................. 5 -40 

5.4.4 Verification and Validation Considerations in Installing Upgrades .................... 5 - 43 

NUREG/CR-6637 vi
CONTENTS (Continued) 

Page 

5.5 Organizational Considerations for Designing and Implementing HSI Upgrades ................. 5 -45 

5.6 Unresolved Issues For Further Research ................................................ 5 - 50 

6 DEVELOPMENT OF GUIDANCE ............................................................... 6 - 1 

7 SU M M A RY ................................................................................. 7 - 1 

8 REFEREN CES .............................................................................. 8 - 1 

PART 2: REVIEW GUIDELINES: HYBRID HSI DESIGN PROCESS 

9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI ....................................... 9-1 

9.1 G eneral Guidance .................................................................. 9 -1 

9.2 HFE Program M anagement .......................................................... 9 -2 

9.3 Operating Experience Review ......................................................... 9 -4 

9.4 Functional Requirements Analysis and Function Allocation ................................. 9 -4 

9.5 Task A nalysis ..................................................................... 9 - 5 

9.6 Staffing .......................................................................... 9 -6 

9.7 Human Reliability Analysis ........................................................... 9 - 7 

9.8 Human-System Interface Design ....................................................... 9 - 8 

9.8.1 HSI Design M ethodology ..................................................... 9 - 8 

9.8.1.1 Development of HSI Design Guidance .................................... 9 - 8 

9.8.1.2 HSI Detailed Design and Integration ..................................... 9 - 8 

9.8.2 HSI Design Process Tests and Evaluations ........................................ 9 - 9 

9.9 Procedure Developm ent ............................................................ 9 - 11 

9.10 Training Program Development ....................................................... 9 - 12 

9.11 Human Factors Verification and Validation ............................................. 9 - 14 

9.11.1 G eneral Criteria ............................................................ 9 -14 

9.11.2 HSI Task Support Verification ................................................ 9- 15 

9.11.3 HFE Design Verification ...................................................... 9 - 16 

9.11.4 Integrated System Validation ................................................. 9 - 16 

9.11.4.1 Test Objectives ..................................................... 9- 17 

9.11.4.2 Validation Testbeds ................................................. 9 - 17 

9.11.4.2.1 M ain Control Room ........................................ 9- 17 

9.11.4.2.2 Representation of Facilities Remote from the Main Control Room .... 9 - 18

vii NUREG/CR-6637
CONTENTS (Continued) 

Page 

9.11.4.3 Plant Personnel ..................................................... 9 - 18 

9.11.4.4 Operational Conditions .............................................. 9 - 19 

Appendix A: High-Level Design Review Principles From NUREG-0700, Rev. 1 ............................. A - I 

G LO SSA R Y ................................................................................... G - I

NUREG/CR-6637 viii

I I
LIST OF FIGURES 

Pace 

1.1 Elements of Human Factors Engineering Program Review Model (NUREG-0711) ....................... 1-3 

3.1 Major Steps in the Development of NUREG-0700 Guidance ......................................... 3-1 

3.2 Technical Basis and Guidance Development Process ................................................ 3-3 

4.1 Causal Links Between Human and Plant Characteristics That Affect System Performance .................. 4-2

ix NUREG/CR-6637
LIST OF TABLES 

Pag~e 

4.1 Types of Slips That May Affect Soft Control Usage ................................... 4-7 

5.1 Primary USQ Analysis Questions from EPRI TR-102348 ............................... 5-2 

5.2 Comparison of Three Categories of Cognitive Task Analysis (CTA) ....................... 5-12 

5.3 Some Knowledge and Skill Dimensions for Assessing Training Needs .................... 5-28 

5.4 Principles for User Acceptance of Automation for Landing Systems ....................... 5-48 

9.1 Some Knowledge and Skill Dimensions for Training-Needs Assessment ................... 9-13

xi NUREG/CR-6637
EXECUTIVE SUMMARY 

The Human-System Interface Design Review Guideline, NUREG-0700, Rev. 1 (O'Hara, Brown, Stubler, Wachtel, and 

Persensky, 1996) was developed to provide human factors engineering (HFE) guidance to the U.S. Nuclear Regulatory 

Commission (NRC). The NRC staff uses NUREG-0700 for: (1) reviewing the human-system interface (HSI) design 

submittals prepared by licensees or applicants for a license or design certification of a commercial nuclear power plant 

(NPP), and (2) performing HSI reviews undertaken as part of an inspection or other type of regulatory review involving HSI 

design or incidents involving human performance. It describes those aspects of the HSI design review process that are 

important to identifying and resolving human engineering discrepancies that could adversely affect a plant's safety. 

NUREG-0700 also has detailed HFE guidelines for assessing the implementation of HSI designs. In developing NUREG

0700, Rev. 1, several "gaps" were identified. The gaps were topics for which there was an insufficient technical basis upon 

which to formulate guidance. One gap is the integration of advanced HSI technology into conventional NPPs. 

The NRC is currently sponsoring research at Brookhaven National Laboratory (BNL) to address this gap and to (1) better 

define the effects of changes to plant HSIs, brought about by the application of digital technology, on personnel performance 

and plant safety, and (2) develop HFE guidance to support safety reviews should it be necessary to examine plant 

modifications involving a safety-significant aspect of HSIs. The research led to the identification of several human 

performance topics (O'Hara, Stubler, and Higgins, 1996). The topics then were evaluated for their potential safety significance (Stubler, Higgins, and O'Hara, 1996). One topic found to be potentially significant to safety and so selected for 

the development of HFE guidance was Design Analysis, Evaluation, and Implementation of Hybrid HSIs. 

This topic addresses the need to consider human factors issues during the analysis, design, and evaluation of upgrades. Of 

particular concern are changes in roles and functions of plant personnel brought about by changes in the HSI. Another 

aspect of this topic is the need to consider human factors when implementing technology changes (upgrades), including the 

installation of the changes and their incorporation into a plant's operating practices. Important factors included the effects 

on personnel of temporary and changing HSI configurations, personnel training, and personnel acceptance of HSI changes. 

Thus, this topic encompasses the life cycle of an HSI upgrade from its initial planning through design, evaluation, and 

installation. The topic title was renamed HSI and Plant Modernization Process to more clearly convey this scope. 

The objective of this study was to formulate HFE review guidance for NPP upgrades to plant systems and the HSI, based on 

a technically valid methodology for developing guidelines. The term upgrade is used to include any type of change, 

modification, or retrofit made to HSI components or plant systems that may influence personnel performance. Several tasks 

were performed, including the following: 

A technical basis was established using human performance research and analyses of upgrades, 

HFE review guidelines were developed in a format consistent with NRC guidance, and 

Remaining issues were identified for which research is insufficient to support the development of NRC review 

guidance. 

The status of each is briefly discussed below. 

Technical Basis Development 

The effects of upgrades on personnel performance were addressed by examining basic HFE literature, literature on complex 

human-machine systems, and industry experience gained from site visits, interviews, and industrial literature. The 

resulting technical basis described human performance in complex human-machine systems in terms of skilled task 

performance and the basic properties of human information processing, such as memory and attention, supporting it. The 

types of knowledge and skills that must be adapted to a new work design after an upgrade was addressed.

NUREG/CR-6637 °.°i
EXECUTIVE SUMMARY

These human performance considerations were then addressed within the context of the NRC's design process review 

guidance. The Standard Review Plan, NUREG-0800 (NRC, 1996), specifies the Human Factors Engineering Program 

Review Model, NUREG-0711 (O'Hara et al., 1994), as the guidance for HFE reviews of design processes. NUREG-0711 

provides criteria for evaluating the incorporation of HFE principles and practices in the design of HSIs in NPPs, and for 

evaluating the implementation of the final design. It was originally developed to support the review of NPP designs for 

certification under 10 CFR Part 52. However, it is being revised and, in accordance with NUREG-0800, will provide the 

criteria for all HFE-related staff reviews, i.e., applicable to Part 50 as well as Part 52 applicants. 

A central tenet ofNUREG-071 1 is that the HFE aspects of the plant should be developed, designed, and evaluated on the 

basis of a structured system analysis using accepted HFE principles. NUREG-07 11 reflects a top-down approach for 

conducting an NRC safety evaluation so that the significance of individual topics may be seen in relationship to the high

level goal of plant safety. Top-down refers to an approach starting at the "top" with the plant's high-level mission goals 

and breaking them down into the functions necessary to achieve the goals. Functions are allocated to human and system 

resources and are split into tasks. Personnel tasks are analyzed for specifying the alarms, information, and controls that will 

be required to allow them to accomplish assigned functions. Tasks are arranged into meaningful jobs assigned to individual 

operators and the HSI is designed to best support job performance. The detailed design (of the HSI, procedures, and 

training) is the "bottom" of the top-down process. The HFE safety evaluation should include HFE aspects of normal and 

emergency operations, tests, and maintenance. 

NUREG-0711 has ten design process review elements: 

* HFE Program Management 

Operating Experience Review 

Functional Requirements Analysis and Function Allocation 

* Task Analysis 

• Staffing 

Human Reliability Analysis 

Human-System Interface Design 

Procedure Development 

Training Program Development 

Human Factors Verification and Validation. 

Each element is divided into four sections: Background, Objective, Applicant Submittals, and Review Criteria. The 

review criteria are oriented to new plant designs. Thus guidance to address the unique considerations associated with 

upgrades had to be developed. 

NUREG/CR-6637 xiv

I I
EXECUTIVE SUMMARY

HFE Review Guidelines 

Guidance for reviewing the design process aspects of upgrades was developed. The guidance was organized according to 

the ten review elements of NUREG-07 11. In addition, a general guidance section was developed. 

Within each element, guidance was further organized into four sections. The first describes the conditions under which the 

particular NUREG-07 11 element is relevant to the review of upgrades. The second category includes modified guidance 

from NUREG-07 11 that focuses on characteristics and considerations for upgrades. The guidance in the third category is 

specifically relevant to upgrades, but does not appear in NUREG-07 11. The fourth category has considerations that have 

potential applications beyond upgrades, and are possible additions to the more general guidance of NUREG-0711. 

Upgrade Issues Requiring Additional Research 

Several human performance issues associated with upgrades were identified. They represent topics for which research is 

necessary before more guidance can be developed: 

* The Effects of HSI Inconsistency on Alternating Use of HSI Components 

* The Effects of HSI Design on Crew Coordination and Cooperation 

* The Role of Training in HSI Skills 

* The Effects of the Installation Process for HSI Upgrades on Personnel Performance 

* Personnel Acceptance of Upgrades. 

This guidance will be integrated into existing NRC review guidance documents and will give the NRC staff the technical 

basis to help ensure that modifications of HSI designs do not compromise safety. Thus, the results of this project are 

expected to contribute to satisfying the NRC's goals of(1) maintaining safety, (2) increasing public confidence, (3) 

increasing regulatory efficiency and effectiveness, and (4) reducing unnecessary burden on personnel.

XV NUREG/CR-6637
PREFACE 

This report has been prepared by Brookhaven National Laboratory for the Division of Systems Technology of the U.S. 

Nuclear Regulatory Commission's (NRC's) Office of Nuclear Regulatory Research. It is submitted as part of the 

requirements of the project Human Factors Topics Associated with Hybrid Human System Interfaces (NRC FIN J6012), 

specifically, as part of Task 3, "Develop Review Guidance." The NRC Project Manager is Joel Kramer and the BNL 

Principal Investigator is John O'Hara.

xvii NUREG/CR-6637
ACKNOWLEDGMENTS 

The authors wish to express their sincere gratitude to our colleagues Lew Hanes; Mike Fineberg and his review team at the 

Crew Systems Ergonomics Information Analysis Center (CSERIAC); and Bill Ruland, Greg Galletti, Jim Bongarra, Clare 

Goodman, Jim Stewart, and Rich Correia of the U.S. NRC for their review of the reports. These reviewers provided 

insightful comments and perspectives on the issues addressed in the report and their knowledge and understanding 

significantly contributed to the study. 

We also wish to thank Barbara Roland, Mary Anne Corwin, and Avril Woodhead for their preparation and careful technical 

editing of the report.

xix NUREG/CR-6637
ACRONYMS 

AECB Atomic Energy Control Board (Canadian) 

BNL Brookhaven National Laboratory 

CALS Continuous Acquisition and Life-Cycle Support 

CSF Critical safety function 

CBP Computer-based procedure 

CDM Critical Decision Method 

COSS Computerized operator support system 

CR Control room 

CRT Cathode ray tube 

CTA Cognitive task analysis 

DCS Distributed control system 

DOE Department of Energy (U.S.) 

EOP Emergency operating procedure 

EPRI Electric Power Research Institute 

GOMS Goals, operations, methods, and selection rules 

HED Human engineering discrepancy 

HFE Human factors engineering 

HRA Human reliability analysis 

HSI Human-system interface 

I&C Instrumentation and control 

IM Interface management 

ISLOCA Interfacing system loss of coolant accident 

KSA Knowledge, skills, and abilities 

NGOMSL Natural-language goals, operations, methods, and selection rules language 

NPP Nuclear power plant 

NRC Nuclear Regulatory Commission (U.S.) 

OFM Operator Function Modeling 

OER Operating experience review 

PRA Probabilitistic risk assessment 

PWR Pressurized Water Reactor (NPP) 

SAR Safety Analysis Report 

SAT Systems approach to training 

SER Safety Evaluation Report 

SME Subject-matter expert 

SRG System Response Generator (system) 

SRP Standard Review Plan 

SSC Structures, systems, and components 

TEM Text-editing model 

TMI Three Mile Island (nuclear power station) 

USQ Unreviewed safety question 

VDU Video display unit 

V&V Verification and validation

NUREG/CR-6637 xxi
PART 1 

Guidance Development and Its Technical Basis

NUREG/CR-6637
1 INTRODUCTION 

1.1 Background 

The Human-System Interface Design Review Guideline (NUREG-0700, Rev. 1) (O'Hara, Brown, Stubler, Wachtel, and 

Persensky, 1996) was developed to provide human factors engineering (HFE) guidance to the U.S. Nuclear Regulatory 

Commission (NRC). The NRC staff uses NUREG-0700 for: (1) reviewing the human-system interface (HSI) design 

submittals prepared by licensees or applicants for a license or design certification of a commercial nuclear power plant 

(NPP), and (2) performing HSI reviews undertaken as part of an inspection or other type of regulatory review involving HSI 

design or incidents involving human performance. It describes those aspects of the HSI design review process that are 

important to identifying and resolving human engineering discrepancies that could adversely affect a plant's safety. 

NUREG-0700 also has detailed HFE guidelines for assessing the implementation of HSI designs. 

In developing NUREG-0700, Rev. 1, several topics were identified as "gaps" because there was an insufficient technical 

basis upon which to formulate guidance. One such topic is integrating advanced HSI technology into conventional NPPs. 

The NRC is currently sponsoring research at Brookhaven National Laboratory (BNL) to (1) better define the effects of 

changes to plant HSIs, brought about by the application of digital technology, on personnel performance and plant safety; 

and (2) develop HFE guidance to support safety reviews should it be necessary to examine plant modifications involving a 

safety-significant aspect of HSIs. This guidance will be integrated into NUREG-0700 and will give the NRC staff the 

technical basis to help ensure that such modifications of HSI designs do not compromise safety. 

The results of this project are expected to contribute to satisfying the NRC's goals of(1) maintaining safety, (2) increasing 

public confidence, (3) increasing regulatory efficiency and effectiveness, and (4) reducing unnecessary burden on personnel. 

Based upon the literature, interviews, and site visits, changes in HSI technology and their potential effects on human 

performance were identified (O'Hara, Stubler, and Higgins, 1996). The topics then were evaluated for their potential safety 

significance (Stubler, Higgins, and O'Hara, 1996). One topic found to be potentially significant to safety and so selected for 

the development of HFE guidance was Design Analysis, Evaluation, and Implementation of Hybrid HSIs. 

This topic resulted from the combination of two separate but related topics: (1) Design Analysis and Evaluation of Hybrid 

HSIs, and (2) Upgrade Implementation of Hybrid HSIs. The first topic addressed the need to consider human factors issues 

during analysis and evaluations conducted when designing upgrades. The objective was to ensure that human performance 

considerations are adequately addressed during the process. Of particular concern were changes in people's roles and 

functions brought about by changes in the HSI. The second topic addressed the need for considering human factors when 

implementing technology changes (upgrades), including their installation and incorporation into a plant's operating 

practices. Important factors included the effects upon personnel of temporary and changing HSI configurations, personnel 

training, and personnel acceptance of HSI changes. Thus, combining these two topics resulted in one larger topic that 

encompasses the life cycle of an HSI upgrade from its initial planning through design, evaluation, and installation. The 

topic title was renamed HSI and Plant Modernization Process to more clearly convey this scope. 

In this report, the term upgrade is used to describe the types of design changes addressed by this review. It is used 

generically to include any type of change, modification, or retrofit made to HSI components or plant systems that may 

influence personnel performance. This usage is consistent with that in industry, as in the Guideline on Licensing Digital 

Upgrades (EPRI TR- 102348) from the Electric Power Research Institute (EPRI, 1993). 

1.2 HSI and Plant Modernization Processes 

The topic of this report is oriented toward the design process for upgrading the HSI or plant equipment rather than toward 

the design details of any one specific technology, such as soft controls or computer-based procedures. Therefore, it is

1-1 NUREG/CR-6637
1 INTR•ODUCTION

important to understand how the NRC staff generally reviews factors in the HFE design process. The guidance produced in 

this study is fully consistent with, and integrated with, the review methodologies already developed. The staff's review 

methods and criteria are primarily described in three main documents, NUREGs 0800, 0711, and 0700. In addition, several 

NRC inspection manuals are relevant. To provide the reader with a better understanding of the guidance developed in this 

document, each of these documents is briefly described below. 

1.2.1 Standard Review Plan (NUREG-0800) 

The Standard Review Plan, NUREG-0800 (NRC, 1996), was originally published in 1981 and was updated in 1996 to 

reflect the staff's and industry's experience with safety reviews. Importantly, one reason for the change was "...to reflect the 

experience of the safety reviews conducted on design certification applications for evolutionary nuclear power plants." The 

revised Standard Review Plan (SRP) is applicable to both 10 CFR Part 50 and 10 CFR Part 52 applicants, thus bringing the 

review of existing license holders and applicants for advanced plants under a common review methodology. 

Chapter 18, Human Factors Engineering, discusses the review of the HFE aspects of NPPs. The purpose of the HFE review 

is "...to improve safety by verifying that accepted human factors engineering practices and guidelines are incorporated into 

the program design." The review covers both the design process and its product. 

The acceptance criteria are divided into ten broad areas of review: HFE Program Management; Operating Experience Review; Functional Requirements Analysis and Allocation Analysis; Task Analysis; Staffing; Human Reliability Analysis; Humanr-System Interface Design; Procedure Development; Training Program Development; and, Human Factors 

Verification and Validation. The SRP provides general criteria for each of these areas of review, and refers to NUREG-0711 

and NUREG-0700, Rev. I (discussed in the next two sections) for more specific details. 

As the SRP in general has broad applicability, the methodology is tailored to the needs of the specific review. "For any given application, the staff reviewers may select and emphasize particular aspects of each SRP section as is appropriate for 

the application.., the staff may not carry out in detail all of the review steps listed in each SRP section in the review of every 

application" (p. 3). In Chapter 18, tailoring of the review is identified as well. The SRP notes that: 

While this review process defines 10 areas of review, not all may be applicable to reviewing an applicant's human factors 

engineering program. Judgement regarding areas of review to be given attention for an applicant's submittal should be 

based on evaluation of the information provided by the applicant, the similarity of the associated HFE issues to those 

recently reviewed for other plants, and the determination of whether items of special or unique safety significance are 

involved. Also, the relevance of each area of review and the appropriate level of detail of evidence should be considered 

with respect to such factors as the purpose of the review, the nature of the HFE concern, the status of the applicant's design 

process, the scope of the design, and the goal of the review (e.g., design certification, licensing). (p. 18.0.2) 

The predominant technical basis of the SRP is the Human Factors Engineering Program Review Model (NUREG-07 11). 

This document is discussed in the next section. 

1.2.2 Human Factors Engineering Program Review Model (NUREG-0711) 

The Human Factors Engineering Program Review Model, NUREG-0711 (O'Hara et al., 1994) provides criteria for 

evaluating the incorporation of HFE principles and practices in the design of HSIs in NPPs, and for evaluating the 

implementation of the final design. It was originally developed to support the review of NPP designs for certification under 

10 CFR Part 52. However, it is being revised and, in accordance with the SRP, will provide the criteria for all HFE-related 

staff reviews, i.e., applicable to Part 50 as well as Part 52 applicants. 

NUREG/CR-6637 1 -2

I I
1 INTRODUCTION

A central tenet of NUREG-07 11 is that the HFE aspects of the plant should be developed, designed, and evaluated on the 

basis of a structured system analysis using accepted HFE principles. NUREG-0711 reflects a top-down approach for 

conducting an NRC safety evaluation so that the significance of individual topics may be seen in relationship to the high

level goal of plant safety. Top-down refers to an approach starting at the "top" with the plant's high-level mission goals 

and breaking them down into the functions necessary to achieve the goals. Functions are allocated to human and system 

resources and are split into tasks. Operator's tasks are analyzed for specifying the alarms, information, and controls that 

will be required to allow the operator to accomplish assigned functions. Tasks are arranged into meaningful jobs assigned 

to individual operators and the HSI is designed to best support job performance. The detailed design (of the HSI, 

procedures, and training) is the "bottom" of the top-down process. The HFE safety evaluation should be broad-based and 

include HFE aspects of normal and emergency operations, tests, and maintenance. 

NUREG-071 1 has ten elements organized by the stages of planning, analysis, design, and verification and validation 

(Figure 1.1). Each element is divided into four sections: Background, Objective, Applicant Submittals, and Review 

Criteria.

Figure 1.1 Elements of Human Factors Engineering Program Review Model (NUREG-0711)

1-3 NUREG/CR-6637
1 INTRODUCTION

Background- A brief explanation is given of the rationale and purpose of each element. 

Objective - The review objective(s) of the element is defined. 

Applicant Submittals - Materials to be provided for the NRC's review are listed. Generally, three reports are 

identified: implementation plan (the applicant's proposed methodology for meeting the acceptance criteria of the 

element), analysis-results report (the results of the applicant's efforts on a NUREG-071 1 element with respect to 

the review criteria), and design-team review report (an independent evaluation of the activities addressed for the 

element by the design team). In addition to reports, the reviewer may require samples of work products for earlier 

elements, and implemented designs for later elements, such as verification and validation (V&V). 

Review Criteria - This section contains the acceptance criteria for design process products and for the final design 

review. Not all existing NRC detailed criteria for the final design are duplicated. For example, NUREG-0700 

contains HFE guidance for detailed reviews of control room design. NUREG-0700 is only referred to in the 

applicable NUREG-07 11 elements. 

NUREG-0711 states that the applicant should undertake the types of activities described for each element using accepted 

HFE practices, as specified by applicable regulatory documents and HFE codes, standards, and guidelines. Each of the 

NUREG-0711 elements lists the documents that may be used. 

A brief overview of the focus of each element follows: 

Element 1 - HFE Program Management 

The overall purpose of the HFE program review is to ensure that 

The applicant has integrated HFE into the plant's development, design, and evaluation. 

The applicant has provided HFE products (e.g., HSIs, procedures, and training) that make it possible to perform 

operation, maintenance, test, inspection, and surveillance tasks safely, efficiently, and reliably. 

The HFE program and its products reflect "state-of-the-art human factors principles" [10 CFR 50.34(f)(2)(iii)) as 

required by 10 CFR 52.47(a)(1)(ii)] and satisfy all specific regulatory requirements, as stated in 10 CFR. 

The objective of this review topic is to ensure that the applicant has an HFE design team with the responsibility, authority, 

placement within the organization, and composition to ensure that the design commitment to HFE is met. Also, the team 

should be guided by a plan to ensure that the HFE program is properly developed, executed, overseen, and documented. 

This plan should describe the technical program elements ensuring that all aspects of HSI are developed, designed, and 

evaluated on the basis of a structured top-down systems-analysis using accepted HFE principles. 

Element 2 - Operating Experience Review 

The main purpose of conducting an operating experience review (OER) is to identify HFE-related safety issues. The OER 

should provide information on the past performance of fully-integrated predecessor systems. This approach is analogous to 

full-mission validation tests, which provide information about the achievement of HFE design goals supporting safe plant 

NUREG/CR-6637 1 -4

I I
1 INTRODUCTION

operation for the integrated system under review. The issues and lessons learned from previous operating experience 

provide a basis for improving the plant design in a timely way; i.e., at the beginning of the design process. 

The objective of reviewing this topic is to assure that the applicant has identified and analyzed any HFE-related problems 

and issues in previous designs that are similar to the current design under review. In this way, negative features associated 

with predecessor designs may be avoided in the current one while retaining positive features. The OER should address the 

predecessor systems upon which the design is based, selected technological approaches (e.g., if touch-screen interfaces are 

planned, the HFE issues associated with using them should be reviewed), and the plant's HFE issues (e.g., generic safety 

issues defined by the NRC). 

Element 3 - Functional Requirements Analysis and Allocation 

The purpose of this element of the review is to ensure that the applicant has defined the plant's safety functional 

requirements and that the function allocations take advantage of human strengths and avoid allocating functions that would 

be negatively affected by human limitations. The operator's role is examined in two steps: functional requirements 

analysis, and function allocation (assignment of levels of automation). 

Functional requirements analysis is the identification of those functions which must be performed to satisfy the plant's 

safety objectives, i.e., to prevent or mitigate the consequences of postulated accidents that could cause undue risk to the 

health and safety of the public. This analysis determines the objectives; performance requirements, and constraints of the 

design, and sets a framework for understanding the role of controllers (whether personnel or system) in regulating plant 

processes. 

Function allocation is the analysis of the requirements for plant control and the assignment of control functions to (1) 

personnel (e.g., manual control), (2) system elements (e.g., automatic control and passive, self-controlling phenomena), and 

(3) combinations of the two (e.g., shared control and automatic systems with manual backup). It seeks to enhance overall 

plant safety and reliability by exploiting the strengths of personnel and system elements, including improvements that can 

be achieved through assigning control to these elements with overlapping and redundant responsibilities. Function 

allocation should be based upon HFE principles using a structured and well-documented methodology that provides 

personnel with logical, coherent, and meaningful tasks. 

Element 4 - Task Analysis 

Task analysis is the evaluation of the demands of performance on personnel to identify the task requirements for 

accomplishing the functions allocated to them. It defines the HSI requirements for supporting their accomplishment of 

tasks. (It also identifies, by exclusion, what is not needed in the HSI). People perform tasks to meet their functional 

responsibilities. Although there is no precise definition of a task in the abstract, a task is a group of related activities that 

have a common objective or goal. 

The objective of this element is to assure that the applicant's task analysis identifies the requirements of the tasks that 

personnel must perform. The task analysis should (1) provide one of the bases for making decisions on design, (2) assure 

that human performance requirements do not exceed human capabilities, (3) be used as basic input for developing 

procedures, (4) be used as basic information for developing the staffing, training, and communication requirements of the 

plant, and (5) form the basis for specifying the requirements for the displays, data processing, and controls needed to carry 

out tasks.

1-5 NUREG/CR-6637
1 INTRODUCTION

Element 5 - Staffina 

Plant staffing is an important consideration throughout the design process. Initial staffing levels may be established as 

design goals early in the process based on experience with previous plants, customers' requirements, initial analyses, and 

government regulations. However, the acceptability of the staffing goals and assumptions should be examined as the design 

of the plant proceeds. The objective of the staffing review is to ensure that the applicant has systematically analyzed the 

requirements for the number and qualifications of personnel, including a thorough understanding of the tasks' requirements 

and regulatory requirements. 

Element 6 - Human Reliability Analysis 

Human Reliability Analysis (HRA) seeks to evaluate the potential for, and mechanisms of, human error that may affect 

plant safety. Thus, it is an essential element in achieving the HFE design goal of providing operator interfaces that will 

minimize operator's errors, allow their detection, and provide recovery capability. HRA has quantitative and qualitative 

aspects, both useful for HFE purposes. The HRA should be conducted as an integrated activity to support both the HFE and 

HSI design and probabilistic risk assessment (PRA). The PRA and HRA should be first performed early in the design 

process to provide insights and guidance both for systems design and for HFE purposes. The quality of the HRA depends, 

in large part, on the analyst's understanding of personnel tasks, the information related to the tasks, and the factors which 

influence human performance. Accordingly, the HRA might be carried out interactively as the design progresses. 

By developing an understanding of the causes and modes of human error, the HRA can provide valuable insights into the 

desirable characteristics of the HSI design; consequently, special attention should be paid to those scenarios, critical human 

actions, and HSI components that were identified by HRA and PRA analyses as being important to the plant's safety and 

reliability. 

The objectives of this review are to ensure that (1) the applicant has addressed human error mechanisms in designing the 

plant's HFE; that is, considered the HSIs, procedures, shift staffing, and training, to minimize the likelihood of personnel 

error, and ensure errors are detected and recovered from, and (2) the HRA effectively integrates the HFE program and PRA 

and risk analysis. 

Element 7 - Human-System Interface Desin 

The selection of available HSIs and the design of n~ew ones should result from a process which considers function and task 

requirements, operational considerations (e.g., the full-mission context within which the HSI will be used), and the crew's 

personal safety. The HSI should be designed using a structured methodology that should guide designers in identifying the 

required information and controls, in identifying and selecting candidate HSI approaches, and in the final design of HSIs. 

It should cover the development and use of iFE guidelines and standards that are specific to the HSI design, and provide 

guidance for resolving differences between different HFE guidelines. It also should address the use of analysis and 

evaluation methodologies for dealing with design issues. The availability of an HSI design methodology will help ensure 

standardization and consistency in applying HFE principles. 

The objective of this review element is to evaluate the process by which HSI design requirements are developed and HSI 

designs are selected and refined. The review should assure that the applicant has appropriately translated function and task 

requirements to the alarms, displays, controls, and aids available to the crew. The applicant should have systematically 

applied HFE principles and criteria (along with all other function, system, and task design requirements) in identifying HSI 

requirements, selecting and designing HSIs, and resolving HFE and HSI design problems. The process and the rationale for 

NUREG/CR-6637 1 -6

I I
1 INTRODUCTION

the HSI design should be documented for review (including the results of trade-off studies, other analyses and evaluations, 

and the rationale for choosing design and evaluation tools). 

Element 8 - Procedure Development 

Procedures are an essential component of the HSI design and should be developed from the same design process and 

analyses as the other components of the HSI (e.g., displays, controls, operator aids), and evaluated in the same way. This 

will help to assure their full integration in, and consistency with, the HSI. 

The objective of this review is to ensure that the applicant's procedure development program will result in procedures that 

support and guide human interactions with plant systems, and control plant-related events. Human engineering principles 

and criteria should be applied, along with all other design requirements to develop procedures that are technically accurate, 

comprehensive, explicit, easy to use, and validated. 

Element 9 - Training Program Development 

Training plant personnel is an important factor in assuring safe, reliable operation. The NRC requires a systems approach 

to training and also requires that it is based on the systematic analysis ofjob and task requirements. The HFE analyses 

associated with the HSI design process provide a valuable understanding of such task requirements. Therefore, training 

development should be coordinated with the other elements of the HFE design process. 

The objective of this review is to ensure that the applicant establishes an approach for developing personnel training that 

incorporates the elements of a systems approach, and 

Evaluates the knowledge and skill requirements of personnel 

Coordinates the development of the training program with the other elements of the HFE design process 

* Implements the training effectively in a manner consistent with human factors principles and practices. 

Element 10 - Human Factors Verification and Validation 

Verification and validation (V&V) evaluations seek to comprehensively determine that the final design conforms to HFE 

design principles, and enables personnel to successfully and safely perform their tasks to achieve operational goals. This 

review involves five V&V evaluations, the objectives of which are to assure the following: 

HSI Task Support Verification - The HSI design provides the information, displays, and controls that are necessary 

to support the operators' tasks, i.e., all necessary alarms, displays, controls, and job-performance aids. 

HFE Design Verification - The manner in which information, displays, and controls are presented in the HSI 

design is consistent with human cognitive and physiological limitations, i.e., conforms to HFE principles, 

guidelines, and standards. 

Integrated System Validation - The integrated design including all information, displays, and controls can be 

effectively operated by personnel to satisfy all performance requirements associated with plant safety and 

operational goals.

1-7 NUREG/CR-6637
1 INTRODUCTION

Human Factors Issue Resolution Verification - The final HSI design resolves all identified HFE issues in the 

tracking system. 

Final Plant HFE/HSI Design Verification - The final HSI design, as built and installed (i.e., at a nuclear power 

station), conforms to the verified and validated final design resulting from the HFE design process. 

Support for the detailed aspects of V&V activities is given in NUREG-0700, Rev. 1. 

1.2.3 Human-System Interface Design Review Guideline (NUREG-0700) 

NUREG-0700, Rev. I (O'Hara et al., 1996) provides HFE guidance to the NRC staff for reviewing HSI designs. These 

reviews may address staff investigations of events involving human performance, voluntary modifications made to HSIs 

(e.g., where there is an unreviewed safety question involving the HSI or human performance), and reviews of the HSI for a 

new plant. The document is divided into two parts. Part 1, Review Methodology and Procedures, describes aspects of the 

HSI design review process that are important to identifying and resolving discrepancies in human engineering that could 

adversely affect plant safety. Part 2, HFE Guidelines, contains guidance on different aspects of HSI technology 

implemented in NPPs. 

The overall purpose of the HSI design review is to ensure that it supports safe, efficient, and reliable task performance. This 

is accomplished by systematically identifying and resolving deficiencies in design, called human engineering discrepancies 

(HEDs), that could adversely affect plant safety. The scope of the HSIs included in the review is defined on the basis of their 

function and not their physical location. Relevant HSIs include all alarms, displays, controls, job-performance aids, and 

workstation and workplace layouts. Also included are the environmental conditions in which the HSIs are operated, such as 

lighting, noise, temperature, and humidity. The methodology includes the following major phases of activity: 

Planning Phase - An HSI design review plan should be developed that adequately defines the evaluation in terms of review 

goals and scope, review team, management process, and technical approach. 

Preparatory Analysis Phase - The analyses that make up the preparatory phase provide valuable information which supports 

the establishment of requirements for the HSI design process. These analyses also provide the technical basis and criteria 

for conducting HSI design verifications and validation, i.e., they identify human performance issues and task requirements 

with which the HSI should be evaluated. A systems approach should be used to identify HSI design requirements. Three 

main analyses include 

Operating Experience Review - Operating experience should be reviewed, including examining plant performance 

reports and other documents, and surveying personnel to identify HSI-related human performance issues. 

Function and Task Analysis - System functions and personnel functions and tasks should be reviewed to identify 

HSI requirements and performance criteria for personnel tasks. 

HSI Inventory and Characterization - An inventory of the HSI should be developed, also describing its 

characteristics, functions, and performance features. 

HSI Design Verifications and Validation Phase - HSI design verifications and validation should be performed to ensure that 

HEDs have been appropriately identified and documented. Because no one method is likely to be sufficiently 

comprehensive, it may be necessary to perform a series of analyses: 

NUREG/CR-6637 1-8

I I
1 INTRODUCTION

HSI Task Support Verification - This evaluation verifies that the HSI supports all identified personnel task 

requirements as defined by task analyses. HEDs are identified for (1) task requirements that are not fully supported 

by the HSI, and (2) the presence of HSI components which may not be needed to support tasks. 

HFE Design Verification - This evaluation verifies that the HSI is designed and implemented to account for 

human capabilities and limitations. HEDs are identified if the design is inconsistent with HFE guidelines in 

NUREG-0700, Rev. 1. 

Integrated System Validation - This evaluation validates that the integrated HSI design enables the performance 

requirements for safe operation to be accomplished without imposing excessive workload. HEDs are identified if 

task-performance criteria are not met, or if the HSI imposes a high workload on personnel. 

HED Resolution Phase - This phase should ensure that HEDs have been assessed, and that important ones have been 

resolved. The assessment evaluates the safety significance of HEDs; those having no particular safety significance may be 

analyzed for improvement, but on a lower-priority basis. Once corrective actions are identified, a plan to implement them 

should be developed, including (I) evaluation of the installation and operation of all HSI modifications, and (2) correction 

of any problems with implementation. 

The overall NUREG-0700 review process accommodates tailoring based on review needs. In Section 1.4.5, Scope of the 

Review Process, the guidance indicates: 

The staff's review of an applicant's HSI design review process as described above and in the remaining sections of Part 1, is a 

comprehensive, detailed evaluation. Under certain circumstances, such as a preliminary review of an applicant's HSI design 

prototype, or a focused review of one aspect of the HSI that might be implicated in an incident involving human performance, 

the applicant's HSI design review, and correspondingly, the focus of the staffs' review, could be more limited. (pp. 14-15) 

Such an approach also is suggested as part of the detailed review process. For example, under the planning phase criteria, 

the NUREG states: 

For a major HSI upgrade or redesign, the applicant's design assumptions or constraints should be clearly identified. An 

assumption or constraint is an aspect of the design, such as the use of touch screens, that is an input to the HSI design process 

rather than the result of HFE analyses. The HSI design review should seek to evaluate the validity of such assumptions for the 

implementing design under review. (p. 17) 

Thus, consistent with the SRP and NUREG-07 11, NUREG-0700 reflects the use of a graded approach to reviewing the 

design process and products. 

In addition to NUREGs 0800, 0711, and 0700, NRC inspection manuals are used in reviews. These are discussed in the 

next section. 

1.2.4 Nuclear Regulatory Commission Inspection Manual 

The NRC Inspection Manual provides guidance for developing and conducting the NRC inspection program. Chapter 

0030, Policy and Guidance for Development of NRC Inspection Manual Programs, states that the objectives of the 

inspection programs are to (1) provide a basis for recommending issuance, denial, continuation, modification, or revocation 

of an NRC permit or license; (2) identify conditions within areas inspected that may adversely affect public safety so that 

appropriate corrective action(s) can be taken; (3) determine the level of effectiveness of the licensee's performance in

1-9 NUREG/CR-6637
1 INTRODUCTION

general, as well as in each inspection program area; and (4) determine the status of compliance with NRC regulations, 

licenses, and orders. These objectives are carried out through routine and reactive inspections. Routine inspections are 

systematic, scheduled reviews of licensee activities. Reactive ones focus on a particular aspect of the licensee's operation in 

response to a specific event or condition. 

The following describes chapters that may be particularly relevant to reviewing hybrid HSIs. 

Inspection Procedure 41500: Training and Qualification Effectiveness 

The objectives of this inspection procedure (NRC, 1995a) are to ensure that training is an appropriate response to identified 

problems in performance and that the training and qualification programs for NPP personnel are developed, implemented, 

evaluated, documented, and maintained as required under 10 CFR 50.120 and allowed by 10 CFR 55. This inspection 

evaluates the performance of NPP workers, the methods for licensee training and qualification (e.g., classroom, laboratory, 

simulation devices, on-the-job), and the effectiveness of implementing the systems approach to training. The applicant's 

training program may be inspected as a result of performance-related operational events. 

Inspection Procedure 52001: Diaital Retrofits Receiving Prior Approval, and Inspection Procedure 52002: Digital Retrofits 

Not Receiving Prior Avvroval 

These two inspection procedures deal with the review of digital retrofits (modifications) in NPPs. Inspection Procedure 

52001 (NRC, 1995b) covers those previously approved by the NRC. Its objectives are (1) to ensure that digital systems 

previously reviewed by the NRC staff are installed, operated, and maintained according to the safety evaluation, and in 

accordance with the manufacturer's design and operating recommendations (as appropriate), and licensee's commitments, 

and (2) to assess failures of the digital systems, their modifications and maintenance for their effect on the systems' 

functions, and for potential generic concerns. 

Inspection Procedure 52002 (NRC, 1995c) focuses on digital retrofits that have not received NRC approval. Its objectives 

are (I) to ensure that the licensee has properly considered the guidance for effective design of the digital system and 

satisfied the plant-specific licensing basis; (2) to ensure that the licensee has properly addressed the regulatory requirements 

of 10 CFR 50.59 on unreviewed safety questions; and (3) to assess failures of digital systems, their modifications and 

maintenance for their effect on the systems' functions, and for potential generic concerns. 

While the detailed criteria of Inspection Procedures 52001 and 52002 differ, their general approaches are similar. Each 

have criteria addressing the following: 

Verification of the design and its documentation against design requirements 

Reliability considerations (setpoints and related uncertainty terms) 

Procedures and practices 

• Training 

Resolution of previous failures (i.e., an operating-experience review). 

NUREG/CR-6637 1 -10

I I
1 INTRODUCTION

1.2.5 Summary 

The NRC's HFE/HSI review and inspection process provides guidance on evaluating the design, evaluation, and 

implementation processes. In applying it to hybrid HSIs in the context of plant modifications, the guidance is limited in two 

respects. First, while it allows tailoring of the review methods and criteria to a unique individual review, there is no 

guidance to assist in identifying the process elements and criteria that are needed. The extent of plant modifications can 

range significantly, e.g., from a replacement "in-kind" of a single HSI component to an extensive modification of the 

control room from analog to digital technology. The effects of the modifications can impact personnel by changing the 

following: 

Personnel role - functions and responsibilities of personnel, e.g., caused by a change in the plant's automation due 

to replacing a control system 

* Primary tasks - specific tasks performed by personnel to accomplish their functions and responsibilities 

* Secondary tasks - methods of interacting with the HSI and the demands it imposes 

• Personnel factors - required qualifications or training of personnel. 

Thus, guidance on tailoring is important and is needed. 

The second limitation of the guidance is that is does not specifically address the unique considerations of integrating new 

and old HSI components. As discussed elsewhere (O'Hara, Stubler, and Higgins, 1996), unique considerations are 

associated with modifying HSIs. For example, there are unique demands on crews during transitions from old to new 

systems. Thus, special considerations for each element of the design process review need to be identified, and the required 

guidance developed. 

It is important in the present effort to consider these limitations, and to ensure that the guidance developed is fully 

consistent and integrated. 

1.3 Earlier Research on HSI Upgrades 

A review of the HFE literature and industrial experience on introducing advanced technologies into traditional HSIs found 

that in such situations special challenges can be posed to personnel performance (O'Hara, Stubler, and Higgins, 1996). 

These concerns were further refined in the characterizations of this issue by Stubler et al. (1996) and Stubler and O'Hara 

(1 996a). Some identified concerns included compatibility of new technology with the existing HSI technology, and new 

demands imposed by new technology (e.g., changes in the operator's roles due to changes in automation), human 

performance considerations associated with developing and conducting personnel training, and those associated with the 

processes by which upgrades are installed and put into service. Stubler et al. (1996) describe this topic and analyze its 

potential safety significance. 

1.4 Organization of the Report 

This report is divided into two parts. Part 1 describes the methodology for developing guidance and its technical basis. The 

objectives of the study are described in Section 2, and the development methodology is described in Section 3. Sections 4 

and 5 present the technical basis for establishing of HFE guidance. Section 4 discusses the information processing 

1 - 11 NUREG/CR-6637
1 INTRODUCTION

capabilities of humans and how performance may be affected by changes in plant systems or the HSI. Section 5 continues 

the technical basis with a discussion of how these human performance issues should be considered during the design, 

development, and implementation of an upgrade. This discussion is organized around the 10 review elements of NUREG

0711. It describes how the general review criteria in NUREG-071 1 should be applied to a specific upgrade. The actual use 

of the technical information for developing guidance is discussed in Section 6. The guidance development effort is 

summarized in Section 7. Full references to the literature cited are listed in Section 8. 

Part 2 of the document contains results of the guidance development process; guidelines are organized according to the ten 

review elements ofNUREG-071 1.

NUREG/CR-6637 1 - 12

I I
2 OBJECTIVE

The objective of this study was to develop HFE review guidance to address design, analysis, evaluation, and implementation 

of hybrid HSIs based on a technically valid methodology. To support this objective, the following tasks were performed: 

* Development of a technical basis for guidance based on human performance research and accepted HFE 

procedures and practices for system development 

* Development of HFE review guidelines in a format that is consistent with NUREG-071 I 

• Identification of remaining issues for which human performance research and accepted HFE procedures and 

practices were insufficient to support the development of NRC review guidance

2-1 NUREG/CR-6637
3 METHODOLOGY 

3.1 Overview 

The overall methodology for guidance development for NUREG-0700 is shown in Figure 3.1; the process is discussed in 

detail elsewhere (O'Hara, Brown, and Nasta, 1996; Stubler and O'Hara, 1996a). The portion of the methodology applicable 

to this project is boxed in the figure. This section of the report describes the general rationale behind developing guidance.

Figure 3.1 Major Steps in the Development of NUREG-0700 Guidance

The methodology for guidance development was directed by the following objectives: 

* Establish a process that will result in valid, technically defensible, review criteria. 

* Establish a generalizable process that can be applied to any aspect of HSI technology for which review 

guidance is needed. 

* Establish a process that optimally uses available resources; i.e., a cost-effective methodology. 

The methodology places a high priority on establishing the validity of the guidelines, which is defined along two 

dimensions: internal and external validity. Internal validity is the degree to which the individual guidelines are based on 

an auditable technical basis. The technical basis is the information upon which the guideline is established and justified. 

Some guidelines may be based on technical conclusions from a preponderance of empirical research evidence, some on a 

consensus of existing standards, while others are based on judgement that a guideline represents good practices. 

Maintaining an audit trail from each guideline to its technical basis serves several purposes by enabling 

* the technical merit of the guideline to be evaluated by others, 

* a more informed application of the guideline, since its basis is available to users, and 

• deviations or exceptions to the guideline to be evaluated. 

External validity is the degree to which the guidelines are subjected to independent peer review. The peer review process is 

a good method of screening guidelines for conformance to accepted HFE practices, and for comparing guidelines to the 

practical operational experience of HSIs in real systems.

3-1 NUREG/CR-6637
3 METHODOLOGY

For individual guidelines, these forms of validity can be inherited from the documents that form their technical basis. Some 

HFE standards and guidance documents, for example, already have good internal and external validity. If validity is not 

inherited, however, it should be established as part of the guidance-development process. 

Figure 3.2 depicts the process used to develop the technical basis and guidance. It emphasizes those information sources 

that have the highest degree of internal and external validity. Thus, primary and secondary source documents were sought 

as sources of guidance first, followed by tertiary source documents, basic literature, industry experience, and other sources. 

From these fundamental data, design principles and lessons from industry experience were identified, and the guidance was 

developed. For specific aspects of the topic, in which there was an inadequate technical basis, unresolved research issues 

were defined. Thus, the analysis of information led to the development of both guidance and issues. The resulting guidance documentation includes HFE guidelines, technical basis, the development methodology, and unresolved research issues. 

Each of the steps of this research - topic characterization, technical-basis development, guidance development and 

documentation, issue identification, and peer review - is discussed in greater detail in the following sections. 

3.2 Characterization 

The first step in developing guidance for an HSI technology is to identify specific design characteristics for which guidance is needed. This characterization provides a framework for organizing guidelines and directing HFE reviews. However, the 

current review topic, HSI and Plant Modernization Process, is directed toward a design process, rather than specific design features. An acceptable methodology for reviewing an HSI design process is given in Chapter 18, Human Factors 

Engineering, of the SRP (NUREG-0800) and in NUREG-071 1. The review elements of this methodology were described in 

Section 1.2.2. The topical organization of these documents provides a framework with which to organize the guidance 

developed in this document. Therefore, it was not necessary to develop a separate characterization. 

Because the NUREG-0711 organization was adopted as the structure for this guidance, each section of the developed guidance relates to one of its ten review elements, and the present guidance complements the NUREG-07 I1 guidance. 

3.3 Development of the Technical Basis 

The technical basis for this topic was developed according to the process described in Section 3.1 and depicted in Figure 3.2. First, primary source documents were sought to provide a valuable starting place. These were HFE standards and 

guidance possessing internal and external validity; that is, the documents generally had their own research bases. The 

developers of these documents considered research and operational experience and, using their knowledge and expertise, developed HFE guidelines. These primary source documents were often extensively peer reviewed. They provided 

tremendous value-added to individual research reports. They were developed by experts who consider the applicability and 

generalizability of research to real systems, include their knowledge and expertise gained through operational experience and applying guidance, and modify the guidance based on extensive peer review. Included in this category were HFE 

review guidance documents that are currently used in reviewing NPP's HSIs, such as NUREG-0800 (SRP), NUREG-0711, 

and NUREG-0700. 

The development of the technical basis also considered the other sources identified in Figure 3.2, except original research. 

Secondary sources were documents for which either internal or external validity had been established. Documents for 

which neither was established were considered tertiary sources. The preference was to use documents for which validity 

was established. 

NUREG/CR-6637 3-2

I I
3 METHODOLOGY

Figure 3.2 Technical Basis and Guidance Development Process

3-3 NUREG/CR-6637
3 METHODOLOGY

In addition to these sources, the results from basic literature were analyzed (articles from technical journals, reports from 

research organizations, and papers from technical conferences). When guidance was based on basic literature, engineering 

judgement was required to generalize from the unique aspects of individual experiments to actual applications in the 

workplace. While research information is a valuable part of the guidance-development process, it usually cannot be adopted blindly; the results must be interpreted in the context of real-world tasks and systems using judgement based on professional 

and operational experience. 

Industry experience was also used, such as published case studies, surveys and interviews with knowledgeable domain 

experts. Included in this category were the findings from visits to a variety of control centers that were made in conjunction 

with this (O'Hara, Stubler, and Higgins, 1996) and other research projects (e.g., Heslinga and Herbert, 1993). Although 

such information lacks a rigorous experimental basis (and thus a measure of validity), it is highly relevant. 

The technical-basis development methodology stopped at this point. Where additional issues are identified, original research can be undertaken. This approach has the advantage of focusing on specific issues of interest, and has both high 

relevance and a sound experimental basis from which to establish validity. It is generally given the lowest priority because 

of the time and cost required to conduct original research. 

The technical basis was organized into two sections. Section 4 discusses the general characteristics of human performance in complex human-machine systems that may be affected by changes to the HSI. It is based on concepts from an accepted 

body of literature on human information processing and is augmented with research data that address specific human

machine systems. Section 5 discusses HFE considerations within the context of the design, evaluation, and implementation 

process. This section presents HFE considerations identified from research studies, reviews of industry experiences and 

practices, and handbooks. It also draws upon the high-level guidance in NUREG-0800, NUREG-071 1, and NUREG-0700. 

3.4 Development and Documentation of Guidance 

Once the technical information was assembled, a draft set of guidelines was developed from it. The guidelines were 

organized and specified in a standard format (discussed in Section 6). The guidelines themselves are given in Section 9 of 

Part 2 of this document. 

3.5 Identification of the Issues 

Where there was insufficient information for a technical basis upon which to develop valid design review guidance, an issue 

was defined. These issues are described in Section 5.6. From a research standpoint, they reflect aspects of the design, evaluation, or implementation of NPP upgrades whose resolution will require additional investigation. From a design review standpoint, these issues reflect HFE considerations that currently must be addressed on a case-by-case basis. For 

example, an issue can be addressed as part of design-specific tests and evaluations. 

3.6 Peer Review 

The resulting document containing the technical basis and guidance was submitted for review by individuals with topical 

knowledge and expertise including the NRC's staff with expertise in human factors engineering and other engineering 

fields. Human factors specialists external to the NRC who have expertise in human performance in complex systems, such 

as NPPs and aviation, also reviewed the document. These external reviews included evaluations of the topic's characterization along the following criteria: clarity, accuracy, and completeness. The evaluation of the technical basis 

NUREG/CR-6637 3-4

I I
3 METHODOLOGY

encompassed the following criteria: organization, necessity, sufficiency, resolution, and basis. Comments from the peer 

reviews were incorporated into this document.

3-5 NUREG/CR-6637
4 TECHNICAL BASIS DEVELOPMENT: HUMAN PERFORMANCE 

IN COMPLEX HUMAN-MACHINE SYSTEMS 

To perform successfully in a work environment, such as a NPP, personnel must develop the necessary knowledge and skills. 

When a plant system or HSI is upgraded, new tasks may be created and existing tasks may change. Personnel must adapt existing knowledge and skills or acquire new ones, subject to the capabilities and limitations of human information 

processing. The following discusses those human characteristics (knowledge, skills, capabilities, and limitations) that are 

relevant to performance in complex systems, and which affect the ability of people to adapt to changes resulting from 

upgrades. This section begins with a discussion of the relationship between personnel performance and the HSI (e.g., how 

human characteristics affect the performance of the plant, and vice versa). This is followed by an outline of primary and 

secondary tasks performed by personnel. Next, the basic properties of human information processing and skilled 

performance that are important for adapting to changes in plant systems or the HSI are described. Finally, we discuss 

generic cognitive tasks undertaken by operators in NPPs and their relationship to the characteristics of the HSI. This 

section forms the basis for Section 5, which discusses human performance in the context of specific stages of the design and 

implementation of upgrades. 

4.1 The Effects of HSI Design on Personnel Performance 

The operator's role in a NPP is that of a supervisory controller (O'Hara, 1994; O'Hara, Stubler, and Nasta, 1997). The 

plant's performance is achieved through the combination of the control actions of human operators and automatic control 

systems that are under the operator's supervision. The supervisory role requires the operator to monitor the behavior of 

systems and automatic control systems, and intervene, as necessary, to attain operational and safety goals. The HSI is a 

primary point of interaction between the operator and the plant. It is a major source of information and provides the media 

through which the operator acts on the plant. 

The operator's interaction with the plant may be modeled as a causal chain (Figure 4.1), as described in O'Hara (1997) and 

O'Hara, Stubler and Nasta (1997). When considering the operator's effect upon the plant, this chain may be viewed as 

starting from the operator's physiological and cognitive processes, and proceeding to task performance, which manipulates the HSI. These manipulations affect the plant's components, systems, and ultimately, its functions. Those of particular importance are the critical safety functions (CSFs), which are necessary for maintaining the plant in a safe condition. 

When considering the effects of upgrades on personnel performance, this causal chain may be viewed in the opposite direction. The design of systems and components affects the behavior of the plant. The status of the plant's functions, 

systems, and components is communicated to the operator via user interfaces of the HSI. This information imposes demands on the cognitive resources and physical capabilities of personnel in their roles as supervisory controllers. In 

addition, the characteristics of the HSI, including the way information is presented and control actions are executed, impose additional demands on operators. 

Changes made to a plant's systems and components, as part of an upgrade, may change the behavior of the affected systems. For example, a newly installed digital control system may introduce new functions and new control modes (e.g., higher levels of automatic control), or cause a system to respond differently to control inputs (e.g., act more quickly and exhibit less 

drift). As an example of new functions, some digital feedwater system upgrades automatically switch-over function from 

the auxiliary to the main feedwater systems during startup; these changes impose new demands on personnel as supervisory 

controllers. 

Changes made to the HSI affect the way that information is presented to personnel. For example, rather than being presented via individual, spatially dedicated instruments, the information may be presented via computer-based video 

display units (VDUs). The HSI modifications may also change the controls used by the operator. For example, physical control devices may be replaced by soft controls (Stubler, O'Hara, and Kramer, 2000), changing both the appearance and

4-1 NUREG/CR-6637
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

Figure 4.1 Causal Links Between Human and Plant Characteristics That Affect System Performance 

behavior of the device. In addition, new HSI components may include higher levels of automation. For example, display 

systems may include automation for selecting, processing, and presenting information. HSI modifications may place new 

cognitive and physiological demands on personnel. New cognitive demands may stem from the need to understand which 

information and controls are being presented, and how to access and use them. Physiological demands may stem from 

limitations on human perception, response time, and response accuracy associated with using the new interfaces. 

4.2 Primary and Secondary Tasks 

The role of an operator can be conceptualized as involving primary and secondary tasks. Primary tasks are those the 

operator performs as part of their functional role of supervising the plant. They require a knowledge of the plant and 

cognitive skills for making decisions and taking actions within the limitations of human information processing 

capabilities. For example, operators may be required to monitor automatic systems and intervene if the systems fail to 

perform acceptably. The primary tasks involve the generic cognitive tasks (i.e., monitoring and detection, situation 

awareness, response planning and implementation) which are described in Section 4.4. Secondary tasks are those that 

operators must perform when interacting with the HSI but which are not directly related to a primary task (O'Hara, Stubler, 

and Nasta, 1997). They include navigating through an information system, manipulating windows on a VDU, and 

manipulating job-performance aids. Secondary tasks result from the design characteristics of the HSI (i.e., some designs 

impose more secondary tasks than others). Secondary tasks require knowledge of the HSI and specific skills for accessing 

NUREGICR-6637 4-2

I I

Functions 

Critical safety function violations 

Systems 

Safety system actuations 

PLAN*T 4 f 

Components 

Major parameter values 

HSI 

Control, displays, & procedures 

Personnel Tasks 

Primary 

Secondary 

PERSONNEL Cognitive Factors 

Attention 

Cognitive workload 

Physiological Factors 

Vision & reach 

Physical workload
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

Selective Attention 

Selective attention (e.g., Parasuraman and Bowers, 1987; cited in Schlager et al., 1989) is the ability to shift attention to 

important things and ignore irrelevant things. A common effect of selective attention is popularly described as the "cocktail 

party phenomenon," in which a person at a crowded party is deeply involved in a conversation, and apparently ignoring the 

other conversations and sounds. Suddenly, the person's attention is drawn to another conversation when his or her name is 

mentioned, despite the fact that the person was not "listening" to the other conversation. The explanation is that people can 

unconsciously attend to many things, but they consciously attend to perhaps only one thing at a time. Certain stimuli, such 

as the mention of one's name, can shift in focus of attention. 

Selective attention is driven by both top-down and bottom-up processes (Wickens and Carswell, 1997). O'Hara (1994) 

gives a general discussion of the application of top-down and bottom-up processing in NPP control rooms (CRs). Bottom

up processes are based on characteristics of the stimuli that affect salience, such as size, loudness, and brightness. Bottom

up processing results in attention to stimuli that have highly salient characteristics. Top-down processes are developed from 

past experiences and are based on one's understanding of the environment (i.e., mental model) and expectations. Top-down 

processing results in attention to stimuli that have special significance, such as one's name as in the example above, or to 

information sources that are likely to produce important information. 

Sustained Attention 

Sustained attention (e.g., Parasuraman and Bowers, 1987; cited in Schlager et al., 1989), also called focused attention 

(Wickens and Carswell, 1997), is the ability to attend to a single source for an extended period. This type of attention has 

been studied in signal detection tasks in which an operator attempts to detect an target signal from a background of noise 

stimuli. 

4.3.2 Cognitive Skills for Overcoming Limitations of Memory and Attention 

The two distinctions described earlier, factual versus procedural knowledge and long-term versus working memory, along 

with attention, establish many of the constraints on the mental activities that accompany skilled performance. Because 

conscious attention can only be given to a limited number of sets of items at one time, personnel must develop cognitive 

skills to handle the large volume of changing information that occurs in complex human-machine systems, such as NPPs. 

The following are four generic learning mechanisms that support the transition from novice to experienced user in a task 

domain (Schlager et al., 1989). They represent different means of coping with the demands imposed on users when new 

technologies are introduced into their work environment. Applying these learning mechanisms within a task domain 

results in the acquisition of specific skills that allow the user to attain a high level of proficiency. 

Chunking of Information 

As described above, people can hold only a limited amount of information in working memory at one time. Consequently, 

when the demands on working memory are high, human performance may suffer. One mark of expertise is the ability to 

handle large quantities of information by forming it into chunks relevant to the tasks of one's particular domain. This 

strategy allows the user to attend to important information while staying within the limits of working memory. For 

example, plant information presented to NPP operators becomes easier to remember and process when groups of facts are 

stored together as meaningful chunks. Indications that correspond to a single fault (e.g., loss of emergency feedwater) or a 

single plant state (e.g., startup) may be organized in working memory as a single chunk, rather than as separate indications. 

Chunking also frees up some of the limited resources of working memory, which can be used for attending to other

NUREG/CR-6637 4-4
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

and using the information and control features. Thus, a goal of HFE in HSI design should be to minimize any negative 

effects of the secondary tasks on personnel performance. 

Section 4.3 describes human information processing limits, capabilities, knowledge, and skills that affect task performance, 

including primary and secondary tasks. 

4.3 Basic Properties of Human Information Processing and Skilled Performance 

Researchers have proposed various models of human information processing. The following describes some concepts 

common to many models. Included are discussions of some basic properties (capabilities and limitations of memory and 

attention) and cognitive skills (Schlager et al., 1989), followed by a discussion of human error. These concepts provide the 

basis for the exploration of generic cognitive tasks in Section 4.4, and are addressed in the personnel-performance topics presented in Section 5. Human information processing is described in more detail in Section 2.3 of Volume 1 of 

NUREG/CR-5908 (O'Hara, 1994). 

4.3.1 Memory and Attention 

A fundamental assumption of most models of human information processing is the existence of multiple types of memory (e.g., Waugh and Norman, 1965). Long-term memory is considered the repository of knowledge (i.e., things that have been 

learned). Its capacity for storing information is large - considered by some to be virtually unlimited. A distinction is 

commonly made between two types of knowledge: factual and procedural. Factual knowledge is stored as statements of 

fact, such as "A reactor coolant pump pumps coolant into the reactor vessel of a pressurized water reactor." This kind of 

knowledge can usually be stated fairly easily. Procedural knowledge is knowledge of how to perform actions, such as using 

a control to start operating a particular piece of equipment or access a specific page from a display. Procedural knowledge 

is not easy to communicate through statements and often requires demonstrations. 

A second type of memory is working memory, sometimes referred to as active memory. It contains things that are currently 

in consciousness (i.e., things that one is considering at the moment). Working memory may be compared to a short list. 

Unlike long-term memory, the number of items that can be stored at any time in working memory is severely limited; some 

say that the limit is about seven items (e.g., Miller, 1956). In attempting to exceed the limits of working memory, some 

items become lost. 

Although there appear to be severe limits on the number of items, the size of an item can vary. Thus, the amount of 

information held in working memory can be increased by increasing the size of the items by constructing a meaningful 

group ("chunk") of information, associated with a single label. This chunk then becomes a single item to be stored in 

working memory. Working memory can hold a large amount of information, despite its limitations, by storing the labels of 

large, meaningful chunks of knowledge (Newell and Simon, 1972; cited in Schlager et al., 1989). 

Another property of working memory is that it is the location where conscious activities are carried out, such as problem solving and decision making. This is accomplished by combining information from the environment and long-term 

memory with intentions (task goals). The gathering and processing of this information is mediated by the mechanism of 

attention, which determines which items of information will enter the working memory. Two types of attention, selective 

and sustained, are important to personnel performance in NPPs. 

4-3 NUREG/CR-6637

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

indications, or for planning and decision making. Chunking may be based on a number of attributes of the information, 

such as causal relationships (e.g., indications A, B, and C are all encoded as "symptoms of condition X") and structural 

relationships (e.g., components A, B, and C are coded as "parts of pump X"). 

Automaticity of Actions 

Automaticity refers to the ability to perform specific sets of actions without committing extensive cognitive resources 

(Mumaw and Gabrys, 1996; Schlager et al., 1989). These sets of actions are also called automatic actions or automatic 

procedures (Schiffrin and Schneider, 1977; cited in Schlager et al., 1989). When a user first learns a new set of actions, 

such as how to operate a particular control device, each step of the set requires attention and must be executed separately 

and consciously by the user. This limited level of performance uses working memory extensively. It requires operators to 

expend most of their limited cognitive resources on performing relatively low-level aspects of their jobs (e.g., remembering 

methods for searching, retrieving, and using displays and controls), thus preventing them from attending to more critical 

tasks, such as evaluating plant conditions, planning responses, and making decisions (Schlager et al., 1989). To become a 

skilled operator, one must learn to "automate" these sets of actions. A familiar example is learning to type on a keyboard. 

Initially, this task requires a great deal of cognitive resources in searching for characters on the keyboard or trying to recall 

which keys correspond to which fingers when touch-typing. However, a skilled typist can type quickly and accurately, 

relying little on cognitive resources. This freed-up capacity may be used instead for deciphering and comprehending a 

complex, hand-written document. 

Not all actions can be performed "automatically." While it is desirable to "automate" low-level sets of actions, such as steps 

for operating a control, other actions require conscious control of cognitive resources, such as those involving decisions on 

whether to operate a control or which value to enter. As another example, some actions require a great deal of attention, 

such as those requiring careful positioning. Other sets of actions cannot be "automated" because some actions are not the 

same every time they are performed. For example, some actions in the set required for operating a control may be unique, 

such as determining a particular value to enter as a control setpoint. In addition, performing actions without extensively 

focusing attention on the task can contribute to errors of execution (Norman, 1988, 1983, 1981; Stubler, O'Hara, and 

Kramer, 2000). 

Use of Selective Attention 

Performers learn through experience where and when to look in their work environment to gain the greatest information, 

and selectively focus attention on these sources. In dynamic environments, such as NPPs, there is a tendency for operators 

to attend to those sources that change most frequently (i.e., contain the most information in terms of bits per unit of time), 

or are likely to change given the current situation. These are examples of top-down processing (e.g., based on their 

understanding of the current situation, operators develop expectations of information sources that will provide the most 

useful information). Monitoring based on top-down processing is referred to as model-driven or knowledge-driven 

monitoring. Proficient NPP operators rely on this capability to allow them to shift their attention between the many sources 

of information in the CR, especially when situations change; they learn which stimuli represent things that are likely to 

require their attention. 

When scanning large volumes of data, personnel also use the bottom-up processes by focusing on information that is highly 

salient. This provides one mechanism for prioritizing information sources (i.e., highly salient information is attended to 

before other less-salient information). Monitoring that is based on bottom-up processes is referred to as data-driven 

monitoring. (Section 4.4.1 describes the application of bottom-up and top-down processes to monitoring tasks).

4-5 NUREG/CR-6637
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

Use of Sustained Attention 

Proficient performers develop the ability to maintain attention on important information sources and detect meaningful 

signals amid random fluctuations with little operational significance. Computer-based automation is sometimes introduced 

as a way of improving an operator's performance in these detection tasks. Rather than eliminating the human monitor, this 

typically combines human and computer monitoring. In some cases, the combined monitoring performance can be worse 

than that of either one alone (Corcoran, et al., 1972; cited in Schlager et al., 1989). This case is especially true when the 

automation's performance increases to the point where the human becomes overly reliant upon it. 

4.3.3 Cognitive Skills for Problem Solving 

In addition to skills developed to overcome human limitations of memory and attention, personnel develop cognitive skills 

for problem solving. These skills are closely linked to knowledge of the specific work domain (e.g., NPP operations). Low

level skills support the execution of the high-level skills. Examples include operating a control or selecting a display from 

the display systems. These skills are often performed under conditions that allow them to be trained for automaticity. 

High-level skills are related to decision making, and are often performed under variable conditions, which do not allow such 

training (Schlager et al., 1989). 

Three high-level skills are particularly relevant to NPP operations: planning, use of strategic knowledge, and use of mental 

models (Schlager et al., 1989). Planning skills are involved with setting goals and determining paths to them. Strategic 

knowledge refers to the use of knowledge of possible strategies (paths) for satisfying a particular goal. A mental model may 

be described as mental representation of the structure and behavior of a system; its use can aid the user in explaining, 

predicting, and controlling the system's behavior (Kramer and Schumacher, 1987, cited in Webb and Kramer, 1990). 

Cognitive skills are usually combined with low-level skills and may be used in complex ways when solving problems. The 

transitions between plans, strategic knowledge, and mental models are not linear. For example, an individual may begin to 

plan, identify one or more sub-goals, access a mental model to determine functional relationships, and then solve the 

problem associated with some of the related sub-goals. Then, the individual may return to the planning activity to identify 

and address more goals and sub-goals. 

4.3.4 Human Error 

The following briefly discusses human error; a more detailed description, applicable to NPPs and other complex human

machine systems, is provided in O'Hara (1994). When cognitive resources are not used effectively, or when they become 

depleted, human performance suffers. Human error occurs when a person does not perform a necessary action, or performs 

an inappropriate one. When a person does not perform a needed action within the required time, it is sometimes called an 

error of omission. When a person performs an inappropriate action, it is sometimes referred to as an error of commission. 

There have been many attempts to define the mechanisms causing errors. One widely accepted scheme classifies human 

errors into two major categories: mistakes and slips' (Norman, 1983 and 1988; Lewis and Norman, 1986; Reason, 1990). 

This distinction is based on consideration of intention, a high-level specification of actions which starts a chain of 

processing that normally results in the actions being carried out. 

An error in intention formation, such as forming an inappropriate plan, is called a mistake. Mistakes are related to 

incorrectly assessing the situation or inadequately planning a response. When a plant is upgraded, mistakes can occur when 

The category of errors described by Reason (1990) as lapses is not discussed here. 

NUREG/CR-6637 4-6

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

personnel plan inappropriate actions because they do not accurately understand the new systems or HSI components. Slips 

are errors in carrying out intentions, during which a person intends to do one thing but accomplishes another. Slips result 

from "automatic" human behavior, when subconscious patterns of actions (schemas) intended to accomplish the intention, 

get waylaid en route to execution (Norman, 1983, 1988). Thus, slips tend to occur with skilled users, rather than beginners 

learning new activities. The highly practiced behavior of an expert leads to the lack of focused attention and increases the 

likelihood that slips will occur. This lack of attention results in the incorrect activation and triggering of schemas. 

Many types of slips have been identified (Norman, 1981, 1983, 1988; Lewis and Norman, 1986). A review of computer

based HSIs used in process control and other complex systems found that soft controls are particularly susceptible to the 

types of slips listed in Table 4.1 (Stubler, O'Hara, and Kramer, 2000); their likelihood may be increased for upgrades that 

include soft controls. 

Table 4.1 Types of Slips That May Affect Soft Control Usage

4.4 Generic Cognitive Tasks 

Generic cognitive tasks include monitoring and detection, situation assessment, response planning, and response 

implementation (O'Hara, 1994; O'Hara, Higgins, Stubler, and Kramer, 2000). These cognitively demanding activities are 

performed by personnel in their roles as supervisory controllers. Any primary task typically can be described in terms of 

one or more of these generic cognitive tasks. The following discusses these generic cognitive tasks, including how the basic

NUREG/CR-6637

Capture Error: An error of execution (slip) that occurs when an infrequently performed action requires a 

sequence of operations, some of which are the same as, or similar to, those of afrequently performed action. In 

attempting the infrequent action, the more frequent action is performed instead. For example, an operator 

intends to perform task 1, composed of operations A, B, C, and D, but instead, executes the more frequently 

performed task 2, composed of operations A, B, C, and E. 

Description Error: An error of execution (slip) that involves performing the wrong set of well-practiced 

actions. Description errors occur when the information that activates or triggers the action is either ambiguous 

or undetected. 

Loss-of-Activation Error: A slip that occurs when an intended action is not carried out due to a failure of 

memory (i.e., the intention has partially or completely decayed from memory). A special case of loss-of

activation errors involves forgetting part of an intended act while remembering the rest (e.g., retrieving a 

display while not being able to remember why it is needed). 

Misordered Components of an Action Sequence: A slip involving skipped, reversed, or repeated steps. Soft 

controls may be prone to this type of slip because they require sequential operations for access and use. 

Mode Error: Performing an operation that is appropriate for one mode when the device is in another mode. 

Mode errors occur when the user believes the device is in one mode when it is actually in another and, as a 

result, performs an inappropriate action. 

Unintentional Activation: A slip that occurs when a set of actions (schema) that is not part of a current action

sequence becomes activated for extraneous reasons, and then is triggered, leading to the unintended actuation of 

an input device.

4-7
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

properties of human information processing and skilled performance (Section 4.3) are incorporated into them and also how 

the characteristics of upgrades may affect them. This is presented first in terms of individual performance, and then in 

terms of a crew's performance. 

4.4.1 Individual Performance 

Monitoring and Detection 

Monitoring and detection refer to the activities involved in extracting information from the environment, such as checking 

the plant's state to determine whether the systems and equipment are operating correctly. This can include monitoring 

parameters indicated on the CR panels and displayed by the process computer, obtaining verbal reports from operators in 

different areas, and sending operators to areas to check on equipment. Monitoring also includes receiving information from 

other personnel, such as when one operator listens to another calling out values of requested variables. Detection is the 

operator's recognition that something is not operating correctly, and that an abnormality exists. Monitoring and detection 

are influenced by two factors: the characteristics of the environment (e.g., the HSI), and a person's knowledge and 

expectations. 

In a CR environment, the HSI is one of the most important sources of information. Monitoring relies on the use of HSI 

characteristics that support the identification of information sources; this is important because personnel must be able to 

find the right indicator or display before it can be read. Some HSI characteristics supporting identification include display 

navigation mechanisms and labels for indications. The inadequate or inappropriate use of such characteristics can result in 

an operator selecting the wrong indicator or display. Capture errors and description errors are two examples of slips that 

may result in wrong choices (Stubler, O'Hara, and Kramer, 2000). 

Detection is influenced by HSI characteristics which affect human perceptual processes and support personnel in noticing 

the information. Examples include display formats, such as digital readouts and graphical (analog) representations, and 

coding schemes, such as color and shapes that affect the salience of information. 

The HSI characteristics that support monitoring and detection interact with people's knowledge and expectations. This 

interaction affects their selective attention capabilities, producing data-driven and model-driven monitoring (described in 

Section 4.3.2). For example, in NPP CRs, alarm systems are basically automated monitors that influence data-driven 

monitoring using visual and auditory salience to draw attention. Auditory alerts, flashing, and color coding are some 

physical characteristics that enable operators to quickly identify important alarms. Data-driven monitoring depends on the 

relative and absolute levels of salience. The strength of a stimulus must exceed some threshold to draw attention. Above 

that value, personnel tend to notice stimuli with the greatest salience. 

Model-driven monitoring is directed by people's knowledge and expectations. It is active monitoring in the sense that 

personnel deliberately direct attention to areas they expect to have useful information, rather than merely attending to 

stimuli based on their relative salience. Model-driven monitoring may be initiated by several factors. First, it may be 

guided by operating procedures or standard practice (e.g., control panel walk-downs that accompany shift turnovers). 

Second, it can be triggered by situation awareness or response planning activities and, therefore, is strongly influenced by 

the individual's understanding of the current situation. This understanding allows the operator to direct attention more 

effectively by focusing monitoring activities on information sources that are likely to change or provide useful information. 

However, model-driven monitoring can also lead operators to miss important information. For example, incorrect 

understanding may cause an operator to focus attention in the wrong place, to fail to observe a critical finding, or to 

misinterpret or discount an indication. 

NUREG/CR-6637 4-8

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

Operators in a NPP CR may be faced with an information environment containing more variables than can be realistically 

monitored at one time. For example, during an early transient, they may be confronted by an overwhelming number of 

alarms and changing parameters. Due to constraints in human information processing, the number of indications that 

operators can attend to at any given time or maintain in working memory is limited. Attention resources can be used more 

efficiently by mentally combining individual indications into related groups using strategies for information chunking 

(Section 4.3.2). The ability to chunk information can be strongly affected by the way it is presented. Display formats, 

physical proximity, and coding methods can enhance or impair the ability of people to perceive patterns and relationships. 

These factors can produce emergent features (Wickens and Carswell, 1997) that convey-higher-level information. For 

example, if all the needles on a set of indicators normally point in one direction, then the presence of a needle pointing in 

another direction conveys at a glance the higher-level message that status is normal for all indicators except the one with 

the abnormal orientation. (The application of emergent features and other concepts to designing computer-based displays 

is discussed in O'Hara, Higgins, and Kramer, 2000.) 

Information chunking is also affected by the operator's mental models. Some chunks are static; the relationships do not 

change with changes in plant status. For example, functional relationships based on the structure of the plant do not 

change. Other relationships between individual indications change based on the operator's understanding of the situation. 

For example, a set of indications may be grouped together if they are thought to be related to a common cause, such as a 

single malfunction or the plant's operating mode. 

CR personnel may also have to filter information by applying selective attention to the most relevant indications. Other 

indications may be ignored or relegated to a lower status that is monitored when resources permit (Mumaw et al., 1994). 

This strategy requires the operator to decide what to monitor and when. Filtering strategies are related to the operator's 

understanding of the current situation, which guides the allocation of attention resources for sampling data from the 

environment based on statistical properties. These include the expected probability that useful information will be found 

and the correlation of related indications (e.g., the way that one variable behaves in relationship to others). Filtering can be 

part of an iterative monitoring process. An initial indication of an anomaly can cause operators to adjust their 

understanding of the current situation, which affects the way future indications are filtered. Thus, the design characteristics 

of the HSI that convey information about plant status to the operator, such as alarms and coding of abnormal indications, 

can significantly affect the use of appropriate filtering strategies. 

During an HFE review, attention should be given to HSI upgrades that change the way that information is presented and, 

thus, affect monitoring and detection, including low-salience characteristics that reduce the detectability of indications and 

high-salience ones that encourage data-driven monitoring. The characteristics of information display characteristics that 

affect model-driven monitoring also should be considered. This includes presentation modes that influence people's ability 

to understand the current state of the plant, such as the overall status of systems. It also includes characteristics of 

personnel's ability to recognize relationships between conceptually related indications. Failures in monitoring can include 

failing to observe parameters, observing the wrong ones, misunderstanding their significance, or failing to obtain needed 

information from operators elsewhere in the plant. Failures in detection can include failure to recognize an abnormality 

despite appropriate monitoring. An error in monitoring or in detection can cause the operator's failure to respond to the 

event within the requisite time. 

Situation Assessment 

Situation assessment refers to the cognitive activities involved in processing stimuli to construct an understanding of the 

situation. Two important ones are interpreting the current state and determining its implications (Mumaw et al., 1994). 

Interpreting the current state entails developing a mental representation of the plant's status based on information from

NUREG/CR-6637 4-9
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

monitoring and detection. This activity is affected by limitations and biases in the human information processing system. 

For example, "garden path" interpretations may occur when the symptoms of the plant's current condition resemble those of 

a familiar or well-learned event. Then, this initial correspondence influences the operator's expectations, leading to an 

incorrect diagnosis. Interpreting the current state involves skills for properly assessing different types of information, 

including that which confirms, disconfirms, or differentiates it from information gathered earlier. Humans in dynamic 

information environments are biased toward seeking only confirmations (e.g., Evans, 1989, cited in Mumaw et al., 1994); 

that is, they are likely to discount information that contradicts their current interpretation of the plant's state or that 

supports other interpretations. This is called the confirmation bias. It poses the risk that personnel will develop incorrect or 

incomplete interpretations (Mumaw et al., 1994). 

Determining the implications of the current state entails "mental simulation" in which future states are anticipated, goals 

for action identified, and priorities for goals established. Mental simulation relies on mental representations of the plant 

and the current situation. Deficiencies in these representations can result in inaccurate assessments of implications. These 

mental representations involve two related concepts: the mental model and the situation model. Each is described below. 

Mental Model - The mental model of the plant is the operator's internal representation of the physical and functional 

characteristics of the plant and its operation. An operator may possess multiple mental models of different parts of the plant 

at different levels of abstraction. They are developed through training and experience and are reinforced by the way 

information is displayed in the HSI, such as instrument layouts depicting relationships between components in a system. 

Mental models are not always fully accurate or complete. When plants are upgraded, personnel may need to modify their 

mental models to accommodate the changes. 

Changes in systems or the HSI can add new levels of complexity that must be incorporated into the operator's mental 

model. Systems often have complicated interconnections with other systems and variables. With the introduction of digital 

I&C technology, these interconnections may become even more complex. That is, with analog control systems, the 

connections between control devices and the equipment tend to be rather direct. However, with digital I&C systems, these 

connections may become more complicated (e.g., include data buses, signal multiplexers, and intermediate processors). In 

addition, introducing more automation in systems greatly increases their complexity, making them more difficult to 

understand. This fact was confirmed during site visits to foreign NPPs (O'Hara, Stubler, and Higgins, 1996). Training 

personnel stated that for the newer plants, which feature more automation, the operators had to learn much more 

information than operators of the older plants to have equivalent understandings of the plant's structure and functions. 

HSI upgrades, such as advanced display systems, alarm systems, computerized operator support systems, and computer

based procedure systems, can support the formation of the operator's mental model of plant systems. For example, 

graphical displays can depict complicated functional or physical relationships between plant components, reinforcing the 

operator's understanding. However, HSI upgrades can also impose new demands on the personnel by increasing HSI 

complexity (e.g., introducing computer-based display systems and new methods of interaction). New information about the 

HSI (e.g., the organization of display structures and the new methods of interaction) must be incorporated into the 

operator's mental model of the HSI. 

Situation model - The situation model is an understanding of the specific current situation. It is based on factors known, or 

hypothesized, to be affecting the situation at a given time. To construct a situation model, operators use their general 

knowledge and understanding about the plant and its operation to interpret the information. The situation model is affected 

by the way the plant's behavior is represented by the HSI and is constantly updated as new information is received; any 

limitations in general or specific information may result in incomplete or inaccurate situation models. 

NUREG/CR-6637 4- 10

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

The ability to form an adequate situation model may be negatively affected by changes in either plant systems or the HSI. 

Changes in systems, such as installing digital instrumentation and control (I&C) systems, can change the plant's behavior, 

such as by increasing or decreasing a system's response time. Introducing additional automation can introduce new control 

modes, which operators must monitor to determine whether they are functioning properly and satisfying plant goals. Such 

changes add new dimensions in plant performance that must be addressed by the situation models. Situation models must 

also account for the status of the HSI; operators must consider possible faults in the HSI when interpreting abnormal plant 

indications. In addition, when HSI upgrades involve higher levels of automation, operators must consider its behavior in 

their situation model. For example, computer-based display systems and decision aids may automatically retrieve and 

present information to the operator, based on their interpretations of plant conditions. Cognitive demands may be imposed 

on the operator in understanding how these systems operate (e.g., why a particular piece of information was displayed), and 

anticipating their future behavior as conditions change. 

Inadequacies in situation awareness may cause potential threats to plant safety to go unrecognized. As a result, operators 

may fail to take action early in a developing transient, and so conditions worsen. The plant's state may be misdiagnosed 

due to misinterpretation of symptoms. Such failures in situation awareness can affect the appropriateness or adequacy of 

response planning (discussed next), thus generating errors (i.e., mistakes). For example, as a result of misdiagnosis, 

operators may plan a response to a particular fault condition when a different one exists. Alternatively, operators may 

correctly diagnose the fault but then select actions that cannot be successfully completed under the plant's current 

configuration of the plant (e.g., a necessary system is not available). 

Response Planning 

Response planning refers to the cognitive task of developing an approach for achieving a goal. The planned response may 

be as simple as deciding to access a particular parameter from a particular HSI component, or it may involve selecting a 

complicated course of action, guided by a procedure. Response planning based on procedures, established practices, or 

heuristics is referred to as rule-based decision making (Rasmussen, 1986). In some cases, personnel may be forced to make 

decisions for situations that are not explicitly or completely addressed by procedures. Then, they must develop plans based 

on their knowledge of the plant, the situation, and possible success paths; this is referred to as knowledge-based decision 

making (Rasmussen, 1986). Roth et al. (1994) describe the use of knowledge-based behavior during simulations in which 

the complexities of the interfacing system loss-of-coolant accident (ISLOCA) scenarios made it difficult for operators to 

merely follow the procedures. Knowledge-based decision making is especially important during severe accident 

management (e.g., Mumaw et al., 1994) in which conditions may extend beyond those explicitly addressed by the 

emergency operating procedures. 

Response planning involves using one's situation model to select goals and one's mental model of the plant to identify paths 

for achieving them. This can place high demands on cognitive resources, such as working memory, long-term memory, and 

attention, especially in unfamiliar situations, when procedures do not appear to be adequate. Individual skills involved in 

response planning include the ability to identify appropriate existing plans (such as a particular emergency operating 

procedure), formulate a response plan, evaluate a response plan, and determine the sequence of actions required to execute a 

plan (Mumaw et al., 1994). 

These response-planning skills may be strongly influenced by features of the HSI. For example, identifying a response plan 

depends on whether personnel can maintain awareness of the plant's changing state, identify the highest priority goals, and 

evaluate the suitability of existing procedures in terms of available time, required availability of plant systems, and required 

entry conditions. This ability may be supported by HSI components, such as alarms and displays, that enhance an 

operator's situation awareness and support identifying goal conflicts, and decision aids that support the evaluation of 

existing procedures for current conditions. Formulating and evaluating response plans are also supported by these features,

4-11 NUREG/CR-6637
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

and may be extended by HSI features that support the assessment of potential consequences and side effects of various 

response paths. Response planning may be negatively affected by decision aids with limited or incomplete guidance or that 

lack mechanisms for examining the rationale and limitations of the guidance. 

Failures in response planning result in the development of inappropriate plans that, if executed, may be described as 

mistakes (Section 4.3.4). Inappropriate plans may have a range of consequences. For example, those that produce 

unnecessary but inconsequential actions are generally ignored in an error analysis. They also may produce inefficiencies 

that are of little consequence. For example, the desired goal may be achieved but with a greater expenditure of resources 

(e.g., time and personnel) than by other available paths. Finally, some inappropriate plans may have significant negative 

consequences, such as the violation of a limit set by technical specifications, loss of a safety function, or the failure to 

effectively isolate a contamination source. It is these mistakes that are of greatest concern for response planning. 

When conducting HFE reviews of NPP upgrades, consideration should be given to changes in a plant's functions that create 

new demands for response planning, and, therefore, new opportunities for mistakes. Attention also should be directed 

toward HSI characteristics such as display design features, procedures, and decision aids, that may affect the ability of 

personnel to make appropriate plans. 

Response Implementation 

Response implementation is the performance of the actions identified during response planning. This can be as simple as 

selecting and operating a single control, or as complicated as executing a control sequence involving multiple controls and 

displays and personnel. The HSI's design determines where and how these actions can occur, and where and how feedback 

can be monitored. 

Two aspects of NPPs that can complicate response implementation are response time and the directness of observation. 

Delays in response time and the accompanying feedback disrupt implementation because they make it difficult to determine 

whether control actions are having their intended effect. The operators then may have to rely on predicting future states 

based on their mental models of the plant's behavior. This prediction is prone to errors (i.e., mistakes) due to the 

complexity of system behavior. For example, errors in predicting a system's response may result in operators providing 

inputs that cause the system to overshoot or undershoot the target value, generating a transient in which the parameter 

swings widely outside acceptable control ranges. Thus, the task of implementing a response can be negatively affected by 

changes in the plant's systems that delay response times. 

While eliminating such delays can avoid problems associated with predicting the system's behavior, a fast response can 

pose other problems. When control systems are upgraded (e.g., with digital systems), their speed may increase to the point 

that they can react almost immediately to inputs from operators. This can create opportunities for errors of execution (i.e., 

slips). A review of industrial experience with digital systems found that, in some cases when an operator made an incorrect 

entry, these systems responded so quickly that the operator did not have sufficient time to correct it, nor halt the system's 

response (Stubler, O'Hara, and Kramer, 2000). For example, if an incorrectly entered setpoint differed greatly from the 

previous value, then system might change rapidly, causing a transient and damaging equipment. Thus, increases in the 

speed of plant systems can have negative affects on the response-implementation task if features for preventing, detecting, 

and correcting errors are inadequate. 

The directness of observation refers to the fact that operators usually cannot directly observe the effects of their control 

actions on systems. Instead, they must be inferred from the HSI's displays. The ability of people to make inferences is 

affected by the way information is presented. For example, the operator may not be able to interpret the status of plant 

systems if the appropriate display cannot be quickly accessed, nor if the display is not well suited to this task. Thus, 

NUREG/CR-6637 4- 12

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

display-design characteristics of HSI upgrades can have important effects on the operator's implementation response 

(O'Hara, Higgins, and Kramer, 2000). 

Failures in response implementation can lead to the operation of the wrong equipment, or the incorrect operation of the 

correct piece of equipment. If an upgrade changes a system's response time, then the potential effects on personnel's 

performance, including errors resulting from either an increase or a decrease, should be assessed. If an upgrade will change 

the way information is presented via the HSI, these potential effects also should be assessed. These assessments should 

include the ability of personnel to select the correct control, execute the control action properly, and correctly evaluate the 

effects of the action. Particular attention should be given to tasks that are potentially safety significant. 

4.4.2 Crew Performance and Team Skills 

A team may be defined as a set of two or more people who interact, dynamically, interdependently, and adaptively toward a 

common and valued goal, objective, or mission, each having specific roles or functions to perform (Salas et al., 1992). In a 

study of team performance in naval tactical training, Oser et al. (1989) identified the following behaviors that were 

important to performance: identification and resolution of errors, coordinated information exchange, and team 

reinforcement. Successful teams actively located errors, questioned improper procedures, and monitored the status of 

others. A NPP operating crew is a team in which the members share information and perform their tasks in coordination to 

satisfy specific operational and safety goals. The crew members interact with each other and with other personnel in the 

plant. The HSI provides the context within which personnel convey information and coordinate actions. The four generic 

cognitive tasks (i.e., monitoring and detection, situation awareness, response planning, and response implementation) each 

require interactions between crew members. The following discusses these tasks from the perspective of crew performance, 

and how performance may be affected by upgrades. 

Monitoring and Detection 

The ability of crew members to communicate information about the plant's status to the primary decision maker (e.g., 

senior reactor operator or shift supervisor) is a major consideration of monitoring and detection. Operators must 

communicate in a way that allows decision makers to rapidly incorporate this information into their developing situation 

models. Communication failures can have many causes. Cognitive causes stem from a lack of a common understanding. 

Much spoken communication relies on a shared context between the speaker and the listener to support the correct 

interpretation of the intended meaning. Typical failures involve hidden differences in frames of reference and assumptions 

(Mumaw et al., 1994). Communication may be impaired to the degree that personnel have different situation models. For 

example, due to differences in their understanding of the plant's state, an operator may misinterpret a request for 

information from the decision maker, or the decision maker may misinterpret information provided by an operator. 

Communication may also be affected by physical considerations. For example, upgrades can change operators' tasks, thus 

increasing the physical distance between operators (e.g., an operator may be required to spend more time at back panels, or 

away from the main control panel). Other physical considerations include modifications that affect the line-of-sight 

between operators, and ones that increase background noise making speech less intelligible. 

Situation Awareness 

During a transient, the primary decision maker attempts to develop a coherent understanding of changes in the plant's 

status from available information. Due to limited cognitive resources for seeking information and drawing inferences, the 

decision maker relies on other crew members for input. Operators in the CR are likely to develop different interpretations 

of the plant's state from the information they access, and differences in their individual representations of system status. 

However, if crew members are aware of the decision maker's interpretation of events, they can seek confirming and

4- 13 NUREG/CR-6637
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

disconfirming evidence. Thus, the interpretation of plant status depends upon the ability of decision makers to convey their 

interpretations, and the ability of crew members to communicate their findings and interpretations. Another element of 

situation awareness, determining the implications of plant state, may require the decision maker to mentally simulate events 

to predict future states. Team performance depends upon the decision maker's ability to communicate the intermediate 

results of these mental simulations to crew members so they may provide input (Mumaw et al., 1994). 

Response Planning 

During response planning, the decision maker formulates plans based on specific goals. Here, interactions between 

personnel are important, especially when the situation cannot be addressed by straightforwardly applying an existing 

procedure (e.g., incidents not explicitly addressed by emergency operating procedures; severe accidents). One important 

skill is the ability of the decision maker to communicate the intermediate results of response planning to others, so they may 

help evaluate it and contribute additional information. Another is the ability of crew members to convey evidence to the 

decision maker showing that the plan is no longer feasible, such as when plant conditions change quickly (Mumaw et al., 

1994). 

Response Implementation 

Response implementation requires the coordinated actions of crew members to execute the selected plan. People must 

maintain awareness of each other's actions, so that parallel activities are coordinated. In addition, operators may monitor 

each other's actions to detect errors in responding. 

Crew performance is supported when personnel can develop a shared view of the plant's status and CR activities. This may 

be supported by HSI characteristics that provide them with access to similar sets of information about plant conditions, 

support their ability to communicate plant information to each other effectively, support coordination of activities by 

allowing them to maintain an awareness of each other's actions, and support monitoring of personnel's performance to 

detect errors. Hutchins (1990) identified three characteristics of the work environment that support team performance: 

Horizon of Observation - This refers to the portion of the team's task that can be seen or heard by each individual. 

It results from the arrangement of the work environment (e.g., proximity of team members) and is influenced by 

the openness of tools and the openness of interactions. By making portions of a task observable, other team 

members can monitor for errors of intent and execution, and for situations in which additional assistance may be 

helpful. 

Openness of Tools - This refers to the degree to which an observer is able to infer information about another crew 

member's ongoing tasks by observing a tool's use. Open tools show characteristics of the problem domain that 

provide an observer with a context for understanding what has been done, and the possible implications. For 

example, a control task carried out by directly manipulating a graphical interface (e.g., mimic representation of a 

system) may be more useful for an observer (by showing which component is being operated and the effect on the 

rest of the system) than if the same task was done by other means, such as with text commands via a keyboard. 

Openness of Interaction - This refers to the degree to which the interactions between team members provide an 

opportunity for others with relevant information to contribute. Openness of interaction depends on the type of 

communication (e.g., discussing actions or decisions in the presence of others) and the style of interaction (e.g., the 

degree to which unsolicited input is accepted). Openness of interaction is also influenced by characteristics of the 

work environment (e.g., openness of tools, horizon of observation) that provide other team members with an 

opportunity to see and hear the interaction. 

NUREGICR-6637 4-14

I I
4 DEVELOPMENT OF THE TECHNICAL BASIS: HUMAN PERFORMANCE

Conventional control room designs typically provide a broad horizon of observation that facilitates the observation of team 

activities (Stubler and O'Hara, 1996b). Conventional hardwired controls and displays may be "open tools" in the sense that 

an observer can infer information about actions (e.g., which plant system was involved, which control was operated, what 

action was taken) by observing the operator's location at a control panel and the action performed. Interactions may be 

considered "open" when most are verbal ones that can be heard by others in the control room. Practices such as calling out 

requests for information and calling back the responses allow other control room personnel to maintain awareness of 

operational activities. 

The crew's performance may be impaired when upgrades change these HSI features and operational practices, as may occur 

when the display or interaction features of HSI upgrades interfere with the ability of personnel to share a view of the plant. 

An example is an individual operator console, which gives individual views of the plant that are not easily seen by others. 

If other crew members cannot view similar information, coordination difficulties may arise. For example, when interacting 

with the operator, greater opportunities for failures in communication may occur because other crew members do not 

understand the operator's frame of reference. If such individual consoles are used to execute control actions, error detection 

may be compromised because other crew members may not be able to observe those actions while performing their own. 

Thus, subtle mechanisms for maintaining awareness of control actions and detecting errors may be lost (Stubler and 

O'Hara, 1996b). A related example is decision aids that provide special information to an individual, but do not make it 

accessible to the crew. Consequently, the user may experience difficulty in communicating complicated concepts to the rest 

of the crew because members do not share a common view of plant conditions. 

The physical characteristics of HSI upgrades also can affect the crew's performance. For example, face-to-face 

communications may be impaired by factors that interfere with their line-of-sight, or new equipment may requires operators 

to spend more time away from the main control panel (e.g., monitoring back panels). Another characteristic of the HSI that 

may impair crew coordination is background noise that can affect the intelligibility of speech.

4-15 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: 

DESIGN PROCESS CONSIDERATIONS FOR HYBRID HSI 

NUREG-07 11 provides a general process for reviewing HSIs. However, when upgrades are made to an existing NPP, 

special considerations may arise. The purpose of this section is to identify issues of human performance associated with 

HFE reviews of upgrades, and to establish a technical basis upon which to develop guidelines. To prepare this section, it 

was necessary to gather and integrate information from many sources, including the nuclear power industry, other process 

industries, and commercial aviation, and from general literature addressing human performance in complex human

machine systems. This section also includes findings from visits to a broad range of facilities, which were in made 

conjunction with this and other research projects. 

Human performance issues are presented in sections organized according to the four phases of NUREG-07 11: Section 5.1 

addresses the planning phase, Section 5.2, the analysis phase, Section 5.3, the interface design phase, and Section 5.4, the 

evaluation phase. Specific NUREG-07 11 elements are described within each of these phases. Finally, Section 5.5 discusses 

the implementation of upgrades, including considerations that may affect personnel's acceptance of them, and the 

successful integration of upgrades into existing work environments. 

5.1 Planning Phase 

5.1.1 General Considerations 

The overall goal of an HFE review should be to ensure that the development of the upgrade is consistent with good HFE 

principles of design. Such reviews are necessary if the upgrade (1) affects the role of personnel or the tasks by which they 

perform their role, and (2) is potentially significant to plant safety. Upgrades affect the role or tasks of plant personnel if 

they impose new or different demands on them for operation, maintenance, or activities related to plant safety. Determining 

the safety significance of upgrades requires considering how safety may be affected by them. The following are three 

approaches derived from existing NRC review criteria. Others are also possible. 

The first approach for assessing the potential safety significance of an upgrade is based on the criteria for unreviewed safety 

questions (USQ). A proposed change to a NPP is considered to involve a USQ if it satisfies any of the three criteria of 

10 CFR 50.59. The Electric Power Research Institute (EPRI) produced the Guideline on Licensing Digital Upgrades, EPRI 

TR-102348, (EPRI, 1993) to assist licensees in licensing digital upgrades using these criteria. This document was favorably 

reviewed with certain specific interpretations by the NRC in Generic Letter 95-02 (NRC, 1995d). EPRI TR-102348 restates 

the three criteria of 10 CFR 50.59 as seven questions, presented in Table 5.1; an affirmative answer to any one of them 

indicates that a USQ exists. An affirmative answer is not a determination of safety significance, but merely an indication 

that there is a question related to safety which has not been reviewed by the NRC. These seven questions may be adopted to 

assess the safety of upgrades. An example of an evaluation method based on these questions is given by Stubler, et al. 

(1996). 

A second approach for assessing the potential safety significance of an upgrade is to base the evaluation on criteria from the 

Maintenance Rule, NRC Inspection Procedure 62706 (NRC, 1995e), which imposes special requirements for monitoring the 

performance or condition of structures, systems, and components (SSC) that may be considered significant to safety. In the 

simplified flowchart of the Maintenance Rule, the following criteria are set out for determining whether a structure, system, 

or component falls within its scope. An SSC is considered safety significant if at least one of the following questions is 

answered affirmatively: 

Is the SSC safety related? 

Does the non-safety related SSC mitigate accidents or transients?

5-1 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Table 5.1 Primary USQ Analysis Questions from EPRI TR-102348

1. May the proposed activity increase the probability of occurrence of an accident evaluated previously in 

the Safety Analysis Report (SAR)? 

2. May the proposed activity increase the consequences of an accident evaluated previously in the SAR? 

3. May the proposed activity increase the probability of occurrence of a malfunction of equipment important 

to safety evaluated previously in the SAR? 

4. May the proposed activity increase the consequences of a malfunction of equipment important to safety 

evaluated previously in the SAR? 

5. May the proposed activity create the possibility of an accident of a different type than any evaluated 

previously in the SAR? 

6. May the proposed activity create the possibility of a malfunction of equipment important to safety when 

the malfunction is of a different type than any evaluated previously in the SAR? 

7. Does the proposed activity reduce the margin of safety as defined in the basis for any technical 

specification? 

* Is the non-safety related SSC used in emergency operating procedures? 

* Could the non-safety related SSC prevent a safety related SSC from fulfilling its function? 

* Could the non-safety related SSC cause a scram or safety system actuation? 

A third approach follows the NRC's recent recommendations in Regulatory Guide 1.174 for using risk information in 

evaluating design changes to a plant's licensing basis (NRC, 1998). The basic method is as follows: 

* Define the proposed change including the aspects of the plant's licensing basis affected 

* Perform engineering analysis to justify the proposed change, including 

- evaluation of defense-in-depth and safety margins, 

- evaluation of the risk impact of the modification using a suitable PRA, 

- comparison of the PRA result with acceptance guidelines, and 

- integrated decision making to arrive at a conclusion about the acceptability of the proposed modification 

after considering all information

NUREG/CR-6637 5-2

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Define an implementation and monitoring program to ensure that there is no adverse impact on safety once the 

changes are in place. 

Thus, an upgrade may be considered potentially safety significant if it constitutes an unreviewed safety question, affects an 

SSC that satisfies one of the criteria of the Maintenance Rule, or satisfies the criteria of Reg. Guide 1.174. 

5.1.2 HFE Program Management 

The planning phase contains one NUREG-0711 element, HFE Program Management. This review should be conducted, in 

accordance with Element 1 in Section 2, for upgrades that affect the role of personnel or the tasks by which their role is 

performed, and are potentially significant to plant safety. For such upgrades, the licensee should prepare an HFE program, 

which is a technical plan describing how HFE considerations will be addressed in designing, developing, and implementing 

the upgrade. The goal of this review is to assess the acceptability of that plan. 

Section 2.4 ofNUREG-0711 has criteria for reviewing an HFE program. The following discusses topics that are especially 

important for reviewing upgrades: 

HFE program's scope 

HFE program's goals 

HFE team and organization 

HFE process and procedures 

HFE issues tracking 

HFE Program Scope 

When defining the program scope, the factors to consider are the type of upgrade and the characteristics of the 

implementation process. 

Type of Upgrade - Section 2.4.1 ofNUREG-071 I specifies a broad range of plant facilities (Criterion 3) and all HSI 

components used for operations, accident management, maintenance, test, inspection, and surveillance (Criterion 4). For 

an upgrade, this scope may be limited to the HSI components included in the upgrade, and to components of the existing 

HSI that interact with them. For example, if the upgraded controls and displays will be used with existing ones, the 

program should assess their consistency. In addition, the scope should include upgrades to plant systems. For example, an 

upgrade may entail modifying a system but not the HSI. In this case, the HFE program should ensure that the existing HSI 

is compatible with the upgraded plant system. If the latter imposes new control and display requirements, then the HFE 

program should ensure that they are identified and dealt with by HSI design, procedures, and training. 

Characteristics of the Implementation Process - The licensee may not implement an entire upgrade all at once; instead, 

portions may be installed in stages. For example, if several systems are to be upgraded, the work may extend over several 

plant outages, with different systems being implemented in each. In addition, the old equipment may not be removed when 

the new equipment is installed, creating a situation in which the composition of the HSI changes over time. That is, the 

composition that exists during intermediate periods differs from both the initial design and the final intended design. In 

5-3 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

addition, temporary "fixes" during these intermediate periods may support the operation of the HSI and plant systems. 

When planning an HFE review, it is important to address these interim configurations because they may impose special 

demands on human performance that are different from those of the original and the final intended HSI configuration. 

The following provides four examples of implementing upgrades, based on the considerations discussed. For each, the 

upgrade may consist of a small number of HSI components, or the entire HSI: 

Complete replacement - The old HSI component(s) are removed and completely replaced with new ones. This is 

accomplished all at one time, as during a single outage. Personnel must adapt to a sudden, complete change. 

Phased implementation - The HSI components are gradually removed and replaced at intervals until all the new 

equipment has been installed. For example, one petrochemical plant installed a new CR, consisting of three 

separate operator consoles, in a separate location near the old CR. The transition was accomplished in three steps. 

First, one of the three consoles was installed in the new CR, and the other two new consoles in the old CR. 

Operators in the new CR coordinated their tasks with operators in the old CR via telephone. Operators controlled 

the plant using both the old and new instrumentation. Second, all of the new consoles were installed in the new 

CR and operations were conducted from the new location. Third, after the new CR had operated for three months, 

some advanced control capabilities that the old HSI design lacked were added to the new one. This completed the 

gradual installation process (O'Hara, Stubler, and Higgins, 1996). However, personnel had to adapt to a 

continually changing work environment, including temporary configurations that may differ from both the original 

and the final configurations. 

Dual installation with delayed switchover - New equipment may be installed in the HSI while the old HSI 

equipment is still present. At some point, a switchover occurs, the new equipment is put into service, and the old 

equipment is taken out-of-service. The old equipment may be removed, or disconnected and tagged out-of-service 

(e.g., abandoned in place). This approach may be desirable in requiring a new system to pass a trial period before 

plant personnel are confident in its operation. For example, Higgins and Pope (1996) describe the switchover from 

an old set of display devices to a newly installed cathode ray tube (CRT)-based display system in the CR of a U.S. 

government facility. Before the switchover, the new display system was driven by a computer-based simulator. 

This gave the operators an opportunity to learn about, and use, the new display system in their spare time, while 

the facility was being operated from the old HSI components. At some point, the new display system was 

disconnected from the simulation, and linked directly to the plant. Here, personnel must adapt to a continually 

changing work environment. They must identify which HSI components are functional, including display devices 

that may be driven by a simulation, rather than by the actual plant. 

Dual installation with no switchover - In some cases, the new equipment is redundant with the existing equipment; 

accordingly, both old and new equipment are operational and personnel may use either one. This approach may be 

more common for display systems (e.g., when a computer-based display system is installed to augment the existing 

displays). Dual installation may be less common for control system upgrades because of practical considerations 

involving redundant controllers (e.g., preventing conflicts due to simultaneous use). Dual installation can allow 

operators to revert to the old equipment if they encounter problems with the new equipment. In this case, 

personnel must adapt to an HSI that is somewhat more complex because it has different HSI components that 

perform similar functions but may have different methods of operation. 

The demands imposed on personnel for adapting to the changes in the HSI are different for each of these examples, and 

should be considered when reviewing an HFE upgrade. 

NUREG/CR-6637 5-4

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

HFE Program Goals 

Criterion I of Section 2.4.1 ofNUREG-0711 describes-the general goals that should be reflected in an HFE program. For 

an upgrade, the introduction of changes to the plant will require modifying and adapting existing work processes and 

practices. The way in which upgrades are implemented may impose special demands on personnel, and the HFE program goals should consider such possible effects. The transition from the existing plant configuration to the upgraded 

configuration should be planned so that minimal demands are imposed on personnel adapting to the change. The 

considerations should include the following: 

Planning the installation process to minimize disruptions to ongoing activities, 

Coordinating training and procedure modification with the implementation of the upgrade to ensure integration, 

and 

Conducting training to maximize operators' knowledge and skill with the new design before using it. 

HFE Team and Organization 

Section 2.4.2 ofNUREG-071 I gives criteria for the responsibility, authority, composition, and staffing of the licensee's 

HFE team. In an upgrade, personnel who will be the ultimate users of the upgrade can be valuable sources of domain

specific information that should be considered when designing, developing, and evaluating upgrades. Their involvement 

should ensure that the following points are considered: 

Knowledge of task demands and constraints of the work environment 

Knowledge of work processes 

Knowledge of organizational goals that affect the use of the upgrade. 

The involvement of workers in the development process also can also help ensure that the final design of the upgrade is 

consistent with organizational concerns such as reliability, maintainability, and supportability (Medsker and Campion, 1997; Price, 1990). For example, Roth and O'Hara (1998) reviewed a new computer-based display system in a NPP. They found that although operations personnel had been actively involved in developing the system, the plant's management believed that their greater involvement would have been beneficial. (Additional discussion of this topic is given in Section 

5.5, Organizational Considerations for Designing and Implementing HSI Upgrades.) 

HFE Process and Procedures 

Section 2.4.3 of NUREG-07 11 provides a criterion for ensuring that vendors and subcontractors address HFE in developing upgrades. The HFE program should ensure that the HFE requirements are specified in agreements with the vendors and 

contractors. This criterion is especially important for upgrades because add-ons or modifications to the HSI and plant 

systems are often purchased as modular units designed to the standard specifications of the vendor, rather than to a 

particular plant's specifications.

5-5 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS -

HFE Issues Tracking 

Section 2.4.4 ofNUREG-071 1 provides criteria for ensuring that the licensee has established a means for documenting and 

tracking HFE issues that arise during the design process. It states that HFE issues should be tracked from identification 

until elimination or reduction to an acceptable level. However, some HFE issues may not be detected during HFE 

verification and validation, but may become apparent during plant operation. For example, when setting up a computer

based display, management decided that it would be beneficial to establish a systematic formal approach for recording 

problems and suggestions arising during final adjustments of the system (Roth and O'Hara, 1998). HFE considerations 

identified in this way should also be recorded in an HFE issues tracking system; they should be documented and tracked as 

described in Section 2.4.4 ofNUREG-071 1. 

5.2 Analysis Phase 

The analysis phase identifies the requirements for new designs. NUREG-07 11 recommends a top-down approach which 

includes identifying functions, tasks, staffing, and human-reliability considerations. In developing an HSI upgrade, the 

focus should be on characteristics of the new design that differ from the predecessor design, and their potential effects on 

people's performance. The extent of these analyses should be commensurate with the scope of the upgrade and the potential 

consequences to performance and plant safety. This section describes human performance considerations and methods for 

analyzing upgrades, and addresses the following considerations: 

Functional requirements 

Task requirements 

Cognitive task analysis 

Crew performance and staffing 

Human reliability and design bases 

5.2.1 Functional Requirements Analysis and Function Allocation 

Element 3 in Section 4 ofNUREG-071 1 describes two analyses: a functional requirements analysis and a function 

allocation analysis. The purpose of the former is to ensure that the plant's functional requirements have been defined. 

Functional requirements analyses should be reviewed, in accordance with Section 4 of NUREG-07 11, for all upgrades that 

are safety significant, that affect the role of personnel, and are likely to change existing functions that are important to 

safety, introduce new functions, or involve requirements that are not clearly defined and may be important to safety. The 

scope of the analyses ma; be restricted to functions related to the upgrade. The purpose of function allocation analysis is to 

ensure that functions are allocated to take advantage of human strengths, and to avoid allocating them to personnel if they 

would be negatively affected by human limitations. Function allocation analyses should be reviewed, in accordance with 

Section 4 of NUREG-07 11, for all upgrades that are safety significant, that affect the role of personnel, and are likely to 

change the allocation between personnel and plant systems of functions important to safety. The scope of the analyses may 

be restricted to functions related to the upgrade. 

Changes in plant systems may change the way existing functions are carried out. For example, hydrogen ignitors could be 

an alternative way to eliminate this gas from the containment. As another example, some upgrades of digital feedwater 

NUREG/CR-6637 5-6

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

systems for PWRs have an automatic switch-over.from the auxiliary to the main feedwater systems during startup. This 

provides continuous automatic control of feedwater over a broad range of reactor power conditions, whereas the original 

configuration may have consisted of separate control systems for the auxiliary and main feedwater systems. Thus, the 

function of switching from one system to another still exists with the new system, but it is performed differently. As a third 

example, an upgrade may change the behavior of a system performing a function, so that it may be performed differently 

(e.g., when digital control systems are installed, the systems often respond more quickly to changes in control setpoints and 

exhibit less drift from the setpoint value). 

These changes should be identified in accordance with NUREG-0711 and NUREG-0700, and indications that the function is 

available, operating, and achieving its purpose should be described. 

Plant upgrades also may affect the allocation of new and existing functions to personnel. Automation of plant systems has 

resulted in a trend away from manual operation, and concomitantly, in more complex automated control modes. 

Accordingly, the operator's role is shifting away from directly controlling plant systems and toward monitoring automated 

systems and intervening when they are not functioning properly (O'Hara, Stubler, and Higgins, 1996). Analyses should 

ensure that the functions allocated to personnel is within their capabilities. An important consideration is the ability of 

personnel to understand the structure and function of these automated systems, so they can detect and respond to 

malfunctions. This understanding is affected both by the design of these systems and the characteristics of the HSI, which 

conveys the information. 

5.2.2 Task Analysis 

Task analysis, as described by Element 4 in Section 5 of NUREG-07 11, is a detailed evaluation of the tasks that must be 

performed to accomplish the allocated functions. It establishes the performance requirements for personnel, and provides a 

basis for HSI design requirements. Task analyses should ensure that tasks resulting from upgrades are within the. 

capabilities of the affected personnel. Task analysis should be reviewed, in accordance with Section 5 ofNUREG-071 1, for 

all upgrades that are safety significant, that affect the role of personnel, and are likely to affect tasks previously identified as 

risk important, cause existing ones to become risk important, or create new risk-important tasks. The scope of the analyses 

should include tasks involving the upgrade and its interactions with the rest of the plant. For maintenance, test, inspection, 

and surveillance, attention should be given to new risk-important tasks and those supported by new technologies (e.g., new 

capabilities for on-line maintenance). 

As described in Section 5.2.1, upgrades to plant systems or the HSI may change the functions and hence the tasks 

performed by personnel. For example, introducing a new automated system may create a new function in which the 

operator must monitor the system and intervene if it malfunctions. Tasks that result from the new or changed personnel 

functions should be the starting point for the task analysis. Cognitive requirements for these tasks may be described in 

terms of the generic cognitive tasks (i.e., monitoring and detection, situation assessment, response planning, and response 

implementation) described in Section 4. In addition, people's tasks may be changed by the upgrade even if their functions 

are not. That is, the same functions may be performed in different ways when a new HSI component is used; therefore, the 

task analysis should also consider such changes. 

The following describes the specific characteristics of tasks that may be affected by upgrades and, therefore, should be 

addressed by the task analysis. First, the task's characteristics are described at the level of the individual. Next, those that 

affect the crew's performance are discussed. Finally, we discuss task analysis methods specifically focused on assessing cognitive demands. These methods may be useful for analyzing knowledge and cognitive skills associated with using an 

upgrade.

5-7 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Changes in Individual Crew Members' Tasks 

Human performance is affected by the limitations of working memory and attention by restricting the amount of 

information that can be processed at any given time. Experienced users compensate by employing specific cognitive skills 

and strategies which are tailored to the task domain, and tailored after the types of information handled, and its sources. 

When systems develop over a long period, as with conventional HSI designs, a natural interactive process may develop 

between users and designers (Rasmussen and Goodstein, 1988). For example, users may develop skills for coping with the 

initial design of the system, and they may become part of a formal or informal training program. Designers recognize these 

skills and incorporate features that utilize these skills in the next version of the product. The users then refine and adapt 

these skills to the new design, which designers recognize and accommodate in subsequent designs. Over time, these design 

characteristics become part of the craft of designing the product or system, and the explicit rationale for why certain HSI 

components have particular characteristics may be lost. Rasmussen and Goodstein (1988) describe this phase as stable 

periods characterized by "horizontal" design improvements. New components or user requirements are incorporated within 

an established framework of which the conceptual basis and "...original reasons can no longer be explicitly formulated" 

(Rasmussen and Goodstein, 1988, p. 179). 

However, when new HSI technologies are introduced, such as computer-based display systems, the features of the HSI may 

change radically from the traditional design. That is, "...the introduction of new technology, as illustrated by the on-march 

of advanced information systems destroys the blanket of established know-how and practices" (Rasmussen and Goodstein, 

1988, p. 179). Consequently, the strategies developed by experienced users, based on the old design, may no longer be 

effective. 

An early example occurred in the nuclear industry when tile-based annunicators were replaced by alarn-list displays. The 

designers considered the alarm system from a fairly narrow perspective, focusing on its function as an automated 

monitoring system which alerted operators to off-normal parameters. It was subsequently discovered that operators use the 

alarm system's information in many ways which may not be obviously based on the original intent of the design. For 

example, the annunciator tiles were used as an overview display to support response planning (O'Hara and Brown, 1994; 

O'Hara, Brown, Higgins, and Stubler, 1994; Pew, et al., 1981). Operators learned the locations of annunciator tiles, and 

could quickly assess the plant's overall condition from the pattern of illuminated ones. The presentation of alarm 

information in the alarm-list displays lacked spatial dedication and permanence (i.e., the alarm messages scrolled off the 

display page). As a result, the workload associated with these list displays increased greatly and they were found to be 

inadequate replacements for the tiles. 

Another example is related to the current trend in the design of new control rooms toward compact, computer-based control 

panels. NPP CRs have traditionally featured broad panels of individual, spatially dedicated controls and displays. While 

this configuration posed many demands on operators (e.g., in integrating information across broadly distributed 

instruments), it also had benefits for coordinating crew activities. Operators shared the same view of plant instruments and 

could monitor each other's performance and detect errors fairly easily. Some of these benefits may be lost in compact CRs 

where operators have separate consoles with individual views of the plant that cannot be readily shared with other operators 

(Stubler and O'Hara, 1996b). 

Thus, while traditional HSI designs may have many limitations, they may support skilled performance in ways that are not 

immediately obvious, and also support functions or tasks not explicitly stated in the design requirements. These functions 

and tasks may have evolved over time from user's strategies for adapting to the specific features of an established design. 

When introducing new technologies, it is important to investigate the skills used previously to discover these functions, 

tasks, and strategies; this may require techniques specifically focused on the skills and strategies that define user expertise 

NUREG/CR-6637 5-8

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

in the old configuration. (See discussion of cognitive task analysis below.) If these user's needs are not identified during 

task analysis and considered in the design activity, the modification may fail as a replacement for the current design. 

When analyzing tasks, attention should be given to the strategies used for performing them, the information obtained, and 

the HSI characteristics that support these strategies. The following describes three types of strategies: task automaticity, 

information chunking, and HSI tailoring. 

Performing Tasks Without Highly Focused Attention - Automaticity, performing actions without committing a high degree 

of conscious mental resources, allows personnel to free-up cognitive resources for other tasks. Automaticity is a mark of 

proficient users. However, it may not be achieved if the HSI does not minimize demands on the user. Task analysis should 

address automaticity in two ways. It should identify (1) actions that would benefit from automaticity, and (2) actions for 

which automaticity should be discouraged. 

According to Schlager et al. (1989), actions may be candidates for automaticity if they: 

Are associated with a discernable event, 

Are performed the same way each time, 

Do not require a high degree of attention (e.g., do not involve a high degree of focused attention in planning or 

executing a response), and 

Are not interrupted by unique actions. 

The discernable event is a stimulus that triggers the response, which may consist of a single action, or a pattern of actions in 

a specific sequence. Actions performed the same way every time a stimulus situation occurs are called consistent task 

components because the stimulus and response are consistently mapped (Scheider, 1985). Variable task components are 

actions that are not carried out identically each time and, consequently, are sensitive to the availability of cognitive 

resources. Consistent task components show large improvements in performance with practice, while variable task 

components show little or none (Schneider, 1985). 

Some activities that satisfy these criteria may include routine control actions and interface management (e.g., accessing 

displays from a display system). Personnel normally are expected to do them without focusing on the details of their 

execution. Candidates for automaticity should be identified during task analysis and included in designing the HSI and 

developing training programs (Mumaw and Gabrys, 1996; Schlager et al., 1989). 

Automaticity should be discouraged when accidental or incorrect operation of a system may have significant consequences 

for plant safety. As described in Section 4.3.4, slips are errors involving the incorrect execution of an intention. They are 

the undesirable consequence of automaticity because they occur when a pattern of actions intended to accomplish a 

particular intention gets waylaid en route to execution (Norman, 1983; Norman, 1988). A review of soft controls used in 

hybrid HSIs found that computer-based user interfaces are especially prone to some types of slips (Stubler, O'Hara, and 

Kramer, 2000); users may operate the wrong control, or carry out the wrong action on the right control. A careful task 

analysis should identify tasks for which automaticity is not desirable, including tasks for which the consequences of errors 

are high, such as those deemed risk important by an HRA, and others that affect the operation of safety systems. Some 

examples include initiating or terminating operation of a plant protection system, entry of data that would affect the 

operation of a plant protection system, and any other control actions, which if performed incorrectly, may initiate a 

transient. They should be addressed in the HSI design process by applying approaches to prevent errors.

5-9 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Information Chunking - Experienced users sometimes handle large volumes of data by using the cognitive skill of 

information chunking, organizing individual bits of information into conceptually related groups. Schlager et al., (1989) 

describe strategies used by experienced air traffic controllers allowing them to handle large volumes of traffic in the air 

sector they control. Thus, the pattern of many aircraft trajectories may be organized into more manageable groups by 

considering those traveling inbound and those traveling outbound, which require different responses from the controller. 

These groups may be further organized into subgroups based on distance, elevation, and aircraft type. 

Users often rely on the characteristics of the HSI to formulate and use information-chunking strategies. The air traffic 

controllers may rely on display icons to distinguish between inbound and outbound aircraft, or they may access displays that 

present them separately. For NPP operations, operators may rely on display characteristics, such as spatial arrangement and 

visual coding (e.g. color, flashing). Different strategies may be employed depending upon how the information is depicted 

(e.g., alphabetically, chronologically, or by function), and users develop skills based on such presentations. 

HSI Tailoring - HFE design principles dictate that the HSI should be designed to be compatible with human capabilities 

and limitations. However, matching the HSI to human capabilities may continue after it is installed. Users may tailor the 

HSI to their specific tasks or personal characteristics. This may be done by augmenting the information provided by the 

user interface with additional data or by manipulating HSI features to draw attention to particular information. One 

strategy is to use reminders which help the user maintain important information in memory, or assist in accessing it later. 

Vicente, Mumaw, and Roth (1997) describe several methods used by CR operators., In one case, operators wanted to focus 

on monitoring a particular variable shown on a strip-chart recorder. Since there were no special mechanisms for drawing 

attention to the recorder or for reminding them that the variable should be watched more often, the operators developed a 

practice of sliding the recorder forward from its normal position in the control panel; its forward projection accomplished 

these aims. 

Also, Vicente et al. noted that operators attached small notes to control and display devices on the control panel to draw 

attention to the device or to record important information, such as the previous values of a closely watched variable or a 

reference value; this reminded them to act when a particular value is reached. (While this may not be a recommended 

practice, it is given as an example of users modifying the HSI to cope with demands unrecognized when the HSI was 

designed.) 

Another example by Schlager et al. (1989) describes how air traffic controllers manipulate their flight strips. Flight strips 

are labels describing aircraft in a particular air sector. They are normally attached to plastic strips and stacked in racks at a 

controller's console so they may be easily viewed. One strategy controllers use to remind them of aircraft requiring special 

attention is to "cock" the flight strip for that aircraft. Alternatively, controllers may mark the strip with large dark letters to 

draw attention to particular details. 

The user's modifications of the HSI are important for two reasons. First, the fact that the operators find the modifications 

necessary suggests that HSI designs may not fully meet the task requirements (i.e., the original design has limitations). 

When developing an upgrade or replacement system, the users' modifications should be studied through task analyses to 

determine how and when they are used to accomplish work goals. The functions and tasks performed using these 

modifications should be included in the design requirements for the upgrade. Second, these modifications illustrate 

practical solutions for particular needs. If this function is needed in the new design, then the design should include some 

mechanism allowing the operators to accomplish the same result. Ideally, that new mechanism should be consistent with 

the method used in the old design, so operators do not have to learn new skills. 

Using the flight strips as an example, an upgrade could replace the plastic flight strips with labels on a CRT. The air traffic 

controllers' strategies of cocking and marking the plastic strips should be analyzed to determine when they are used and 

NUREG/CR-6637 5 - 10

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

what information they impart. Features should be included in the new design that convey the same information provided by cocking and marking the strips. Training requirements may be reduced to the degree that the new methods are similar to 

those of the original design. 

It is important to ensure that skills and strategies developed by users for the original design are either supported by the new 

design or no longer needed due to improved work methods. Situations should be avoided in which the new work methods 

are not a significant improvement, and the old strategies are not applicable to the new design. In such cases, the end result 

may be that workload is increased rather than decreased by the new design. 

Changes in Crew Tasks 

When analyzing a task, it is important to consider the distribution of demands among personnel. Additional 

communications and coordination may be necessary to facilitate new demands associated with monitoring and detection, 

situation assessment, response planning, and response implementation. Section 4.4.2 described the crew's performance and 

team skills within the context of these four generic cognitive tasks. Important factors included a shared context (e.g., 

common understanding of the plant's state) between the speaker and the listener to correctly interpret intended meanings, the ability of decision makers to communicate the intermediate results of assessment and planning, and the ability of other 

personnel to gather and evaluate information in support of the primary decision maker. The characteristics of the work 

environment (e.g., horizon of observation, openness of tools, and of interactions) can help personnel in communicating and 

in coordinating work. The task analysis should identify any changes in such demands and also identify design characteristics that support them in the current work environment but which may not exist in the same form in the upgraded HSI. These factors should be highlighted to support evaluations of staffing, HSI design, procedures, and training in later 

stages of the HFE review. 

Methods of Cognitive Task Analysis 

Conventional task analyses tend to focus on explicit and objective knowledge, such as actions that can be easily observed or 

described in procedures. While such analyses are necessary for defining human performance for routine work, they may be 

insufficient for situations in which high demands are imposed on cognitive capabilities (Klein, Calderwood, and 

MacGregor, 1989). These analyses may need to be supplemented with techniques specifically focusing on how personnel 

access and process information in the work environment. Cognitive Task Analysis (CTA) methods have grown out of the 

need to understand the information processing demands imposed by working in complex human-machine systems. The 

methods can identify the range of problems that personnel encounter in a domain, define the dimensions of task complexity, and define the requirements for information and problem solving. CTA is similar to standard task analysis in its concern 

for inputs, processing, and outputs. However, its focus is on cognitive operations undertaken to acquire and transform the 

information for executing an action or making a decision (Sanquist, Lee, and McCallum, 1995). 

CTA may be applied to standard function and task analyses to examine them in greater detail. For example, for functions 

and tasks particularly important to the role of the operator, CTA can identify the individual cognitive transactions necessary 

to extract and process specific data. It can be useful for examining the role of different technologies in operators' cognitive tasks. For example, it may be used to define the effects of automation on a person's decision-making role or the effects of 

information technologies in supporting cognitive functions associated with specific tasks. The findings can guide the 

development of design requirements for upgrades, or establish requirements for training. 

Roth and Mumaw (1995) identify three categories of CTA: function-based, empirical, and cognitive simulation. They note 

that most CTA methods incorporate features from more than one of them. Table 5.2 lists some important characteristics of 

these approaches.

5-11 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Table 5.2 Comparison of Three Categories of Cognitive Task Analysis (CTA)

CTA System Being Studied Cognitive Factors Analyst's Activities Analyst's 

Category S T I Expertise 

Empirical Existing (actual or Specific knowledge and Conducting Cognition 

simulator) skills used in actual observations, interviews 

practice and walk-throughs with 

subject-matter experts 

Factors that distinguish 

levels of expertise 

Function- Existing or future Information Analyzing information Systems 

based requirements for specific requirements engineering 

decisions 

Cognitive Existing or future Human response time Eliciting expert Cognition and 

Simulation and errors knowledge and software 

representing it in development 

Consequences of errors software 

Empirical CTA - These approaches rely on empirical techniques to analyze how people perform tasks in either real or 

simulated environments. Their focus is to identify the knowledge and skills used by practitioners in a specific domain 

(Klein, Calderwood, and MacGregor, 1989; Roth and Woods, 1988). They are particularly well suited for identifying 

factors that account for performance differences between experts and less-experienced practitioners. Empirical CTA 

techniques require the analyst to have a background in HFE with an understanding of cognition. These techniques may 

employ a variety of collection methods to obtain data about the skills and strategies used by working personnel. For 

example, walk-through exercises may be used in which personnel demonstrate their skills while using particular equipment. 

Carefully crafted probing questions may be used to focus on problem-solving strategies. 

Empirical CTA approaches are particularly appropriate for developing training programs for existing systems because they 

encompass the address dimensions of expertise (e.g., special knowledge and skills) required for the current design (Roth 

and Mumaw, 1995), including the user's strategies, such as automaticity, information chunking, and HSI tailoring. Also, in 

studying performance under defined event scenarios, the frequency of use of specific strategies and skills can be counted. 

These counts, together with users' assessments of the task's difficulty, may determine what should be addressed in the 

training program. 

Empirical CTA approaches also can be used in developing new HSI designs and upgrades. Frequency counts of various 

activities may serve as crude measures of workload when evaluating design alternatives. For example, an analysis may 

show that certain mental-processing operations can be eliminated when a decision-support aid is used (Sanquist, Lee, and 

McCallum, 1995). A CTA may reveal the skills and strategies personnel use and so identify the design characteristics of 

the HSI to support them. This analysis also may determine the relevance of these design characteristics and the associated 

user strategies for the new design. Empirical CTA approaches can assess the consistency of the HSI. If they indicate 

difficulties in using multiple HSI components for a particular task, this may suggest that the interaction methods across the 

HSI are inconsistent.

NUREG/CR-6637 5- 12

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

The Critical Decision Method (CDM) is a retrospective interview technique for studying the general knowledge, specific information, and reasoning processes used by personnel in environments characterized by high time pressure, high 

information content, and changing conditions (Klein, Calderwood, and MacGregor, 1989). In such environments, 

personnel often resort to recognition-primed decision making. That is, they select actions based on recognizing critical 

information and on prior knowledge rather than by engaging in deliberative evaluations of all possible courses of action. 

The CDM focuses on three aspects of expertise: explicit and objective knowledge, tacit knowledge, and perceptual knowledge (perceptual-motor feel). Explicit and objective knowledge is typically the focus of a traditional task analysis. 

The other two aspects represent categories of knowledge that are critical to skilled behavior but resistant to being articulated 

(e.g., the ability of experts to judge that a situation is familiar, without first analyzing it in detail). The CDM focuses on 

actual incidents considered non-routine in which expertise is most important, and cognitive skills can be most readily 

examined. 

The method consists of five steps: (1) selecting the incident to be explored, (2) obtaining an unstructured account of it based 

on the SME's recollection and verbal description, (3) constructing a time line for the incident, (4) identifying points at 

which important decisionswere made, and (5) probing the decision points. Probing is the step in which details are obtained 

about cognitive skills and knowledge. A standardized set of probes (focused questions) is used to explore such topics as 

cues used by the SME to assess the situation, prior knowledge and experience used, goals considered in planning responses, 

and options that existed at the decision points. 

One output of this method is a descriptive decision model, i.e., a model of important decisions made by the SMEs that 

identifies specific decisions and their characteristics. A second output is a critical cue inventory, a categorization of all 

informational and perceptual cues used in assessing a situation or considering options. Together they provide an 

understanding of user strategies for adapting to the current design and HSI characteristics that support these strategies. They may be used in establishing design requirements for an HSI upgrade or for training requirements. A third output is 

the identification of non-routine incidents. Because these are actual incidents that challenge the HSI design and personnel 

expertise, they form a basis for developing HFE validation scenarios. 

Function-Based CTA - Function-based CTA relies on a formal functional analysis of the applicable domain that is usually based on some form of goals-means functional decomposition (e.g., Rasmussen, 1986; Vicente and Rasmussen, 1992; Lind, 

1993, cited in Roth and Mumaw, 1995). It depicts goals and functions hierarchically. For each function, it defines higher level goals and lower-level functions (i.e., paths for accomplishing the goals). This analysis provides the underpinning for 

defining personnel tasks. Each function identified represents a decision point in the operator's role. For each function, a 

set of questions are posed, derived from an operator's decision-making model (Rasmussen, 1986). They investigate the 

information requirements and control capabilities for supporting the cognitive activities involved in ensuring the operation 

of that function. The questions address the following topics: monitoring, situation awareness, planning, and control. The 

answers define the HSI requirements that operators need to support these activities, as well as the rationale for the plant's 

parameter measurement and control requirements (provided to systems engineers), and the basis of design descriptions for 

displays. 

The heavy reliance of this approach upon functional analyses has advantages and disadvantages. A disadvantage is that 

such methods require more analysis than empirical CTA approaches. An advantage is that the methods can be employed for new systems with no close current analogies. Thus, function-based CTA methods can be used when empirical methods 

cannot. A second advantage is that function-based CTA uses very structured, pragmatic methods. It does not require 

expertise in behavioral science and can be undertaken by analysts with engineering backgrounds; this can be a benefit for 

engineering organizations developing advanced human-machine systems. Hence, Roth and Mumaw (1995) recommend 

function-based CTA for first-of-a-kind design projects.

5- 13 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Cognitive Simulation CTA - Cognitive simulation constructs a computer-based model that simulates the cognitive activities 

required to perform a task. The computer model explicitly describes knowledge requirements and decision processes 

(Zachary, Ryder, Ross, and Weiland, 1992, cited in Roth and Mumaw, 1995; Hollnagel, 1992; Roth, Woods, and Pople, 

1992). System design factors can be entered into the model, such as the type of information presented, its rate of 

presentation, and its reliability, so that when the simulation is run, the effects on cognitive activities (e.g., the amount of 

knowledge and processing) can be determined. Comparisons can be made for alternative designs, and changes in design 

can be entered and the operator's and system's response observed. In addition, simulations can be valuable research tools 

for studying human cognition. Insights can be obtained by revealing differences between human performance and 

performance predicted by the computer model. Cognitive simulation techniques require the analyst to have an 

understanding of cognition, so that relevant dimensions of human performance can be modeled. Representing information 

processing in a computer-based model requires a background in software development. 

The following describes the System Response Generator (SRG) (Hollnagel, 1992), a computer-based simulation tool 

developed to support system design. It illustrates some important components of computer-based simulations and their 

functions. SRG systematically examines the possible ways in which a scenario can develop as a result of an operator's 

interactions with the system. It is intended to "...clarify precisely how the configuration of the physical system and the 

[user] interface may sometimes transform human erroneous actions into accidents" (Hollnagel, 1992, p. 216). SRG has four 

modules: (1) the event driver, which triggers events and controls the progress of the simulation from break point to break 

point, (2) the process response generator, which simulates the response of the controlled system (e.g., an aircraft) as it reacts 

to events and control inputs from the operator, (3) the operator response generator, which simulates the perceptual and 

decision-making processes of the operator in response to the outputs of the process response generator, and (4) the response 

interpreter, which examines the output frori the operator response generator and determines the implications of the 

operator's actions for the event driver. Thus, the cognitive simulation (i.e., the operator response generator) is immersed in 

a larger simulation that includes the system being controlled and the environment. 

Cognitive simulations offer several benefits over empirical approaches (Hollnagel, 1992): 

Events or operator actions that do not lead to anticipated consequences may go unnoticed unless special efforts are 

undertaken to observe them. (Empirical techniques often rely on well-defined procedures for capturing data. 

When a system's or human's behavior goes outside the analyst's assumptions, capturing data can become difficult.) 

Extremely long observations may be needed to obtain reliable data on low-frequency events. (Although scenarios 

can be created that increase the likelihood of specific behaviors, they require time to set up and execute. 

Computer-based scenarios can be created and run faster than scenarios involving real people.) 

The scope for generalizing results is often limited. (Computer-based simulations test a broader range of 

conditions, thus increasing the generality of the results.) 

Incompatible classification schemes for tasks, actions, causal factors, and contributing factors may interfere with 

the transfer or translation of data to new applications. (Due to the complexity of dynamic human-machine 

systems, it may be difficult to apply classification schemes consistently within or across scenarios. Computer-based 

simulation gives greater control over causal, contributing, and response factors, allowing more consistent use of 

classification schemes.) 

The exact boundary between acceptable performance and failure may be difficult to define for non-proceduralized 

tasks. (Computer-based simulation can provide greater control over scenario variables, facilitating the 

specification of criteria for acceptable performance.) 

NUREGICR-6637 5 - 14

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Roth et al. (1992) describe two potential disadvantages of cognitive simulations: 

A cognitive simulation may be a continually evolving tool due to such factors as the breadth of cognitive activities 

in a particular domain, the expanding knowledge base in cognitive science, and pragmatic factors in large software 

development projects. Hence, it may be difficult to see a cognitive simulation as a finished system. 

Cognitive simulations provide examples of concepts about human cognition, but are not the concepts themselves. 

They should be seen as tools for modeling cognitive performance rather than the actual model. Much value can be 

obtained by comparing predictions with actual performance. 

Finally, the initial expense of developing a cognitive simulation may be high, so this approach may be more applicable for 

large design projects. However, once developed, these simulation tools can be used to evaluate very small to very large 

design changes. 

5.2.3 Staffing Analysis 

Staffing requirements should be reviewed, in accordance with Element 5 in Section 6 of NUREG-07 11, for upgrades that 

are potentially significant to plant safety, affect the role of the personnel or their tasks, and are likely to change staffing 

requirements. The scope of the staffing analysis should cover demands resulting from the upgrade and its interactions with 

the plant. 

Minimum staffing levels are initially determined during the plant's design, often in conjunction with the CR design. 

However, a licensee may establish policies for higher staffing levels based on special operational activities (e.g., operational 

maneuvers; on-line testing) or other conditions that may impose unusual burdens on CR personnel (e.g., lack of some HSI 

components). Most plant and HSI upgrades are intended to reduce demands on operators. However, changes in the plant 

or the HSI can increase demands on personnel, especially during transients. If these demands are sufficiently high, the 

adequacy of CR staffing should be considered. Upgrades could affect the number of personnel, and also their required 

expertise. 

Number of Personnel 

The number of personnel needed in the CR may be affected by factors that increase workload or compromise the crew's 

performance. For example, the operators' roles may be changed if they must perform new or changed functions. Then, the 

assumptions underlying the original staffing levels may no longer be valid, and more people may be needed. 

Even if the operator's functions remain unchanged, changes in their tasks may alter the required staffing levels. Additional 

personnel may be needed if the frequency or difficulty of the tasks increase, such that they cannot be performed effectively 

by the current staff. This may occur when an HSI upgrade increases the amount of information to be monitored, the 

number of controls to be operated, the frequency of needed monitoring or control actions, or decreases the accessibility of 

information and controls. It may also occur when mental workload increases, such as when a modification to a system 

increases the plant's complexity and increases demands for situation awareness and response planning. 

Staffing requirements may also be affected by upgrades affecting the coordination of activities between personnel; examples 

include those that 

5-15 NUREGICR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Decrease the availability of operators, as when the upgrade increases the demands for operators to perform tasks 

outside the CR (e.g., fire brigade) or away from its main control area (e.g., tasks at the back panels). 

Increase the demands for operators to coordinate work activities (e.g., changes that increase the need for personnel 

to gather and coordinate information and control actions). 

Decrease the ability of personnel to coordinate work activities in the CR (e.g., changes such as background noise 

that interfere with speech, or interfere with people's line-of-sight, such as layout changes or installing equipment 

that obscures vision). 

Backgrounds of Personnel 

The required backgrounds of staff may be affected by any change that requires them to have different knowledge, skills, or 

abilities than those personnel currently possess. For example, an upgrade may require balance-of-plant operators to have a 

more advanced understanding of thermodynamics than they currently have. Also, the design of the HSI may require CR 

operators to have special skills in human-computer interactions. 

NUREG-07 11 provides criteria for analyzing staffing. It states that the analysis should be iterative. Initial staffing goals 

should be reviewed and modified as analyses associated with other NUREG-07 11 elements are completed. 

5.2.4 Human Reliability and Design Basis Considerations 

The purpose of human reliability analysis (HRA) is to evaluate the potential for, and mechanisms of, human error that may 

affect plant safety. It is an essential element in achieving the HFE design goal of providing operator interfaces that will 

minimize operators' errors, and will allow their detection and mitigation. HRAs focus on risk-important human actions, 

those actions which may challenge plant safety if performed incorrectly. The HRA review should be conducted in 

accordance with Element 6 in Section 7 ofNUREG-071 1, for all upgrades that are safety significant, affect the role of 

personnel, and may affect tasks previously identified as risk important, cause existing tasks to become risk important, or 

create new risk-important tasks. The scope of the human reliability analysis should address personnel actions resulting 

from the upgrade and its interactions with the rest of the plant. 

HRAs are based on many considerations, such as plant failure modes and the role of personnel in recovering from failures. 

For example, as part of the defense-in-depth design philosophy of NPP design, operators are credited with performing 

certain acts, such as initiating the operation of plant-protection systems when they fail to operate automatically. HRAs 

should be based on an understanding of the causes and modes of human error, such as those described in this report; careful 

analysis is especially important for actions determined to be risk important in the HRA. 

When upgrading systems or the HSI, the potential effects of these modifications on the assumptions underlying the existing 

HRA should be considered. For example, HRAs often assume that operators can detect a fault and initiate a manual 

response within a specific time. Changes to the plant's systems may change the bases of these assumptions. Some may 

simplify the roles of personnel, making the plant safer; others may complicate the operator's role. Thus, as plant systems 

become more complex, as through increased automation, the operator's role in detecting malfunctions may become more 

challenging; that is, it may be more difficult to determine that the system is operating incorrectly. Also, the actions 

required to respond to these conditions may be more complex, introducing new opportunities for error. Stubler, Higgins, 

and Kramer (2000) describe events that occurred in NPPs due to errors by operators and maintenance personnel while 

performing at-power maintenance on digital systems. 

NUREG/CR-6637 5- 16

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Changes in the HSI also may affect the HRA assumptions about personnel performance. For example, an HRA may take 

credit for a particular manual action. If the relevant controls and displays are upgraded with computer-based HSI 

technologies, these tasks may be altered. The computer-based HSI may lead to errors in navigating a display system to find 

the correct controls and displays. The execution of control actions using soft controls also may be prone to special types of 

errors (Stubler, O'Hara, and Kramer, 2000). These factors decrease the probability that the required action will be 

completed correctly within the required time limit. 

Risk-important actions should be explored when examining the effects of a plant system or HSI upgrade on the HRA. The 

following questions should be considered: 

Are the HRA assumptions valid for the upgraded design? 

Will changes in plant systems or the HSI increase the likelihood of mistakes (e.g., errors from inappropriate plans 

based on incorrectly assessing the plant's state)? 

Will changes in the HSI increase the likelihood of slips (e.g., errors in executing actions)? 

Will the consequences of these errors exceed the bounds established in the HRA? 

5.2.5 Summary of Analysis Phase 

The analysis phase addresses human performance considerations associated with identifying functions, tasks, staffing, crew 

coordination considerations, and human reliability factors. The analysis of these topics should be commensurate with the 

scope of the upgrade and its potential consequences to personnel performance and plant safety. Functional requirements 

should be examined to identify and evaluate changes in plant safety functions that may occur from the upgrade. They 

should identify functions that are new or changed. Also, the allocation of functions between personnel and automation 

should be evaluated. These changes in functions and allocations partly establish a basis for new personnel tasks. 

Task analysis looks at the specific characteristics of personnel tasks to establish a basis for design requirements for 

upgrades. Highly proficient users often employ special strategies to cope with work demands. This section described three 

categories of strategies: task automaticity, information chunking, and HSI tailoring, all of which should be identified and 

analyzed in the current work environment. The new design should either have features to support -these strategies, or 

should present control and display capabilities in ways that eliminate the need for them. Traditional techniques of task 

analysis emphasize observable behavior. In some cases, a more detailed analysis of problem solving and decision making 

requirements may be needed. This section described three categories of cognitive task analyses (empirical, function-based, 

and cognitive simulation) that support the analysis of the cognitive activities. The failure of a licensee to address cognitive 

tasks and strategies during task analysis may indicate that proper attention was not given to HFE considerations during the 

requirements-analysis process for upgrades. 

Finally, a review of the requirements-analysis phase of an upgrade should consider staffing and human reliability. 

Upgrades generally are intended to reduce personnel's workload and enhance reliability and safety. However, upgrades 

sometimes may be at variance with the assumptions underlying earlier analyses. When upgrades affect risk-important 

activities or impose new demands on the number and qualifications of plant personnel, then it may be appropriate to review 

them in detail.

5 -17 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

8.3 Interface Design Phase 

The interface design phase described in NUREG-07 I1 addresses the design, development, and testing of the various 

interfaces between personnel and the plant. This section addresses the HFE considerations for the design of the following 

aspects of upgrades: 

* HSI 

* Procedures 

* Training. 

5.3.1 HSI Design 

The purpose of this review is to ensure that design requirements were appropriately translated into the detailed HSI design 

by systematically applying HFE principles and criteria. This review may evaluate both the HSI design process, and the 

product of that process. The process and criteria for conducting a thorough HFE review of a new HSI design is set out in 

Element 7, Human-System Interface Design, in Section 8, NUREG-071 1. This review should be conducted for all upgrades 

that are significant to safety, affect the role or tasks of personnel, and 

Change existing HSI components used for risk-important tasks, or 

Provide new HSI components for risk-important tasks. 

The scope of the HSI design review, specified in Section 8.4.1.1 of NUREG-07 11, may be limited to the upgrade and its 

interactions with the other components of the HSI. Inputs to the HSI design process specified in Section 8.4.1.2, may be 

limited to those from HSI design process analyses that are applicable to the upgrade. Similarly, the HSI design procedures, 

specified in Section 8.4.1.3, may be limited to those design process analyses applicable to the upgrade. 

The following is a discussion of HSI design considerations that are particular to HSI upgrades. They were identified by 

reviewing literature pertaining to complex human-machine systems, and from reviews of industry experiences. They should 

be dealt with, along with the other considerations in Section 8.4 ofNUREG-071 1, during HSI design reviews for upgrades. 

The following are discussed: automaticity; consistency between new and replaced HSI components; consistency between 

new HSI components and the rest of the HSI; the functional integration of HSI components; and crew coordination and 

cooperation. 

Design for Automaticity 

Tasks that may benefit from automaticity, such as those that must be performed quickly and do not have high negative 

consequences, should be addressed by developing user interfaces that impose minimal demands upon attention or mental 

processing to operate them. They should be designed according to the two high-level HSI design principles for secondary 

task control that suggest ways to minimize cognitive and response workload (NUREG-0700, Appendix A; also Appendix A 

of this report). For example, they should not require a great degree of fine motor skill, such as positioning a cursor on a 

small target, or of physical effort, such as moving a switch with a high activation force, nor should they impose high 

demands on working memory, such as requiring the user to remember numerical values or identification codes. Similarly, 

they should not impose high demands on long-term memory, such as requiring the user to remember an unusual pattern of 

NUREG/CR-6637 5 - 18

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

actions. The means of operating these user interfaces should be consistent with population stereotypes. In addition, user 

interfaces should be designed consistently so that actions with similar goals can be performed the same way using interfaces 

that appear similar. 

To encourage consistency in user interfaces, the design organization should develop its own design standards. Their 

development should be based on HFE reviews of HSI technologies, design approaches, and user's requirements. The 

standards should address the appearance, function, and means of operation of user interfaces, and should be documented for 

the design organization, as described in Section 8.4.2.1, HFE Design Guidance Development, NUREG-071 1. 

The effectiveness of design approaches for supporting automaticity should be evaluated through performance-based tests 

during the design process. They should ensure that the user interfaces can be operated quickly, accurately and without 

apparent difficulty. 

Separate design techniques should be used for those tasks for which automation should be discouraged, because it may result in errors with significant negative consequences. A review of soft controls used in complex human-machine systems 

found that these interfaces are especially prone to the following categories of slips: unintended actuation, description errors, 

capture errors, mode errors, misordered components of an action sequence, and loss-of-activation errors. Descriptions of 

these categories and specific design approaches for protecting against errors are given in Stubler, O'Hara, and Kramer. 

(2000). Performance-based tests should be conducted to identify the types of errors that may be made, to evaluate the 

effectiveness of alternative means of protection against them, and to demonstrate the effectiveness of the final design. 

Design for Consistency Between New and Replaced HSI Comnonents 

When the HSI is upgraded, such as when a component is replaced, some of the design characteristics that support user's 

performance may change, sometimes in ways that have safety significance. For example, an indicator light on a controller 

could represent normal status in the original design, but represent a serious condition in the replacement. If an operator 

applied knowledge of the old controller when viewing the new one, an important problem may not be diagnosed. Also, an 

action that produces a small adjustment in feedwater flow in the old controller may produce a large one in the new 

controller; applying the old skill to the new controller may upset feedwater flow. In such cases, the old and new HSI 

components may be considered to have poor consistency. An HFE review should attempt to identify such inconsistencies, 

their potential consequences, and the need for remediation. 

A model of transfer (e.g., Osgood, 1949) is often used as a framework for describing these effects (Holding, 1987; Swezey 

and Llaneras, 1997; Sawyer, Pain, Van Cott, and Banks, 1982). Positive and negative transfer, respectively, refer to the 

facilitative and inhibitory effects of prior learning upon performance in new (transfer) circumstances. A positive transfer of 

training is said to have occurred if the experience of leaming to use the old component facilitates leaming the new 

component. For example, if a new feedwater controller is installed in the CR and the operators who were skilled in using the old controller learn to use the new controller faster than operators who did not use the old one, then a positive transfer 

of training has occurred. 

In some cases, prior learning with an old component may inhibit the ability to learn the new one. This negative transfer of 

training can increase training times, decrease response time when using the upgraded HSI, and lead to errors. 

Transfer conditions are usually described as a function of the similarity of stimulus and response. The Osgood model 

represents gradients of similarity between the stimuli and responses of the training and operational (transfer) environments 

using a three-dimensional surface. Stimulus similarity is presented on one axis, response similarity on a second axis, and 

the amount and direction of transfer (i.e., positive versus negative) on the third dimension. The effect of variations in the

5-19 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

required responses changes from maximum positive transfer when the responses are nearly identical, to zero (no transfer), 

to negative transfer when antagonistic (i.e., opposite) responses are required. When responses are identical and the stimuli 

are varied, transfer drops to zero as the stimuli's similarity decreases. However, with antagonistic responses, transfer 

increases from negative to zero as the stimuli's similarity decreases. 

Attempts to replicate the predictions of the Osgood model in experiments have had mixed results. While the model 

characterizes transfer performance with simple tasks, such as learning verbal paired associates, the transfer surface is 

inadequate for complex tasks (Swezey and Llaneras, 1997). Contradictory results were obtained in predicting negative 

transfer (Holding, 1987). Numerous studies of real-world learning also contradicted the Osgood model's prediction of 

positive transfer by demonstrating that transfer may increase (rather than decrease) by reducing the fidelity of a training 

simulator, thus reducing the similarity between the training and the transfer environment (Swezey and Llaneras, 1997). 

One problem with applying this model to the complex tasks performed in NPP CRs is the difficulty in defining the degree of 

similarity of stimuli or responses which may have many important dimensions for personnel performance. For example, a 

simple HSI component, such as display, may be described by its position, shape, size, color, information content, 

informational change, and spatial relationships. The response, such as operating a switch, may be described in terms of 

discrete versus sequential actions involving identification, discrimination, reading, and movement (Sawyer et al., 1982). 

Further, the work environment may contribute other dimensions to the stimuli. For example, a simple action such as 

operating a switch may be associated with stimuli such as procedure steps, verbal instructions, alarms, and various 

indications on a control panel. The response, flipping the switch, may also be associated with sequential relationships with 

other controls and displays, and the operator's intentions and expectations. Because stimuli and responses may vary in 

many dimensions, it is difficult to define the degree of similarity of different conditions, imposing practical obstacles to 

applying the transfer model. 

Another problem is that much of the empirical work and resulting HFE guidance pertains to transfer effects between a 

training environment and an operational environment, primarily addressing ways to modify the user interface to enhance 

positive transfer. However, for reviewing HSI upgrades, the primary concern is to identify and assess factors that contribute 

to negative transfer, for which there is less guidance. 

Sawyer et al. (1982) reviewed the transfer literature and developed a set of initial principles on NPP CR modifications. 

Although not comprehensive, these principles are consistent with current transfer theory, and provide a basis for 

considering inconsistencies between the original and upgraded HSI that may affect personnel performance. Thus, these 

considerations deserve attention when reviewing an HSI upgrade; the following are derived by Sawyer et al. (1982): 

(1) Negative transfer may be produced when responses, which conflict with the original ones, are introduced into the 

modified HSI design. 

(2) A possible exception to consideration 1, is when the original controls and displays conflict with population 

stereotypes, or are not designed consistently with each other. Then, personnel performance may already be affected 

by these inconsistencies, and modifying the responses may improve it. 

(3) Positive transfer usually occurs when responses are unchanged between the original and modified components, 

even if the stimuli change. For example, if the new HSI component looks different but responds in the same way, 

personnel often adapt quickly (i.e., positive transfer occurs). 

(4) The greatest amount of positive transfer is generally produced when the old and new component have conceptual 

similarity. To the degree that their stimuli and responses are similar, positive transfer will usually occur. 

NUREG/CR-6637 5-20

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

(5) If both the stimuli and responses are different between the old and new conditions, little or no transfer can be 

expected. 

(6) There is little evidence to either support or refute the notion that increased stress causes prior patterns of response 

to regress. 

Transfer from the old HSI components to new ones may be mediated by several factors. The following are based on review 

considerations offered by Sawyer et al. (1982): 

(1) The amount of positive or negative transfer partly depends upon the degree of similarity or difference between old 

and new stimuli or responses. However, all the factors that contribute to these similarities and differences may not 

be known. 

(2) Positive transfer may be enhanced by enforcing the user's understanding of how stimuli and responses relate to the 

function of the HSI component and the associated plant system. [This consideration is based on the concept of 

mediation (Sawyer et al., 1982).] 

(3) Positive transfer may be enhanced when verbal cues or names associated with the HSI changes can be learned by 

users. [This consideration is based on the concept of predifferentiation (Sawyer et al., 1982).] 

Schlager et al. (1989) have a different perspective on transferring knowledge and skills. They indicate that in complex task 

environments, such as control room settings, personnel employ several cognitive skills, such as chunking of information 

and automaticity of actions (see Sections 4.3.1 and 4.3.2), to enable them to overcome information processing limitations. 

Many of these skills are supported by particular characteristics of the HSI design, such as its spatial arrangement and 

coding schemes. When the HSI is upgraded, transfer performance is enhanced to the extent that the new configuration 

supports the existing strategies. 

This can be illustrated with four cases. In the first case, the design characteristics allow these strategies to be applied in the 

same way in the original and upgraded HSI environments; hence, personnel readily adapt to the change and transfer is 

high. In the second, the strategies used in the original configuration are applicable but must be slightly modified to 

accommodate the upgrade so that transfer declines a little. For example, personnel may select and sort information the 

same way but use different HSI characteristics when doing so (i.e., an air traffic controller sorts inbound and outbound 

aircraft by the color of the icons rather than by their shape). In the third case, the design characteristics do not support the 

user's strategies and new skills must be developed. This may be very disruptive to personnel performance, especially if 

users must discover new strategies on their own. If they do not, performance may decline. 

In the fourth case, the HSI undertakes some of the operators' task of information processing. For example, a new air traffic 

control display system may automatically separate inbound and outbound aircraft so that users are not required to do this 

mentally. The upgrade organizes information in a way that is consistent with the user's task, while reducing some of the 

user's cognitive-processing demands. In addition to reducing the mental workload of experienced users, this approach 

benefits newer users by allowing them to achieve high levels of performance without having to learn the special 

information-grouping skills of the experts. An example from air traffic control is given by Schlager et al. (1989): 

The Inset Display is capable of presenting graphically a conceptual chunk of aircraft [traffic] such as 'all aircraft 

landing in a particular airport' while filtering out all others. Thus, the potential exists for these features to facilitate 

the performance of full performance level controllers as well as new [personnel] by greatly reducing the amount of 

information they must keep in working memory. (p. 48)

5 -21 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

These benefits can only be achieved after users learn these new features and incorporate them into their work. Schlager et 

a]. note that careful analysis and design may be necessary to ensure that the way information is organized in the displays is 

consistent with the ways in which users organize the information. This process may be a focus area of an HFE review. 

Design for Consistency Between New HSI Components and the Rest of the HSI 

HSI upgrades often are installed in NPPs on a component-by-component basis (O'Hara, Stubler, and Higgins, 1996). For 

example, in a transition to digital technologies, the old analog component is removed from the control panel and a digital 

HSI component is installed. However, many replacement and upgrade products are generically designed, such as flat panel 

displays and input devices, and each manufacturer may have different standards for them. Therefore, after several upgrades 

have been installed, the HSI may contain many similar-looking user interfaces with different conventions for presenting 

information and different mechanisms for accessing and controlling it. This can affect the overall consistency of the HSI. 

For example, information coding conventions may have different means on different devices, and the correct method of 

operating one device may be incorrect for a similar one. 

In this setting, personnel may make frequent transitions between the various HSI components. Norman (1983) describes 

errors that occur when user interfaces lack consistency. When users lack the knowledge to properly operate some aspect of 

an interface, they may try to do so correctly through analogy with other, similar aspects of the device. This derivation may 

be entirely unconscious. Errors occur when the user derives an action sequence that is inappropriate because the user 

interface has a command structure that differs from (is inconsistent with) that on which the analogy was based, even though 

the commands appear to be related and share a common description of purpose, action, and part of the command format. 

Norman states that in addition to affecting input, this phenomenon may influence the user's behavior when reading 

displays. 

Tanaka et al. (1991) describe this as a problem of alternating use of interfaces, and contrast it with the transfer paradigm in 

which the user is exposed to one set of stimuli (i.e., a user interface) and then another, but does not return to the first set. 

This situation occurs when an old user interface is replaced with a new one. In the alternating use model, the user 

continually switches between various user interfaces, as when an HSI contains a variety of user interfaces. The transfer 

model predicts that performance will be enhanced when transferring between two interfaces in which the responses 

(methods of operating the interface) are similar. For example, if stimulus 1 (user interface 1) and stimulus 2 (user interface 

2) differ but their required responses are similar, then performance will be enhanced when transferring from 1 to 2. 

However, Tanaka et al. predicted that when the user alternates between user interface 1 and user interface 2, this similarity 

in response will impair rather than enhance performance; this high overlap of methods used in the interfaces will cause 

errors and slow execution times. Performance suffers because the user cannot remember which methods correspond to 

which interface. 

Eberts (1994) provides the following example. The DOS computer operating system is command based; a directory is 

displayed by entering the command "DIR." The VMS operating system is also command based, but users can enter as 

many characters as they wish, as long as the minimum needed to distinguish the command from others is provided. For 

example, to display a directory in VMS the user may enter the minimum character string, "DIR," the maximum character 

string, "DIRECTORY," or one in between (e.g., "DIRECT"). However, in DOS, any characters beyond "DIR" are 

unacceptable and result in an error message. Because of the high overlap in these methods, the user may have difficulty 

remembering which command to use in switching between the two. By contrast, the transfer paradigm would predict that 

performance would be enhanced. 

Kieras (1988; cited by Tanaka et. al., 1991) investigated ways in which a natural-language goals, operations, methods, and 

selection rules language (NGOMSL) may be used to evaluate aspects of human-computer interaction tasks. Kieras's model 

NUREG/CR-6637 5-22

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

predicts a high gain in user performance due to consistency between DOS and VMS. However, a separate model, also based 

on NGOMSL but developed by Tanaka et al., predicts a decrement due to what they consider inconsistency. As another 

example, Tanaka's model predicts that a user should encounter very little difficulty when switching between the DOS 

system and the Apple Macintosh system because they have very different methods for performing this task and so little 

confusion should occur. Unlike DOS, the Macintosh operating system has a direct manipulation interface. A directory is 

retrieved by using a mouse to double-click on a folder icon. This prediction conflicts with those based on the Kieras model 

and the transfer paradigm. 

Tanaka et al. used a NGOMSL task analysis to quantitatively determine consistency between interface designs that were 

used alternatively. They counted the number of changes that would be needed to change the methods of one task to those of 

another. This analysis was similar to that used by Kieras to determine gains due to consistency. However, there was one 

important difference; Tanaka's analysis treated the overlap in task methods in a different way. In Tanaka's analysis, if two 

tasks were the same except for the insertion or deletion of some steps, then they were considered inconsistent, but if the two 

task methods were the same except for replacement of specific words in the steps, then they were considered consistent. 

Tanaka et al. tested this model empirically. Task execution times were faster for those tasks that had been determined to be 

consistent than for inconsistent tasks. In addition, these differences were enhanced when users returned to the tasks after an 

absence. 

Tanaka et al. conclude that the practical significance of their model and experiments was that consistency can be 

determined only within the context of the tasks used and their sequencing. The same set of interfaces can be consistent or 

inconsistent, depending upon whether use is transferred from one interface to the other, or is alternated. This conclusion 

suggests that conventional HFE principles of consistency and standardization of the interface may need to be applied 

differently in a computer-based control system. In particular, HFE reviews should 

Distinguish between similarities in the appearance of the user interfaces and the methods used to operate them. 

Determine which user interfaces will be used on an alternating basis by the same person. 

Greater attention should be focused on interfaces that have similar methods and will be used alternatively by the same user. 

Based on their results, Tanaka et al. developed the text-editing model (TEM) to quantitatively predict the effects of different 

kinds of computer display and interaction-consistent mappings on the prolonged performance of computer-based tasks. 

Because TEM is based on the goals, operations, methods, and selection rules (GOMS) model (Card, Moran, and Newell, 

1983), it can be readily and advantageously applied to any human-computer interaction situation that was previously 

examined by GOMS. This model is being refined and may eventually provide a useful analytic tool. 

Design for Functional Integration of the HSI 

A problem that may occur when plant systems and HSI components are upgraded is a lack of apparent functional 

integration. This describes the situation in which different control or display devices appear to be functionally related, 

when in fact they are not. It may occur when the user interfaces of the HSI do not provide sufficient information about the 

relationships between devices. 

The following example is from a chemical plant. Part of the plant's alarm system was replaced with a digital system. However, some signals from older analog systems also were fed into it. This led to misunderstandings by the operators, 

some of whom incorrectly assumed that any alarm appearing on the digital alarm display indicated a fault with one of the

5 -23 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

digital control systems. This misunderstanding led to difficulty in diagnosing the problem when the alarms were generated 

by the analog systems (O'Hara, Stubler, and Higgins, 1996). 

In another example, a computer-based display system was installed as a CR upgrade to help operators monitor the trip 

parameters for one of the reactor's two safety shutdown systems. Operators could access displays that indicated the current 

value of the parameters and their difference from the trip limits. The safety shutdown system was controlled by a separate 

computer and, periodically, the trip limits had to be manually changed to reflect changing characteristics of the reactor core. 

However, the new display system did not communicate with the safety shutdown system's computer. Consequently, the trip 

limits had to be manually entered into both computers when the shutdown system's trip setpoints were changed. It was not 

apparent from the interface that these computers were not in communication. If the data were entered incorrectly into the 

computer-based display system, it could create opportunities for mistakes; operators monitoring the reactor trip parameters 

might be unaware that the margin between the actual value and the trip limit did not represent the true state of the plant 

(Vicente and Burnes, 1995). 

When a plant upgrade is developed, the analysis of design requirements should identify any changes in the degree of 

integration between systems. These changes should be clearly indicated in the HSI, and also should be addressed by 

training. In addition, this analysis should identify cases in which the new design makes systems appear to be functionally 

integrated when they are not. The HSI design should ensure that the true relationship is accurately conveyed in displays, 

procedures, and training. 

Design for Crew Coordination and Cooperation 

The design of an HSI upgrade should consider the effects of crew coordination discussed in Section 5.2.2, Task Analysis. 

General review guidance for workplace design is given in NUREG-0700, Rev. 1. However, computer-based HSI 

technologies can affect crew coordination in complex ways, as was recognized for such diverse topics as operator-aiding 

systems in NPPs (Dien and Montmayeul, 1995), group-view displays for NPP CRs (Stubler and O'Hara, 1996b), computer

based procedures of NPPs (O'Hara, Higgins, Stubler, and Kramer, 2000), and design of ships' bridges (Hutchins, 1990). 

Dien and Montmayeul state that the HSI design should be compatible with the organizational structure of the crew and 

should support manageable levels of workload while ensuring plant safety. They specifically identify the following topics of 

concern: using HSI design to support the crew's coordination, and the flexible allocation of tasks between members. 

Guidance is slowly evolving; meanwhile, Dien and Montmayeul recommend using expert judgement coupled with extensive 

testing with conceptual prototypes. 

5.3.2 Procedure Development 

Element 8 in Section 9 of NUREG-07 11 states that the licensee's procedure development should result in procedures that 

support and guide human interactions with the upgraded HSI and plant systems, and support the appropriate response to 

plant-related events and activities. Because procedures are an essential component of HSI design, they should be derived 

from the same design process and analyses as other HSI components and be similarly evaluated. A procedure review should 

ensure that HFE principles and criteria are applied along with all other design requirements to develop procedures that are 

technically accurate, comprehensive, explicit, easy to use, and validated. This review includes both the procedures and 

procedure development. When the HSI or plant systems are upgraded, procedures also should be modified to reflect these 

changes. Procedures should be developed for all upgrades that are significant to safety and affect people's roles, addressing 

all personnel tasks affected by the upgrade and its interactions with the rest of the plant. 

The following describes some major considerations of procedure development from NUREG-07 11, applicable to upgrades. 

NUREG/CR-6637 5 -24

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Content 

The review should ensure that the procedures provide acceptable guidance to support personnel in using the upgrades to 

ensure plant safety. Criterion 2 of Section 9.4 of NUREG-0711 describes the bases that should be considered in developing procedures; for upgrades, the following may be particularly relevant: 

System-based technical requirements and specifications, especially where plant systems were modified by the 

upgrade 

* Results of task analysis, especially for tasks placing high demands on human capabilities, or for which errors have 

consequences important to safety 

• Risk-important human actions identified in the HRA and PRA that are affected by the upgrade 

• Initiating events considered in the emergency operating procedures that may be affected by the upgrade. 

Integration 

Modifications should be integrated across the full set of procedures; that is, modifications made to particular parts of the 

procedures should not conflict or be inconsistent with other parts, such that their overall use is impaired. 

Applicability to Temporary HSI and Plant System Configurations 

Procedures should be developed for all configurations of plant systems and HSI components with which plant personnel interact. Sometimes upgrades are implemented over time, resulting in temporary configurations of plant systems or HSI 

components. For example, a control system may be installed, but not all of its operating modes (e.g., higher-level automatic 

modes) may be available initially. In addition, control or display devices with different characteristics than the final design 

may be temporarily installed. Thus, a display device may have a scale that is labeled differently, or it may present a 

variable that will only be monitored during the system's temporary configuration (e.g., a special radiation detector 

temporarily installed near a piece of plant equipment.) The procedures should accurately reflect the characteristics of these 

temporary configurations and the required personnel tasks. 

Evaluation 

The procedures should be evaluated to ensure their adequate content, format, and integration. The degree of evaluation 

should be consistent with the degree of change to personnel tasks and the safety significance of those changes. If the 

upgrade substantially changes tasks that are significant to plant safety, then verification and validation should ensure the 

acceptability of the procedures. Validation, described in Section 5.4, should ascertain that they correctly reflect the 

characteristics of the upgraded plant and can be carried out effectively to restore the plant's state. Additional documents 

with guidance for developing procedures are cited in Element 8, Procedure Development, NUREG-071 1. 

5.3.3 Training Development 

Training, described in Element 9 (Section 10) NUREG-07 11, should insure that personnel acquire the required knowledge and skills; it is an important factor in ensuring the safe, reliable operation of nuclear power plants. Previous sections of this 

report described new demands that may be imposed on operators by upgrades to systems and the HSI. These demands stem

NUREG/CR-6637 5 -25
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

from changes in the operators' functions and tasks. This section discusses HFE considerations of establishing training for 

upgrades, including the relationship of training development to other HFE activities. The objective of this review is to be 

certain that the training programs adequately address using the upgrades within the context of plant operations. This 

review should also ensure that the process used to develop training is based on the systematic analysis ofjob and task 

requirements. Element 9 ofNUREG-071 1 recommends developing training in coordination with the other elements of the 

HFE design process because this can provide a valuable understanding of what is required of personnel. For an upgrade, 

the training program should include all tasks that are affected by it or its interactions with the rest of the plant. 

5.3.3.1 Determining Relevant Knowledge and Skills 

The systematic approach to training, described by NUREG-07 11, is a top-down process by which programs are developed to 

address specific objectives related to the trainee'sjob. It consists of five steps: (1) a systematic analysis ofjobs, (2) the 

derivation of learning objectives from analyses describing the performance desired after training, (3) design and 

implementation of training, (4) evaluation of mastery of knowledge, and, (5) evaluation and revision of the training. The 

type of performance desired after training is often described in terms of knowledge, skills, and abilities (KSA) (Prien, 1977; 

cited in Goldstein, 1987): 

Knowledge - An organized body of knowledge, usually factual or procedural, whose application allows adequate 

job performance. It is the foundation upon which abilities and skills are built. However, possession of knowledge 

does not insure that it will be used. 

Skills - The capability to perform ajob easily and precisely. The term is often applied to both psychomotor and 

cognitive activities. For example, a psychomotor skill may entail precisely operating a control device. Cognitive 

skill may entail using strategies for rapidly finding and accessing information from the HSI, as discussed in 

Section 4.3.2. When a skill is specified, a standard of performance often is implied. 

Abilities - Usually applied to cognitive capabilities necessary to perform a job. Abilities usually require applying 

some knowledge. 

For NPPs, abilities are generally considered when selecting personnel. Training usually focuses on gaining knowledge and 

skills. 

Identifying the knowledge and skills needed for complex human-machine systems, such as NPP upgrades, is not always 

easy. Mumaw et al. (1994) state that while the systems approach to training (SAT) has dominated the development of 

training systems in many domains due to its standardized methodology for setting up comprehensive and manageable 

programs, it is most easily applied to procedural tasks in which discrete tasks are readily identified and described. SAT 

typically is less effective in the development of instructional materials for less-structured jobs that rely extensively on 

cognitive skills. SAT encourages the use of task analyses to identify tasks and their associated physical and cognitive skills. 

However, it lacks techniques for analyzing the components of cognitive skills critical to successful performance. Thus, 

while critical decision-making skills may be identified, the SAT processes provide little support for defining their important 

elements or determining effective ways for acquiring these skills. 

Others concur about the difficulty of defining knowledge and skill requirements for complex human-machine systems. 

Sanquist et al. (1995) state that in human-machine systems that featured traditional HSIs, training needs were often 

specified by observing and describing operator actions. However, for modem automated systems, in which greater demands 

are placed on the operators' cognitive capabilities, they consider that greater emphasis must be placed on analyzing aspects 

of judgement and decision making, some of which are not easily observed. 

NUREG/CR-6637 5-26

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Experiences in the nuclear power industry and other process-control industries suggest that personnel are not always 

appropriately trained for new technologies because the analyses of training needs were inadequate. The following are two 

examples. In the first case, a NPP operator was insufficiently trained in the skills required to operate a new digital 

feedwater control system and, observing fluctuations in steam generator level during power ascension, incorrectly concluded 

that the full-range, digital feedwater control system was malfunctioning. The operator assumed manual control of the 

system and tried to "bump" open the feedwater valve with a series of short intermittent key presses. However, the operator 

was unaware that each press corresponded to only about 0.1% demand, and so the series translated into negligible changes 

in valve position. Consequently, the plant tripped on low steam generator level. 

The second case involves training given to operators in a chemical plant, which had just installed its first major digital 

upgrade to the production system. The HSI for this plant originally consisted of traditional hardwired analog 

instrumentation. When the digital upgrade was installed, management was concerned about operators' ability to interact 

with a new HSI, which featured CRT-based displays and soft controls. Their training program focused on interface

management skills, such as using on-screen pointers, and navigating display systems. However, the upgrade extended 

beyond the HSI and included the control system, changing its structure and behavior. These changes were not given as 

much attention in the training program as was interface management. While the operators were generally able to operate 

the interfaces, they lacked knowledge and skills related to the control-system changes and therefore had difficulty operating 

the plant after this upgrade. When the second phase of the upgrade was planned, greater emphasis was placed on the 

plant's control characteristics (O'Hara, Stubler, and Higgins, 1996). 

These examples indicate the failure to consider all aspects of human performance when developing training programs for 

plant upgrades. In the first case, the operator was inadequately trained in using the interface. (The problems were further 

complicated by a deficiency in the design of an associated indicator.) In the second case, operators were trained in using the 

interfaces but the training did not adequately address the changes in plant behavior resulting from modifications to systems. 

Thus, if the necessary dimensions of personnel performance are not recognized when analyzing training needs, the resulting 

training program may have deficiencies that may not be recognized until an incident occurs. This may be important to 

plant safety. 

Table 5.3 shows some categories of personnel performance that should be considered when defining training needs for an 

upgrade. Three topics are identified in the first column: the plant, the HSI, and plant personnel. The first row, "Plant 

Interactions," covers changes to plant systems and equipment, and relates to the primary tasks of plant personnel, as 

described in Section 4.2. These include the generic cognitive tasks: monitoring and detection, situation awareness, response 

planning, and response implementation. The second column in this row addresses the types of knowledge that personnel 

may need for these modified systems and equipment, such as how they are structured, how they function, how they may fail, 

and how they behave under various conditions. The third column has specific skills personnel require for their primary 

tasks that may include perception skills for monitoring and detection; processing skills for recognizing patterns, applying 

heuristics, generating hypotheses, making decisions; and execution skills for manipulating systems and equipment. 

The next row, "HSI Interactions," describes changes that may be made to the controls and displays of the HSI in the 

upgrade. This topic was a primary focus of NRC research directed toward HSIs, especially hybrid ones. Considerations in 

this row relate to interface management tasks, described as secondary tasks in Section 4.2. The second column addresses 

the specific categories of knowledge people may need for the HSI. Examples include understanding its display structure, 

navigation paths, input methods, and failure modes. It may also include knowledge about the types of errors that may be 

made during interactions and the types of strategies for recovery.

5 -27 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Table 5.3 Some Knowledge and Skill Dimensions for Assessing Training Needs 

Topic Knowledge Skill 

Plant Understanding of plant processes, Skills associated with monitoring 

Interactions systems, operational constraints, and detection, situation awareness, 

and failure modes response planning, and response 

implementation 

HSI Understanding of HSI structure, Skills associated with interface

Interactions functions, failure modes, and management tasks 

interface-management tasks 

(actions, errors, and recovery 

strategies) 

Personnel Understanding of information Skills associated with crew's 

Interactions requirements of others, how interactions (i.e., teamwork) 

(In the CR and in the Plant) actions must be coordinated with 

others, policies and constraints on 

the crew's interactions 

The final row, "Personnel Interactions," addresses possible changes in the interactions between personnel after an upgrade. 

It includes interactions between personnel in the CR, between the CR and the rest of the plant, and between personnel 

located in the plant. This category was described in the discussions of crew performance and team skills (Sections 4.4.2 and 

5.2.2). Its primary focus is interactions involving CR personnel that may be important to plant safety. The second column 

addresses the types of knowledge that support coordination of information and actions between personnel, such as a 

common understanding of the plant and how it affects everybody's roles (shared mental model), of the information 

requirements and tasks of others, and of the plant's policies and practices (e.g., administrative procedures) related to 

people's interactions. The third column addresses skills that are important to teamwork. Examples may include perceptual 

skills (e.g., maintaining awareness of CR activities), processing skills (e.g., identifying when coordination is needed), and 

other communicating and coordinating skills. 

In studying training requirements for severe accident management, Mumaw et al. (1994) developed a detailed 

categorization of cognitive skills that extends beyond the knowledge and skill distinction in Table 5.3. Their categorization 

is based on a framework by Newell and Simon (1972, cited by Mumaw et al., 1994) in which behavior is directed by a series 

of comparisons between the current state and the desired state of the system. For example, an operator faced with a variable 

that deviates from a desired value must determine another appropriate path to the goal using available means. The 

problem-solver's task is to continually modify the problem (e.g., the variable) until it is identical with the goal (e.g., the 

desired value). The actual and goal value may be compared periodically to evaluate progress. Mumaw et al. propose that 

when developing training programs for complex, cognitively demanding tasks, analysts should first explicitly model the 

characteristics of highly skilled performance. They identified five elements: task-relevant knowledge, mental 

representation, rules for processing knowledge, domain goals and subgoals, and strategies: 

Task-Relevant Knowledge - This refers to knowledge about the characteristics of the plant, HSI, crew, and performance 

expectations relevant to a specific task. Types of knowledge that may apply to particular tasks include thermodynamic

NUREG/CR-6637 5-28

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

theory, plant systems (e.g., their structure, interconnections, and operating characteristics), transient phenomena, the logic 

underlying procedures, plant-specific facts, and plant policies. 

Mental Representation - An initial step in problem solving is to develop mental representations of the current state of the 

system and the goal. These skills allow personnel to group information from the task (e.g., indications from alarms and 

displays) into meaningful patterns relating to the user's understanding of the plant (situation model). They are critical for 

monitoring and interpreting data about the plant. As Sections 4.3.2 and 4.3.3 describe, proficient operators develop skills 

for extracting information from their environment based on their ability to recognize meaningful patterns and to understand 

functional relationships between objects and events. These skills are important because they allow individuals to manage 

their mental resources effectively. 

Rules for Processing Knowledge - Skilled problem solvers often apply rules to modify the representation of the problem so 

that it more closely matches the goal. Thereby, the problem solver can address the problem in a series of stages, or restate it 

in a form that can be more easily solved. A single rule may be applied to simple problems. More complicated ones may 

require multiple transformations involving many rules. For example, determining whether the reactor core is adequately 

cooled during an emergency may require the operator to assess the status of several systems and evaluate the performance of 

multiple flow paths and heat sinks. Each time a processing rule is applied, a particular subgoal may be resolved and the 

problem state brought closer to the goal. 

Domain Goals and Subgoals - Applying knowledge-processing rules to transform the problem usually requires an 

understanding of the relationships between a string of subgoals to approach the ultimate goal. By understanding such 

relationships, personnel can apply knowledge processing rules effectively. 

Strategies - Mumaw et al. describe mental strategies as the ability to string together individual knowledge-processing rules 

to achieve a task-relevant goal. For example, in electronics troubleshooting, skilled technicians use a "split-half' strategy to 

isolate faults in complex circuits (Stubler, Higgins, and Kramer, 2000). If a circuit has a test point that produces a bad 

signal and another that produces a good signal, the technician will test a point half way between them to determine whether 

the fault lies upstream or downstream. By applying this strategy repeatedly on the bad half, the problem is transformed to 

include progressively smaller portions of the circuit, allowing the technician to locate the fault without testing each point of 

the circuit individually. 

Thus, by analyzing the tasks of skilled personnel and expressing them in terms of task-relevant knowledge, mental 

representation, rules for processing knowledge, domain goals and subgoals, and strategies, analysts can identify training 

objectives that address the knowledge and cognitive skills required for skilled performance. 

5.3.3.2 Approaches to Training Cognitive Skills 

There are numerous techniques and methods for training cognitive skills. In examining methods for training of NPP 

personnel, Mumaw et al. (1994) identified a taxonomy of 19 methods to address the categories of cognitive skills described 

above, and grouped them into seven training goals. Mumaw et al. acknowledge that these categories are arbitrary since a 

single training method may cover more than one goal depending upon how it is carried out. However, this categorization is 

a useful way of matching established training methods to the dimensions of cognitive skill. Six of these seven categories are 

applicable to training programs directed toward upgrades; five goals are described below. The sixth training goal, the 

teaching team's skills, is discussed in Section 5.3.3.6. Mumaw et al. (1994) give a thorough discussion of all seven 

categories.

5 -29 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Teaching Knowledge - The primary concern here is knowledge that personnel should be able to access from their memory, 

rather than from external sources such as procedure documents and displays. For upgrades, this training should include 

knowledge about those changes made to systems or the HSI upgrades that relate to personnel tasks. This may include the 

unique characteristics of the upgrades, their relationships to the rest of the plant, changes in requirements for coordination 

(e.g., when and how CR operators must coordinate their actions), and new expectations about personnel or plant 

performance that stem from the upgrades. 

Mumaw et al. recommend that knowledge training address two types of knowledge failures: knowledge that is not tied to 

task performance (i.e., it is inert) and knowledge that is forgotten when needed. Inert knowledge (Bereiter and 

Scardamalia, 1985, cited in Mumaw et al. 1994; Mann and Hammer, 1986) can be applied in a training setting by the 

trainee but cannot be used effectively in the task setting. It fails to become part of the trainee's usable store of knowledge 

because the it is not tied to the context of task performance. Mumaw et al. recommend defining the job's context during 

training and representing the task meaningfully; this links the knowledge to the context of the job. Mann and Hammer 

(1986) state that to avoid inert knowledge, theoretical training should be integrated with training in the use of procedures. 

Mumaw et al. describe six training methods which address both types of knowledge failures. Considerations about 

determining the appropriate level of conceptual knowledge to be provided by training is discussed in Section 5.3.3.4. 

Teaching Knowledge Representation - Simply having task-relevant knowledge is not sufficient for skilled performance. 

Personnel must learn to organize information extracted from the work environment so it can be used effectively by the 

available mental resources. This training should include knowledge and skills required for understanding the structure and 

behavior of the plant system or HSI upgrades. It is particularly important that people can develop adequate mental models 

of automated components (e.g., new control modes) and the HSI (e.g., features of the display system that manipulate or 

manage the presentation of information to users). 

While users may develop these knowledge-representation skills on their own after extensive experience with an upgrade, it 

is preferable to train them in these skills to avoid initial decrements in performance that may occur before they have enough 

experience to refine these skills by themselves. Knowledge-representation training for an upgrade should focus on two 

areas. First, people should be trained in recognizing perceptual patterns in the new HSI, and taught how to apply existing 

perceptual skills, developed for the old HSI, to the upgraded one. If existing skills are not applicable, then new skills should 

be taught. Second, the knowledge-representation training should discuss mental models to give an adequate understanding 

of the structure and behavior of the upgrades. 

Teaching Rules Applied to Decision Making - When developing this training for upgrades, analysts should identify the 

applicable rules. Thus, for system upgrades, the knowledge-processing rules may pertain to interpreting symptoms 

associated with malfunctions. For example, automated systems sometimes can compensate for the performance of a slowly 

degrading plant system and mask the symptoms of the malfunctions (O'Hara, Stubler, and Higgins, 1996). When new 

levels of automation are introduced, operators and maintenance personnel may need to learn new rules for interpreting the 

plant's symptoms and detecting these malfunctions. For HSI upgrades, such as a computer-based display system, personnel 

may need to learn rules for interpreting the information from the new HSI components or detecting failures. They may 

need to learn new rules for interacting with new types of information, or new control and display capabilities provided via 

graphical. user interfaces. 

Training that is intended to teach rules for decision making should seek to eliminate incorrect rules, called "buggy rules" 

(Brown and Burton, cited in Mumaw et al., 1994). Skilled personnel develop rules to support them in making decisions 

(e.g., alarm B always occurs shortly after alarm A). However, these rules may not invariably reflect the true behavior of the 

plant, or may be overgeneralized to non-applicable situations. Buggy rules may also include ones that worked correctly 

before the upgrade but are no longer appropriate. 

NUREG/CR-6637 5-30

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Teaching Strategies, Goals, and Subgoals - This training addresses problem solving and decision making which entail 

decomposing a problem into an organized collection of goals and subgoals, and then identifying specific rules that can be 

applied to each. It may involve using strategies - a sequence of rules used to achieve a subgoal or goal. Together, the use 

of strategies, goals, and subgoals is essential to response planning. 

This training may be particularly important when plant systems are upgraded. Such changes may affect the relationships between goals and subgoals, and the paths available for controlling the plant's state. In the upgraded plant, some subgoals 

may no longer be relevant to higher-level goals and personnel may need to learn new strategies. The introduction of more 

advanced levels of automation may change the considerations involved in selecting response paths. For example, a new 

automation system may simultaneously control a large number of plant variables, and the control actions may involve a 

different set of side effects than existed before the upgrade. For example, a larger number of plant systems may be affected 

by a decision to place this system in a manual-control mode. 

Changes in the HSI may affect the strategies that personnel can use for accessing information about the plant or carrying out control actions. Older strategies for scanning spatially distributed alarms and displays may be inappropriate in an 

upgraded HSI with computer-based displays. From a review of eight HSI upgrades, Heslinga and Herbert (1993) concluded 

that where conventional HSI equipment was still available after the upgrade, operators tended to return to these well-known 

information sources. Operators tended to develop strategies for finding information quickly from conventional HSI 

equipment, such as control panels and displays, because their arrangement provided overviews of status and they were more 

directly accessible. By contrast, obtaining an overview and accessing controls and displays were considered to be more 

difficult in hierarchically structured VDU display systems. 

In reviewing the implementation of a computer-based display system in a NPP, Roth and O'Hara (1998) found that training in strategies for using the HSI was very important. The relevant findings are summarized below. 

Because training for NPP operators tends to focus on controlling the plant, adequate attention may not be given to 

effectively using HSI capabilities during events. In this study, the utility had focused little attention on such training. There had been little discussion on how the HSI components were used, or how they could be used better; this was left to 

individual crew members to decide. Consequently, there was much individual variability within and between crews in the 

extent to which and how they used the different systems (e.g., the graphic display system). For example, the computer based display system, especially the computer-based procedure (CBP) component, reduced the demands on the attentional 

resources of operators under some conditions, but not all operators may take advantage of this benefit. One shift engineer pointed out that the board operators in his crew took advantage of their freed attentional resources, and, as a result, kept better track of the event. However, he was unsure of whether other board operators also did so. He felt there may be a need 

to specifically train operators to take advantage of the attentional resources that were freed by introducing the CBP. 

The lack of explicit training in using the HSI systems reflects variously (a) an acceptance or expectation that operators will 

develop familiarity with the various features of the systems informally on their own (i.e., that there is no need to teach this 

knowledge and skill), and (b) an acceptance or expectation that different individuals will tailor the systems to their own 

personal style (e.g., that they will put up different graphic displays or different collections of windows on their VDUs). There was a similar lack of formal training in fossil-fuel power plants and chemical plants visited as part of this project (O'Hara, Stubler, and Higgins, 1996). Operators developed informal practices for arranging particular displays in these 

computer-based CRs. Arrangements varied between control rooms even in multiple-unit plants where the systems and HSIs 

were nearly identical. In one chemical plant, operators failed to detect the early stages of a serious cascading failure of a 

system because their arrangement of display pages did not allow them to immediately understand the problem (O'Hara, Stubler, and Nasta, 1997). While it is important to guard against over-regimentation and to allow operators flexibility in 

tailoring their VDU work space to their particular needs, such flexibility increases the demands on operators to engage in 

5 - 31 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

tasks not directly related to process control. Because operators are often reluctant to perform such tasks in high-workload 

situations (O'Hara, Stubler, and Nasta, 1997), training may improve performance during these situations. Because 

empirical studies are lacking, it is unclear how this training should be conducted, or the extent to which performance may 

be improved. The training recommendations of Mumaw et al. (1994) provide some initial direction. 

In addition to requiring personnel to learn new strategies, introducing new technology may create different demands for 

maintaining the old strategies. This is especially true when an old HSI technology is maintained as a backup for use when 

the new HSI technology is inoperable or unavailable. In this case, personnel must maintain their skills for both systems and 

develop skills to transfer smoothly from one to the other. An important example is making the transition between 

emergency operating procedures (EOPs) that are in computer-based form to EOPs that are in paper-based form when the 

CBP system fails (O'Hara, Higgins, Stubler, and Kramer, 2000). Paper-based procedures require the operator to mentally 

track many things, including transitions between EOPs, cautions and warnings that apply to multiple steps, and steps that 

may require time for completion (e.g., monitoring a variable until a particular value is reached). CBPs may require a 

different set of skills if these tasks are automated. For example, operators must develop skills for interacting with a 

computer-based interface. In addition, CBPs may impose new demands such as the need to manage the automation and 

ensure its proper operation. In a study of the introduction of CBPs into a NPP (Roth and O'Hara, 1998), operators stated 

that it will be important to be trained in using paper-based procedures so they do not lose their valuable skills in this 

medium. However, this concern is not limited to CBPs. Any time an older technology is kept as a backup to a newer HSI 

technology, personnel may be required to maintain separate sets of skills; this is likely to increase training demands. 

Mumaw et al. (1994) state that strategies, goals, and subgoals training should support personnel in 

Determining the goals that are most relevant to the current plant state 

Selecting paths for achieving those goals 

Ensuring that actions specified by procedures are consistent with those goals and paths. 

Teaching Management of Mental Resources - Training personnel in managing their mental resources is important both for 

increasing the effectiveness of personnel training programs, and for enhancing performance on the job. This training is 

particularly important when the HSI has been-upgraded. It should address changes in the ways that the HSI presents 

information and control capabilities, and the actions that personnel may use to interact with it. Training should focus on 

reducing the amount of conscious attention required for tasks such as searching for and retrieving information from the 

display system and for routine control actions. One such method, automaticity training, is discussed further in Section 

5.3.3.4. 

5.3.3.3 Training for Conceptual Knowledge 

As new technologies are implemented in NPPs, such as increased automation in plant systems and advanced HSI 

capabilities, greater demands may be placed on personnel for understanding their structure and operation. Most approaches 

to training development advocate conducting training within the context of the individual's job by linking knowledge 

requirements to specific tasks. However, determining the appropriate level of conceptual knowledge (e.g., facts about the 

structure and behavior of the plant) poses some concerns. The following describes some considerations identified from HFE 

literature and interviews with personnel. 

Interviews with personnel from nuclear, fossil-fuel, and chemical plants, conducted in this project indicated that operators 

who are superior performers have a greater understanding of what makes the plant work and how it will behave (O'Hara, 

NUREG/CR-6637 5-32

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Stubler, and Higgins, 1996). While less knowledgeable operators view their tasks as a series of control manipulations, 

superior operators have a greater understanding of why they are performing them. Furthermore, the importance of training 

operators in conceptual knowledge was indicated by empirical studies. For example, troubleshooting by professional 

engineering officers in the engine CR of a supertanker was studied in a high-fidelity CR simulator (van Eekhout and Rouse, 

1981; cited in Rouse, 1990). It was found that errors associated with inappropriately identifying failures had a high 

correlation with a lack of knowledge about the functioning of the basic system and its automatic controllers. The errors 

tended toward mistakes (errors of intention) rather slips (errors of execution). That is, the officers planned inappropriate 

actions based on their inadequate or incorrect knowledge of the system. Errors in diagnosing failures of automatic 

controllers were often associated with a lack of understanding of their failure modes which then affected the officers' 

abilities to interpret the symptoms of failure. Officers were misled by the cues produced by these failures, drew incorrect 

conclusions, and then proceeded to act on them. This phenomenon was reportedly widespread in a series of simulations. 

However, interviews with NPP instructors suggested that training oriented toward conceptual knowledge rather than 

procedural knowledge (e.g., how to do something) can cause operators to overestimate their ability to diagnose and correct 

plant anomalies. For example, operators may deviate from procedures because they feel they understand where the 

procedure is heading and what is needed to correct the problem. Deviating from the procedures can create new problems. 

In other cases, operators who feel they understand a plant problem may attempt to act alone in resolving it, rather than 

obtaining support from specialists. Difficulties can occur when the operator's course of action does not fully account for the 

range of factors that the specialist may consider (O'Hara, Stubler, and Higgins, 1996). 

Further support for the notion that training in conceptual knowledge can allow operators to use that knowledge 

inappropriately was found by Mann and Hammer (1986) studying two groups trained to operate a simulated generic 

process-control plant. One group was provided with procedures for controlling the plant; the other group also had 

instruction in the theory of the plant, such as a description of the physical principles of its operation. One finding was that 

the latter group committed more blatant errors (called blunders) and omitted more procedure steps. The blunders included 

obviously inappropriate actions, such as ignoring procedure branching, using a procedure when the situation did not call for 

it, ignoring an action called for by a procedure, and ignoring parameter ranges specified by a procedure. The experimenters 

suggest that participants in the procedures plus theoretical instruction group may have selectively omitted or ignored 

procedures because their theoretical training caused them to think they could achieve better performance by improvising. 

They concluded that procedure-only training resulted in better compliance than training on procedures plus theory. 

As the plant systems and HSIs of NPPs increase in complexity with the introduction of new technologies, the importance of 

questions about the necessary level of conceptual knowledge is likely to increase. Further research is required to develop 

more definitive guidance. 

5.3.3.4 Training for Skill Automaticity 

Schlager et al. (1989) describe the typical process by which novices learn to "automate" certain actions. During a trainee's 

initial introduction to a task, the required actions are typically presented as a set of ordered steps. Each step requires 

attention and is consciously initiated by the trainee. At this learning stage, the actions are stored as factual knowledge and 

the trainee usually has little difficulty in describing each step. When an entire set of actions is linked to a discernable event, 

the set can be developed after sufficient practice into a single, coherent unit that is executed "automatically." When the 

event occurs, the individual can undertake the set of actions, focusing little attention on the individual ones. This is 

essentially a conversion of factual knowledge into procedural knowledge (Anderson, 198 1; cited by Schlager et al., 1989). 

Automaticity training allows this process to occur in the training setting rather than on the job; one purpose is to allow the 

trainee to attain a high degree of skill early in the program. This can free-up the cognitive resources of the trainee for

5 -33 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

higher-level activities, which can make the training program more efficient and, possibly, more effective. That is, the 

trainee can focus on learning higher-level skills, such as problem solving, decision making, and coordination of concurrent 

activities, without being distracted by the details of these lower level actions. A second purpose of automaticity training is 

to enhance performance on the "automated task." Empirical studies have shown large improvements in the speed and 

accuracy of perceptual searches and motor responses after automaticity training (Schneider, 1985). 

The following are suggestions for training operators to achieve automaticity, adapted from those offered by Schlager et al. 

(1989): 

Identify opportunities for automaticity training by identifying user-system interactions that are performed many 

times in the same way. 

Identify the conditions of applicability for each new user-system interaction to ensure that they are covered by 

training. 

Identify user-system interactions that are likely to become confused, and point them out to the trainee. 

For each user-system interaction, identify and make explicit each step required. 

Provide sufficient practice under the appropriate conditions so that users can perform each interaction 

automatically. 

User-system interactions that are likely to become confused are ones with inconsistencies between new interactions and the 

old ones. Confusion may also result from inconsistencies between similar new interactions in the new HSI (e.g., the new 

interactions of an HSI upgrade may be inconsistent with each other). Both of these cases were described in Section 5.3.1. 

In these cases, trainees should be taught practices to avoid errors, and rules for identifying and correcting them. 

Automaticity training typically involves practicing the task in isolation from other tasks over a period until the response 

time and demands for attention decrease. This may require the trainee to perform many trails over days (Mumaw et al., 

1994). Next, the task should be integrated with other tasks in dual- or multiple-task training exercises. When automated 

skills are integrated back into a multiple-task setting, an initial decrement in performance typically occurs because skills 

learned in isolation may be learned inefficiently. By practicing the skill when additional requirements for mental resources 

are present, such as a second task, the trainee can learn needed strategies for sharing attention and memory resources. After 

the skill has been mastered in this setting, the trainee is ready to apply it efficiently in a realistic task setting. 

When developing training for automaticity, it is important to consider the scope of the upgrade and the effect that the initial 

limited level of performance may have on overall plant operations; it may have little effect on plant operations if the scope 

of the upgrade is limited to HSI components that are used infrequently and have little associated safety significance. 

However, automaticity training may be important if the upgrade includes components that are significant to safety, or used 

so frequently during transients that poor performance may interfere with other tasks that are important to safety. In such 

cases, operators should be trained in the use of the upgrade to an extent that they can perform the necessary actions without 

having to recall or attend to the details of the actions. 

5.3.3.5 Team Training 

Team training should be undertaken when the design of an upgrade significantly affects personnel's ability to interact, and 

if these interactions are important to the plant's performance and safety. Studies of team performance in complex human

NUREG/CR-6637 5-34

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

machine systems have found that it may be negatively affected by introducing computer-based technologies (Hutchins, 

1990). 

Roth and O'Hara (1998) reviewed the introduction of a computer-based display system into an NPP CR. One component in 

particular, a computer-based procedure system, changed the ways in which CR personnel interacted when using procedures. 

Many of the operators' comments focused on the need for more practice on crew-interaction skills, including focused 

training on their new roles and responsibilities, communication, and coordination. Several participating crews pointed out 

that training as a crew will become more important with the new HSI systems. Because the operators now have more 

freedom in how they do their jobs, it is more important to understand how teammates perform their tasks. For example, when performing procedure-based scenarios, different shift supervisors verbalize the EOP steps to different degrees. With 

the new system, it is becoming more important for operators to understand how shift supervisors do this, and for the shift 

supervisors to understand the information requirements of the operators. It was concluded that shift supervisors should be 

trained in how to keep their crews informed as they move through a procedure. In this plant, crews did not always train as 

a unit if there were scheduling constraints. Instead, operators sometimes trained with people from different crews. 

Other findings by Roth and O'Hara (1998) that relate to team training include the following: 

The study reinforced the notion that the quality of decision making depends on the ability of every team member to 

be aware of important control actions about to be taken, and to evaluate their appropriateness based on their own 

knowledge and perspective. Several critical incidents were observed in which inappropriate control actions, 

suggested by the shift supervisor based on the CBP, were caught by the board operators or the shift engineer who 

were accessing different sources of information. 

The study further highlighted the fact that, in a control room environment, key information is distributed among 

crew members. Advanced HSI systems increase the knowledge and understanding of individual crew members 

through increased availability of information. On the other hand, more information can result in increased 

compartmentalization of knowledge, such that different crew members are more likely to possess different data. 

This places an increased premium on effective communication among crew members to share information. 

A useful technique for enhancing communication within a crew may be to cross-train operators so that each is 

aware of the communication needs and burdens of the other. For example, board operators may be trained in the 

shift supervisor's tasks of using the CBP. This may help operators understand the type of information that the shift 

supervisor has and the type of information that is needed from the operators. 

o A wide variability was observed among crews in the extent and content of their communication. The impact of 

such differences on individual and crew situation awareness and quality of team decision making is not well 

understood. More research is required on this topic to provide better guidance for developing and evaluating 

training programs. 

The study of generic factors that contribute to the successful performance of teams currently is growing. Models describing the components of teamwork and methods for measuring and enhancing teamwork are evolving from research (Swezey and 

Llaneras, 1997; Salas, Dickinson, Converse, and Tannenbaum, 1992; Oser, McCallum, Salas, and Morgan, 1989). For 

example, in a study sponsored by the NRC, a set of six performance dimensions was identified for describing team 

performance in NPP CRs (Montgomery et al., 1992, cited in Mumaw et al., 1994): communication, task coordination, 

maintaining task focus in transitions, adaptability, openness and participation, and team spirit. A similar set of 

performance dimensions was identified for aviation crews (Franz et al., 1990, cited in Mumaw et al., 1994):

5 -35 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

communication, situation awareness, decision making, mission analysis, leadership, adaptability and flexibility, and 

assertiveness. 

Observation is a common approach for exploring team skills in complex work environments. Analysts observe team 

performance to identify behaviors linked to specific skills. For example, when a new piece of information about the plant's 

status becomes available in a CR, analysts may watch for indications that crew members are communicating effectively to 

share the new information (Mumaw et al., 1994). An approach similar to the following may be used for developing team 

training programs that address upgrades: 

Identify relevant upgrade characteristics - The design characteristics of the upgrade that are important to crew 

performance should be identified from the function and task analyses performed in developing the design of the 

upgrade. 

Identify relevant team skills - Team skills involved in performing the crew's functions and tasks should be 

identified. 

Identify specific behaviors - Next, specific behaviors should be identified which demonstrate those skills. These 

may be specific tasks involving use of the HSI by many team members. For example, the skill needed for 

communicating the plant's status to the primary decision maker occurs at many points in plant operations, such as 

when operators periodically report core-reactivity values during a startup. 

Develop training scenarios - A set of training scenarios that encompass these behaviors should be developed. 

Although each may address different skills, the combined set should include all the relevant skills. 

After training scenarios have been identified, personnel must be instructed in team skills. Teams should be trained in the 

specific details of team-related tasks including how the roles of individual members relate, how the performance of the team 

depends upon the specific performance of the individuals, and what makes the team's task different from the sum of the 

individual ones (McIntyre et al., 1988, cited in Swezey and Llaneras, 1997). 

Salas and Cannon-Bowers (1995, cited in Swezey and Llaneras, 1997) identified three generic methods for training teams: 

information-based, demonstration-based, and practice-based. The information-based methods involve the presenting of 

facts and knowledge via lectures, discussions, and similar techniques. These low-cost, group-based delivery methods may 

be used to help team members understand what is expected of them, what to look for in specific situations, and for 

exchanging general knowledge. 

The demonstration-based methods show, rather than describe, the desired team behaviors. These behaviors are acquired 

through modeling (learning by example). For example, a crew could be videotaped while responding to a scenario in a way 

that demonstrated vital team behaviors. A training instructor could then show the videotape to a new crew pointing out 

their effective behaviors (Mumaw et al., 1994). Such methods provide shared mental models among team members (i.e., a 

shared understanding of such things as the current state of the task, the needs and expectations of crew members, and 

required control actions). In addition, these methods give examples of how individual members are expected to handle 

themselves during complex, dynamic situations. 

The practice-based methods for team training use participatory activities, such as role playing, that provide opportunities to 

perform and obtain feedback on activities associated with specific learning objectives. Using repeated practice sessions, the 

trainees gradually achieve the desired level of performance. 

NUREG/CR-6637 5-36

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

5.3.3.6 Evaluation of Training Systems 

Childs (1996) identifies three evaluation phases for assessing training programs: formative, summative, and operational: 

Formative Evaluation - Formative evaluation is applied to the training program as it develops and may continue through to 

its delivery. Formative evaluation assesses the effectiveness and efficiency of the training system. Effectiveness is related to 

the relevancy of the knowledge, skills, and abilities (KSAs) that are addressed. Efficiency is related to the amount of 

training resources used to achieve the desired level of proficiency in the trainees. From a safety perspective, training effectiveness may be of more concern than efficiency because effectiveness is concerned with the acquisition of required KSAs. However, reviews of efficiency measures may be useful in determining the degree of attention that is given to 

specific topics. 

Training system components may be assessed by sampling and assessing the following: 

• Training standards 

o Instructor's effectiveness 

* Training methods and media 

* Training measures and metrics 

• Training scenarios 

• Instructional strategies 

* Coaching and mentoring techniques 

* Adequacy of training products (e.g., courseware) and services relative to training standards 

* Potential for transfer of KSA to the job setting 

* Trainee's retention of knowledge and skills. 

Childs (1996) suggests using pilot tests (tryouts) to assess the validity of training program courseware. In such tests, 

instructors use the materials to train a representative sample of surrogate trainees. Childs offers the following evaluation 

criteria for examining the instructional materials and instruction delivery: 

* Courses, lessons, and tests are structured to facilitate the learning process 

* Length of courses, lessons, and tests matches the objectives and level of detail required for mastering KSA 

* Methods and media are effective and linked to objectives 

• Trainees' interactions with the courseware provide the necessary feedback 

* Sequence and pacing of delivery holds trainees' interest 

5-37 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Trainees are assimilating the intended KSAs. 

These criteria may be evaluated through questionnaires and focus groups, or analyses of courseware test items. 

Summative Evaluation - A summative evaluation is conducted when training is completed or when the trainee begins the 

job for which the training was conducted. The evaluation assesses the fit between the training system and the initial 

requirements for operational performance (i.e., the degree to which graduates can fulfil all the requirements on the job), as 

well as specific tasks under specified conditions. Performance is evaluated against predefmed quantitative or qualitative 

standards. 

Operational Evaluation - Operational evaluation is conducted as part of the job setting to verify that the trainee has 

acquired, and can use the necessary KSAs. It evaluates whether training satisfies operational requirements considering 

whether graduates are meeting or exceeding their job requirements, specific training components are facilitating job 

performance, job standards are consistent with training objectives, and whether training could be better. 

5.3.4 Summary of Interface Design Phase 

This phase addresses HFE considerations associated with designing, developing, and testing user interfaces associated with 

plant upgrades. The interfaces include the HSI, procedures, and training. 

The review of the HSI design process should ensure that design requirements were appropriately translated into the detailed 

HSI design by systematically applying HFE principles and criteria. Attention should be given to human performance 

considerations related to the user's adaptation to the new design, including automaticity, consistency between new and 

replaced HSI components and between the new ones and the rest of the HSI, and the functional integration of HSI 

components. Human performance should be addressed in developing and applying human factors guidance documents, 

tests, and evaluations. 

Plant procedures are reviewed to ensure that any modifications reflect the characteristics of the upgraded plant, and that 

they are properly integrated with the rest of the procedures. The review should consider the procedures' content, format, 

and integration. The extent of the evaluation should reflect the changes in personnel tasks and their safety significance. 

Developing training requires systematically considering the skills and knowledge required of operators using the upgraded 

systems. This process is driven by a recognition of required personnel characteristics, rather than the physical 

characteristics of the new design. Hence, training development is similar to the top-down analyses described in the 

requirements' analysis phase (Section 5.2). Two key focuses for HFE reviews are to identify the new knowledge and skills 

required of plant personnel, and the appropriate means for conveying them to personnel. This section identified categories 

of knowledge and skills associated with upgrades and discussed their acquisition. 

5.4 Verification and Validation Phase 

This section covers the verification and validation (V&V) of NPP upgrades. V&V evaluations seek to comprehensively 

determine that an implemented design conforms to HFE design principles and enables personnel to successfully carry out 

their tasks to achieve safety and other operational goals. Element 10, Human Factors Verification and Validation, found in 

Section 11 ofNUREG-0711 identifies the following five V&V activities: 

HSI Task Support Verification - A check to ensure that there are HSI components addressing all identified tasks 

NUREG/CR-6637 5-38

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

HFE Design Verification - A check to determine whether the design of each HSI component reflects HFE 

principles, standards, and guidelines 

Integrated System Validation - Performance-based evaluations of the integrated design to ensure that the HFE/HSI 

supports safe operation of the plant 

Human Factors Issue Resolution Verification - A check to ensure that the HFE issues identified during the design 

were accepted and resolved 

Final Plant HFE/HSI Design Verification - A check to ensure that the final in-plant installation of the upgrade 

conforms to the design resulting from the HFE design process and the V&V. 

NUREG-07 11 suggests that while activities generally should be performed in the order listed, the process is iterative. 

NUREG-0711 has comprehensive guidance for V&V of the entire HSI. In addition, NUREG-0700 gives detailed 

methodological considerations for HSI task support verification and HFE design verification. For an HFE review of an 

upgrade, all five of these V&V activities may not be necessary. For example, verification that a human factors issue was 

resolved may not be necessary if no HFE problems are found in the preceding V&V. Also, the scope and level of analysis of 

each V&V should be consistent with the scope and potential safety significance of the upgrade. The following describes 

considerations associated with the first three activities, followed by a discussion of V&V specifically related to installing 

upgrades. 

5.4.1 HSI Task Support Verification 

The objective of this review is to ensure that the design provides all necessary alarms, displays, and controls to support all 

identified personnel tasks. It should verify that all aspects of the HSI (e.g., controls, displays, procedures, and data 

processing) required to accomplish personnel's tasks and actions, as defined by the task analysis, emergency operating 

procedure analysis, and the risk-important actions of the PRA and HRA, are available through the HSI. In addition, it 

should be ascertained that the HSI does not include unnecessary information, displays, controls, or other features. 

Verification of HSI task support should be conducted for all upgrades that are important to safety and affect the role of plant 

personnel as described in Section 11.4.2 of NUREG-0711. 

5.4.2 HFE Design Verification 

The objective of this review is to ensure that the design conforms to HFE principles, guidelines, and standards. All HFE 

aspects of the upgrade HSI should be examined to ensure that the design is appropriate for personnel's task requirements 

and operational considerations, and as defined by design specifications. Deviations should be justified based on documented 

rationales, such as trade studies, literature evaluations, operational experience, and tests and experiments (see NUREG

0711). HFE design should be verified for all upgrades of the HSI and its components that are significant to safety and affect 

the roles of plant personnel; its scope should also include the interactions of upgraded components with the rest of the HSI. 

When verifying HFE design, it is important to understand personnel tasks within upgraded plant so that HFE principles, guidelines, and standards may be interpreted and applied properly. As described under function-requirements analysis and 

function allocation, people's roles may change after changes to the HSI or plant systems. Therefore, an important step is to 

characterize the changes in components and personnel functions and tasks in the upgraded plant before applying HFE 

parameters. This characterization should reflect functions and tasks identified by evaluations earlier in the design process.

5 -39 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

The methodology for HFE design verification in NUREG-0700, Rev. 1 suggests ways for characterizing an HSI. In 

addition, the guidance and technical basis documents developed under this and other NRC projects provide frameworks for 

revealing important design characteristics for specific HSI technologies. When reviewing an upgrade, these frameworks can 

identify those features of the HSI most important to personnel performance which should be addressed by the review. These 

frameworks can also help the reviewer in interpreting FIFE guidance because they link HSI design characteristics and 

human performance and so highlight the importance of particular characteristics. 

Another consideration in HFE design verifications is the need to focus on the consistency of HSI upgrades with the rest of 

the HSI. The high-level HSI design review principle of consistency in NUREG-0700, Rev. I (NRC, 1996, p. A-2) states 

There should be a high degree of consistency between the HSI, the procedures, and the training systems. At the HSI, the 

way the system functions and appears to the operating crew always should be consistent, reflect a high degree of 

standardization, and be fully consistent with procedures and training. 

Inconsistencies between the upgrade and other aspects of the HSI were discussed in Section 5.3.1, HSI Design and Testing; 

they should also be addressed in HFE design verification of the final design. 

5.4.3 Integrated System Validation 

The objective of this review is to ensure that the design can be effectively operated by personnel for all requirements. A key 

consideration to evaluate user-system interaction within the context of the integrated (i.e., entire) HSI during realistic 

scenarios; extensive guidance on validating integrated systems is provided in NUREG-07 11 and NUREG-0700, Rev. 1, and 

its technical basis is discussed in NUREG/CR-6393 (O'Hara, Stubler, Higgins, and Brown, 1997). These documents 

address the comprehensive review of the HSI, including the entire main CR and portions of the HSI that are outside of the 

CR, such as local control stations. This guidance assumes that a new HSI design is being evaluated, but depending upon 

the scope and safety significance of an HSI upgrade, such a comprehensive review may not be necessary. Much of the 

procedural guidance describes ways to represent characteristics of future work environments, including the installed design 

and anticipated users. However, a review of an upgrade will involve an existing facility with an operating history, 

established training programs and procedures, and experienced personnel. This should be kept in mind during validation 

tests. 

The following describes four considerations for applying integrated system validation to HSI upgrades: applicability, 

personnel training, scenarios, and validation testbed. 

Applicability - Integrated system validation should be reviewed, in accordance with Section 11.4.4 of NUREG-07 11, for all 

upgrades that are significant to safety and may (1) change personnel's tasks; (2) change task demands, such as their 

dynamics, complexity, or workload; or (3) interact with or affect other HSI components in ways that may lower 

performance. Personnel tasks should include those identified as risk important in HRAs and PRAs. Integrated system validation may not be needed when an upgrade results in minor changes to tasks that may reasonably be expected to have 

little or no overall effect on workload and the likelihood of error. The scope of the integrated system validations should 

encompass the upgrade and its interactions with the rest of the plant. 

Personnel Training - NUREG-07 11 states that plant personnel who participate in the validation study should be trained to 

ensure that their knowledge of the operator's role, concept of operation, plant design, and use of the HSI is representative of 

anticipated users. This training should address the design characteristics and personnel requirements associated with the 

upgraded plant rather than with its former configuration. 

NUREG/CR-6637 5-40

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

NUREG-07 11 also states that plant personnel should be trained to near asymptotic performance (i.e., stable, not 

significantly changing from trial to trial) and tested before the actual validation trials. Asymptotic training assumes that 

personnel's performance improves quickly as a function of training, but then levels off after a high degree of skill is 

attained. 

Section 5.3.1 discussed the concept of transfer effects. When new HSI components are not consistent with the ones they 

replace, then negative transfer may occur. That is, performance may be slowed and errors may occur when people perform 

tasks on the new HSI components using actions and strategies that are appropriate for the old design. Personnel should 

have sufficient training with the new design to extinguish old response patterns and acquire new ones. In addition, the 

principles of good HSI design dictate that potential errors are anticipated and addressed by either designing upgrades that 

are consistent with the old components or by providing features to protect against errors. The validation study should 

examine the combined effects of HSI design and personnel training; this can only be done if personnel are adequately 

trained. 

Test Objectives and Scenarios -NUREG-0711 states that detailed objectives should be developed for validation tests. They 

should address the roles of plant personnel, shift staffing, adequacy of HSI support for people's allocated functions, their 

ability to perform tasks within time and performance criteria, adequacy of HSI features for their intended functions, the 

ability of personnel to make effective transitions between features, and the tolerance of the integrated to HSI failures of 

individual features. The objectives should also identify factors that may negatively affect integrated system performance. 

These objectives should be reflected in scenarios developed to validate the upgrades. In addition, the scenarios should focus 

on dimensions of personnel performance that are of particular concern in designing upgrades and were addressed in the 

analysis and interface design phases. These dimensions include the following: 

New and changed personnel functions resulting from the upgrade 

Use of upgrades in coordination with the rest of the HSI 

Crew coordination. 

These scenarios should exercise the HSI features developed to address the specific design considerations identified in the 

analysis and interface-design phases. The scenarios should provide opportunities to observe personnel performance under 

high-demand conditions to ensure these HSI characteristics are compatible with cognitive skills and capabilities. 

Validation Testbed- NUREG-0711 and NUREG/CR-6393 (O'Hara, Stubler, Higgins, and Brown, 1997) give extensive 

guidance on the characteristics of the testbed to ensure that the HSI and work environment are represented with an 

appropriate level of fidelity. NUREG-07 11 assumes a high-fidelity simulator will be used to represent the main CR. Seven 

dimensions of fidelity for the main CR testbed are described: HSI completeness, HSI physical fidelity, HSI functional 

fidelity, data completeness fidelity, data content fidelity, data dynamics fidelity, and environment fidelity. 

Existing NPPs are required to have full-scope, high-fidelity simulators for training personnel. They can provide an 

excellent environment for validation studies. However, HSI upgrades are generally not installed in these simulators until 

after the main CR has been upgraded. Licensees are generally required to change the training simulators within a specified 

period following the installation of upgrades. However, they may be required to upgrade the simulators prior to upgrading 

the main CR as a condition for licensing an upgrade if it involves changing technical specifications or specifications in the 

plant's safety analysis report (SAR) or NRC's safety evaluation report (SER). Therefore, depending upon the potential 

safety significance and other licensing considerations, validation of an upgrade may be conducted in the training simulator. 

If the behavior of systems is expected to change after the upgrade, then the plant simulation model should be modified 

5-41 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

accordingly. This simulation may be based on engineering estimates of plant behavior if actual performance data do not 

exist. NUREG-071 I gives criteria for the fidelity of the plant simulation (i.e., data dynamics fidelity) to ensure that the 

plant behavior is accurately portrayed. 

If trials are not performed in the training simulator, then other suitable test beds must be used. The main control room may 

be acceptable, although it may only be suitable for validation trials that require limited levels of fidelity. For example, if 

walk-through exercises are conducted in the main control room, it may be difficult to represent factors that are important to 

performance, such as time stress, changes in conditions, and requirements for the coordinated use of multiple HSI 

components. While such evaluations may be helpful for assessing considerations related to HSI task support verification 

and HFE design verification, they are of limited value for integrated system validation for an HSI upgrade. 

Another approach for validating upgrades is to use part-task simulators, devices that simulate the behavior of user interfaces 

and plant systems for a limited portion of the HSI. Part-task simulators may be appropriate for validating limited upgrades. 

For example, they may be appropriate when the part-task simulator can represent all interactions with HSI components auid 

plant systems related to the upgrade. 

NUREG-0711 indicates that a simulation or mockup should be considered for assessing timely and precise human actions 

undertaken at complex HSI components that are remote from the main CR. The important characteristics of task-related 

HSI components and the task environment should be considered (e.g., lighting, noise, heating and ventilation, and 

protective clothing and equipment). Two types of complex HSI components suitable for this type of evaluation are local 

control centers and maintenance interfaces for digital systems. Brown, Higgins, and O'Hara (1994) discussed HFE 

considerations associated with local control stations. A review on the maintenance of digital systems found that human 

performance is a leading cause of failures of these systems (Stubler, Higgins, and Kramer, 2000). Design features 

associated with errors in maintenance for digital systems include soft controls in the user interface, multiple modes for 

operation and testing, and automatic switching between redundant processors. In addition, the trend toward performing 

maintenance while the plant is at power can increase the likelihood of maintenance errors and their potential consequences 

(e.g., initiation of plant safety systems and plant trips). 

When developing validation testbeds for operational and maintenance tasks at local control stations and local maintenance 

interfaces, the following should be considered: 

Representation of the control and display characteristics, including dynamic plant information, the operation of 

controls, and the behavior of automatic HSI features (e.g., automatic mode changes). 

Representation of system faults 

Representation of HSI faults 

Representation of communications with CR personnel 

Representation of task environmental factors that may affect performance. 

NUREG/CR-6637 5-42

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

5.4.4 Verification and Validation Considerations in Installing Upgrades 

HFE verification and validation may be affected by the way in which an upgrade is implemented. In reviewing Canadian 

NPP event reports, Ragheb (1992) found about 50 events that were caused by, or that occurred during, the design and 

implementation of plant modifications. The causes included the following: 

Faulty change in design 

Use of faulty temporary solutions during delays to the final upgrade 

Failure to verify or test changes after implementation or installation 

Leaving equipment in an unsafe state when making a change 

Improper control of changes in design 

Improper, inadequate, or outdated documentation supporting the design change, such as design manuals, operating 

manuals, and work plans. 

The review underscored the importance of tests and evaluations when developing and implementing upgrades. 

As described in Section 5.1.2, upgrades are not always implemented all at once. Instead, there may be an interim period in 

which the plant differs from both its original configuration and desired final one. Section 5.1.2 describes four different 

patterns for implementing upgrades: complete replacement, phased implementation, dual installation with delayed switch

over, and dual installation with no switch-over. New opportunities for personnel error may occur during the transition from 

the old HSI to the new one. Therefore, when verifying and validating upgrades, the following factors should be considered. 

HFE deficiencies in temporary HSI and plant configurations - Temporary configurations of HSI or system 

components may be created during phased implementations or the dual installation of upgrades. That is, the plant 

may be operated using combinations of HSI components and system components that differ from both the original design and the final design. Because they are temporary, these configurations may not have the same level of HFE 

review as did the original or final configurations. Therefore, the design may have negative effects on performance. For example, the presence of old and new components in the HSI may increase its overall complexity, and so 

impose special demands on personnel for retrieving information and executing control actions. Also, if the 

temporary configuration of HSI systems is not fully compatible with that of plant systems, then the state of the 

plant's equipment may not be accurately conveyed to personnel. 

Inadequate temporary fixes - When the HSI or plant systems are modified, it is sometimes necessary to make 

temporary "fixes" to address problems ("bugs") peculiar to the current configuration. As Ragheb describes, these 

"fixes" may pose new challenges to personnel performance. 

Continuous change - The phased implementation of upgrades may create a period when personnel must 

continually adapt to the changing characteristics of the plant and HSI, imposing high demands on them for 

keeping their knowledge of the plant up-to-date. Performance may be degraded when users apply strategies 

incompatible with the current design.

5 -43 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Improperly planned or implemented changes - As Ragheb describes, plant events may occur as a result of upgrades 

that are inadequately planned or poorly implemented. 

Inadequate procedures and technical documentation - Ragheb also states that plant procedures and other 

supporting documentation may not accurately .reflect the current configuration of an upgrade. When the systems 

and HSI are continually changing, as a phased implementation, this consideration may be particularly important. 

Attention should be given to evaluating these HFE features of temporary configurations of the HSI and plant systems. The 

following are considerations related to HSI task support verification, HFE design verification, and integrated system 

validation: 

HSI Task Support Verification - HSI task support verification should address temporary configurations and deactivated HSI 

components left in the HSI. Temporary configurations should undergo HSI task support verification if they will be used by 

operations or maintenance personnel when the plant is not shut down. In applying the NUREG-07 11 criteria for this 

verification, attention should be given to unique information and control requirements that may result from temporary 

configurations. It should ensure that if the temporary configurations of plant systems have any special characteristics (e.g., 

unique response rates, control modes, or operating ranges) that impose special monitoring or control requirements, then the 

HSI should have the information and control capabilities to satisfy them. Also, this verification should examine the 

characteristics of HSI components that may be installed temporarily to ensure that they also provide all needed control and 

display capabilities. 

Criterion 2 of Section 11.A.2 of NUREG-07 11 states that the HSI should not contain any information, displays, or controls 

that do not support operator tasks. Therefore, this verification should identify deactivated HSI components that may have 

potentially negative effects on performance. Examples include obstructing the view of important information, or adding 

visual clutter that may interfere with monitoring tasks. If further evaluations are needed to assess the severity of effects on 

performance, then these deactivated HSI components should undergo HFE design verification or integrated system 

validation. 

HFE Design Verification - HFE guidance should be applied to the review of temporary configurations, especially to the 

consistency of HSI upgrades with the rest of the HSI. Also, the consistency of "fixes" with HFE principles, standards, and 

guidelines should be examined. 

Integrated System Validation - Validation tests should examine the potential effects on personnel performance stemming 

from the presence of both the old and new components in the HSI. For example, if both are functional, as in the case of 

dual installation with no switch-over, problems may occur due to differences in the methods of operation. Section 5.3.1 

described the potential problems associated with alternating use of inconsistent HSI designs, this may be particularly 

important when the same information or control capabilities can be accessed from different devices with different means of 

operation. Personnel may have difficulty recalling the correct use of one device if the other was used recently. Also, the 

strategies that personnel employ when using a device (e.g., ways of rapidly scanning the displayed information or executing 

control actions) during specific operations may not translate well between different devices and, therefore, may result in 

errors. 

Validation tests should evaluate temporary configurations; considerations include the adequacy of information and control 

capabilities, the integration of controls and displays, and the adequacy of training, procedures, and technical documentation. 

Particular attention should be given to the consistency of these HSI upgrades with the rest of the HSI, and to the conformity 

of temporary "fixes" to HFE principles, standards, and guidelines. 

NUREG/CR-6637 5-44

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Safety significance is an important consideration in determining those conditions that should be evaluated through 

integrated system validation. Two important factors are the risk importance of personnel's actions, and the probability that 

errors may occur. For temporary configurations, the error probability is affected by the time that a particular configuration 

may exist. As personnel's exposure to the temporary configuration is reduced, the likelihood of an error is also reduced. 

Therefore, validation tests may not be required for some temporary configurations if risk-based analyses show that they are 

not risk important. The basis of the analyses may include the length of time that personnel will be exposed to these 

configurations. 

Summary of Evaluation Phase 

The installation and evaluation phase addresses the HFE verification and validation of the final upgrade's design, as 

described in NUREG-07 11. However, the V&V methodology and criteria must be focused on the relevant characteristics of 

upgrades. Particular attention should be given to understanding how personnel tasks will be changed by the upgrades so 

that HFE guidance may be properly applied, to assessing the consistency of the upgrades with the rest of the HSI, and to the 

suitability of any temporary configurations during incremental upgrades. 

5.5 Organizational Considerations for Designing and Implementing HSI Upgrades 

When plants are upgraded or modified, personnel and organizations must adapt to the change. The failure of personnel to 

accept changes in technology in their work environment was identified as a problem in many domains (Medsker and 

Campion, 1997; Price, 1990). In some NPPs, personnel had difficulty adapting to the new HSI technologies when they 

were introduced. The following are two examples: 

A utility introduced a new NPP that featured a CR that was quite different from its predecessor designs. The plant 

had digital control systems and an HSI which prominently featured a CRT-based display system and computer

based input devices (e.g., light pens and keyboards). Many operators came from an older NPP that was being 

permanently shut down. Even though they were involved in developing the CR and many HFE evaluations were 

held, acceptance was not universal, especially in using the HSI (Fenton and Duckitt, 1991). 

A NPP, visited as part of this research project, set up a computer-based system for analyzing, storing, and 

retrieving chemistry data. About six months later, management stated that although the system provided many 

benefits, about one-half of the users had problems with, or expressed some resistance to, the new system. They had 

difficulty with the change in computer technology because the new digital system had characteristics different from 

the old computer system formerly used for chemical analysis. Difficulties in adapting apparently were not related 

to job level, but were evenly distributed among engineers, technicians, and other support personnel (O'Hara, 

Stubler, and Higgins, 1996). 

Resistance to technology changes also has been encountered in fossil-fuel power plants, chemical plants, and 

manufacturing: 

When a multi-unit fossil power plant upgraded some of its control centers, some operators were unable to adapt to 

the CRT-based control centers that replaced the original conventional ones. Consequently, some operators were 

permanently assigned to conventional control centers, while others were rotated between the computer-based and 

conventional control centers (Bames and Ryan, 1995).

5 -45 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

At one multi-unit chemical facility, visited during this research project, operators requested transfers to an older 

unit that had an analog control system, rather than remain at the units that were to be upgraded with digital control 

systems (O'Hara, Stubler, and Higgins, 1996). 

Discussions with personnel from a foreign power-generation organization, also part of this research project, 

indicated that significant resistance from operators was encountered at one of its fossil plants when digital I&C 

systems were installed and the CR was converted to one featuring computer-based operator consoles (O'Hara, 

Stubler, and Higgins, 1996). 

An estimated 50% to 75% of new manufacturing technologies in the United States have failed. The disregard for 

human and organizational concerns was considered a greater contributor to these failures than were technical 

problems (Medsker and Campion, 1997). 

Resistance and nonacceptance of changes in the work environment are of special concern in NPPs because of the 

importance of personnel performance to plant safety. It may result in the failure of personnel to manage or use new 

technology effectively. For example, because of a lack of acceptance of new automated systems, personnel may not actively 

monitor the systems to ensure that the automation is operating correctly. Alternatively, lacking trust in automation, 

personnel may attempt to take the system out of the automatic mode and control it manually. Furthermore, because 

personnel do not accept advanced HSI capabilities, they may be reluctant to undertake interface management to search for 

additional information during transients, but may instead depend upon alarms (O'Hara, Stubler, and Nasta, 1997). Plant 

performance and safety then may be affected. 

From reviewing relevant research, Medsker and Campion offer a set of principles for reducing resistance to change by 

involving personnel in the development of the upgrade. The following were derived from their principles: 

Workers' involvement in the change - Workers should be informed of changes in advance, involved in diagnosing 

current problems, and in developing solutions. Resistance may be decreased if participants feel a sense of 

ownership in the project (i.e, they do not feel it is being imposed upon them). 

Strong support by top management for the change - Workers may be more likely to take the project seriously if 

they feel management is strongly committed to its success. 

Change made consistent with workers' needs and existing values - Workers need to see the change as benefitting 

them. Resistance may be reduced if change is perceived as something that will reduce burdens, offer interesting 

experiences, not threaten their autonomy or security, and not conflict with other goals and values of the 

organization. Resistance also may lessen if proponents of the change empathize with those opposed to it by 

recognizing valid objections and relieving unnecessary fears. 

An environment of open, supportive communication - Resistance may fall if participants trust each other and are 

supported. Misunderstandings and conflicts should be expected as a natural part of the innovation process. 

Provisions should be made for clarifying misunderstandings. 

Allowance for flexibility - Resistance may be reduced if the project remains open to revision and reconsideration as 

experience is gained. 

Price (1990) states that technology alone is unlikely to improve the performance of an organization unless attention is given 

to the people who use it. Change is likely to be rejected if it is poorly understood by users, perceived as being imposed by 

NUREGICR-6637 5-46

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

management or as being risky or threatening, or if it does not improve the work environment. The introduction of 

automation can pose the following particular problems in acceptance: 

Workers' fear of automation that has intellectual functions, such as expert judgement and decision making 

Workers' fear of the unreliability of a device 

Workers being held accountable for the output of an automated system when they feel they have limited control 

over it. 

Price gives a set of principles related to the acceptance of automation by workers, based on an earlier study of automated 

commercial all-weather landing systems. These principles are shown in Table 5.4. They are consistent with those offered 

by Medsker and Campion. 

Experiences in NPPs indicate the importance of making the change consistent with the operator's needs. However, 

experience also shows that obtaining comments from users about their needs can be difficult during the early stages of a 

design project. For example, a NPP that was developing a new computer-based display system wanted early feedback from 

operators about the system (Roth and O'Hara, 1998). They installed an early version of the new display system in the CR so 

that operators could become familiar with its capabilities and see its information within the context of plant operations. 

Because this was an early version, not all data were presented, and, due to some known but uncorrected software "bugs," 

some messages it generated were incorrect. Consequently, operators had difficulty overlooking the limitations and 

envisioning the potential benefits of the fully implemented system. Instead, they expressed concern over the usefulness and 

reliability of the system, based on the limitations. In other words, the system was not viewed as being consistent with their 

needs and existing values. 

Several conclusions can be drawn from this experience. First, it may be undesirable to introduce a display system or other 

personnel aids into a work situation until the design deficiencies have been corrected. Instead, it may be better to introduce 

personnel to the system in a different setting, such as a testing facility. A goal in designing the user interface is to reduce 

the workload associated with its use (i.e., it is often stated that the interface should be "transparent" to the user). When the 

device's capabilities are inoperable or when information displayed is incomplete or incorrect, the workload is likely to 

increase. In addition, when the potential consequences of errors are serious, such as in NPPs, users may react negatively to 

incomplete or incorrect information. Second, this experience also shows the importance of telling people about the 

limitations of early designs. While early involvement by users is generally considered desirable, exposure to early designs 

may also lead to a lack of acceptance. 

A model for the successful transfer and application of new technology in the workplace is offered by Mackie and Wyle 

(1988). The model comprises three major activities which are conducted in parallel: communicating with potential users, 

involving users in the development process, and designing for acceptance. Each is described below. 

Communicating with potential users - This activity begins early in the design process. The intent is to gather 

information from the intended users about their needs and attitudes regarding new technology and keep users 

informed about how the planned technology will affect them. Information gathered from potential users may 

include operational needs, their understanding of new technologies including positive and negative beliefs, and 

features considered desirable by the potential users. Information disseminated to the intended users may include 

operational needs currently being addressed or considered, new features being developed or considered, and 

potential benefits to the users.

5 -47 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Table 5.4 Principles for Users' Acceptance of Automation for Landing Systems 

(Adapted from Price, 1990)

Role Expectancy 

People are generally accepting of system roles that: 

* Provide opportunities to exercise and maintain skills they feel are important to their maintaining their 

position in their occupational and social status system 

* Allow initiative and flexibility in their practices and manner of accomplishing tasks (i.e., work practices are 

not "mechanical") 

* Provide opportunities for and allow learning 

Automation Acceptance 

People are generally more accepting of automation: 

* With increased exposure and experience with the system 

* If their position has status, responsibility, and authority 

* Where failure of the automated function is not associated with high consequences (e.g., does not endanger 

human life) 

* For tasks that must be performed over long periods of time 

* For functions that do not haVe a large decision-making component 

Involving users in the development process - This activity includes users' participation in the identification of 

design inputs and the development of testbeds. Because it requires the participation of individuals having particular 

knowledge relevant to the new product or system, this activity may involve fewer potential users than the previous 

activity - communicating with potential users. Two important considerations from a program management 

perspective are the identification of key personnel from the potential user population and then their utilization. 

Key personnel may participate in the development of design inputs, the review of designs and design concepts, and 

the development of tests for systems or subsystems. In addition, some key personnel may act as change agents by 

providing feedback to the general user population regarding the developing design. For the development of 

testbeds, potential users may be involved in the selection of the test setting (e.g., mockup, simulator, or operational 

setting), the development of test scenarios, and the identification of criteria for selecting test participants. 

Designing for acceptance - This activity relates to the engineering design of the new product or system. Design 

considerations identified through interactions with potential users (as described in the two previous activities) are 

translated into design requirements. Both general criteria for user acceptance and specific criteria related to the 

particular type of innovation are addressed. General design criteria include consideration of operational 

constraints, operability, reliability, compatibility, maintainability, supportability, and training concerns. Specific

NUREG/CR-6637 5 -48

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

criteria address the characteristics of particular design features, such as those for selecting and manipulating 

information. The activity addresses the suitability of these design features for the tasks and characteristics of the 

users. 

The three activities of the Mackie and Wyle model converge during a demonstration phase, which corresponds to the later 

stages of Element 7, Interface Design, of NUREG-07 11. During this phase, testbeds incorporating features that were 

developed during the earlier activities of the process are tested with the participation of potential users. These tests are 

intended to address such considerations as operational validity, operational limitations, reliability, and supportability. 

This process appears to address many of the points raised earlier by Price (1990) and Medsker and Campion (1997). For 

example, involvement of workers in the development process can help focus the new technology on their needs and existing values. The design for acceptance activity can ensure that the design is consistent with other organizational concerns such 

as reliability, maintainability, and supportability. The model does not explicitly deal with the need for the support of top 

management for the change. However, it is implied by the degree of the workers' participation in the process. Also, the 

need for open, supportive communication and for flexibility are not explicitly covered but are implied by the model; 

however, probably both would be necessary when integrating information from the three activities. 

When reviewing an HSI upgrade, some of the considerations discussed here may be easier to assess than others. For 

example, the workers' involvement in planning the change may be assessed by reviewing the design process and noting when operators and maintenance personnel provided input; this may include inputs at such stages as the operating 

experience review, requirements analysis, tests, and evaluations. The flexibility allowed in the development process may be 

assessed by noting the degree to which the HSI design, procedures, and training were modified by user's inputs or operating experiences. Top management's support, communication, and the consistency of the change with the users' needs and 

existing values may be more difficult to assess. 

The concept of participatory ergonomics has received growing interest as a means to enhance the design of complex human

machine systems (Wilson and Haines, 1997; Noro, 1991; Sen, 1987). It is defined as "...the involvement of people in 

planning and controlling a significant amount of their own work activities, with sufficient knowledge and power to 

influence both processes and outcomes in order to achieve desirable goals" (Wilson and Haines, 1997). The potential 

benefits of participatory ergonomics stem from the user's participation in design and implementation. Using participation techniques in change analysis, development, and implementation may generate in the users greater feelings of ownership, 

and subsequently a greater commitment to such changes. Also, involving current users in redesigns may improve the flow 

of information and the generation of ideas during the process because users may be intimately familiar with the positive and 

negative characteristics of the current situation. 

Despite the potential benefits, Wilson and Haines caution that participatory techniques have limitations and potential 

disadvantages. Some limitations are related to the intentions and motivation of participants and facilitators, and others 

have to do with the types of outputs that can be produced by participation. Some situations require professional ergonomic expertise and cannot be adequately addressed by substituting input from users. Finally, Wilson and Haines note that few 

studies have given solid evidence of the cost effectiveness of participatory techniques, while anecdotal evidence exists of 

participatory projects that have fallen short of expectations. 

Thus, a better understanding is needed of the causes of poor operator acceptance, its impact on plant operation and safety, appropriate countermeasures, and appropriate methodologies for dealing with these challenges. These considerations are 

important both for the transition from traditional to computer-based technologies and for subsequent upgrades. The 

flexibility of computer-based systems and the rapid pace at which they become technologically obsolete may increase the 

rate at which HSIs are upgraded in future. New HSI design and evaluation processes may be needed to ensure user 

5 -49 NUREG/CR-6637
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

acceptance, and training processes may have to become more adaptable to the complex, frequently changing computer

based technologies and the diversity of HSI technologies in hybrid plants. Design and implementation of HSI upgrades will 

need to cover the broader range of skills needed to operate hybrid interfaces, the potential for negative transfer of skills 

associated with upgrades, and the resistance of personnel to the associated new demands. 

5.6 Unresolved Issues For Further Research 

This section summarizes the human performance issues associated with designing and implementing NPP upgrades that 

were detailed in this review. These issues represent topics on which research is necessary before additional guidance can be 

developed. 

The Effects of HSI Inconsistency Upon Alternating Use of HSI Components 

Empirical studies have found that human performance may show a greater decrement when different devices in an HSI have 

overlapping similarities in their methods of operation than when these methods are very different (Tanaka et al., 1991). Overlaps in operating computer-based user interfaces can disrupt human performance by increasing the likelihood of errors 

or increasing the amount of time and mental effort needed to generate the correct response. Much of the existing HFE 

guidance on consistency is directed toward the appearance of interfaces, and a greater understanding is needed of the 

dimensions of consistency as related to the operation of computer-based user interfaces. New methods for assessing consistency are being developed, such as the text-editing model (TEM) (Tanaka et al., 1991), for predicting the effects of 

different kinds of computer displays and of interaction-consistent mappings on the performance of computer-based tasks. 

These analytical methods and supporting empirical studies require further review to (1) identify dimensions of inconsistency 

relevant to tasks performed by operations and maintenance personnel in NPPs, and (2) develop techniques appropriate for 

reviewing NPP HSIs. 

The Effects of HSI Design on Crew Coordination and Cooperation 

Computer-based HSI technologies can affect crew coordination in complex ways. For example, the requirements for 

communication and coordination may increase when computer-based display systems and decision aids do not make 

information equally accessible to all members of the crew. Computer-based HSI technologies can also interfere with the 

ability of members to monitor each other's actions to maintain situation awareness and monitor for errors. Dien and 

Montmayeul (1995) state that the HSI design should be compatible with the organizational structure of the crew and support 

manageable workloads while ensuring safety. They specifically identify as topics of concern the use of HSI design to 

support crew coordination and the flexible allocation of tasks between crew members. This first concern was recognized for 

such diverse items as operator-aiding systems in NPPs (Dien and Montmayeul, 1995), group-view displays for NPP CRs 

(Stubler and O'Hara, 1996b), computer-based procedures of NPPs (O'Hara, Higgins, Stubler, and Kramer, 2000), and ship 

bridge design (Hutchins, 1990). However, guidance in this area is evolving slowly, and more research is needed to identify the particular dimensions of HSI design that affect the crew's performance and the degree to which plant performance and 

safety may be affected. 

The Role of Training in HSI Skills 

In many domains, computer-based HSI technologies provide advanced capabilities for presenting and processing 

information, but while they can support the operator's information processing functions, they can also degrade performance. 

For example, they may increase an operator's workload by creating tasks that are not directly related to operating the plant. They also may create control and display configurations that increase the likelihood of errors (Moray, 1992; O'Hara, 

NUREG/CR-6637 5-50

I I
5 TECHNICAL BASIS DEVELOPMENT: DESIGN PROCESS

Stubler, and Nasta, 1997). Reviews of industrial experience and practices suggest that operators often receive little or no 

training in strategies for using these capabilities effectively. Training in using cognitive strategies for accessing information from the HSI can be an effective approach for supporting high levels of operator performance (Mumaw et al., 1994), but such skills are not commonly taught. Performance may benefit from combined training that address both the use 

of HSI capabilities, and skills and strategies for making the best use of information processing capabilities. This is a 

growing area of research and further work is needed to assess training approaches and how far performance may be 

improved. The training recommendations provided by Mumaw et al. (1994) provide some initial direction. 

The Effects of the Installation Process for HSI Upgrades upon Personnel Performance 

Little empirical research has specifically focused on how human performance is affected by installing upgrades in an 

existing HSI. Upgrades may be incorporated in many different ways. In the simplest case, the old equipment is removed at 

the same time that the new is put in. However, industrial experience indicates that the old HSI components are often not 

removed immediately. Instead, a gradual transition may be made in which the new and the old components exist together before the old components are deactivated and removed. In other cases, both the old and new components may remain 

present and functioning in the HSI, giving the user the choice of which to use. In addition, when plant systems are 

modified, additional instrumentation may be temporarily added to the HSI to provide special control and display capabilities 

that reflect the temporary configurations of plant equipment. 

The presence of old non-functional components (i.e., abandoned in place) represents a potential source of clutter, and may distract people's attention during monitoring and information retrieval. If both the old and new components are functional, 

there may be problems due to differences in methods of operating them. The potential problems associated with alternating 

use of inconsistent HSI designs, described in Section 5.3.1, may be particularly important when the same information or 

control capabilities reside in devices that operate differently. For example, personnel may have difficulty recalling the 

correct method for interacting with one device if the other was used recently. Also, strategies for using the devices, such as 

ways for rapidly scanning the displayed information or executing control actions during specific operations, may not 

translate well between devices and, therefore, cause errors. 

One objective of HSI task support verification is to ensure that the HSI does not contain information, controls, and displays that do not support personnel tasks. However, there is little guidance to address the case in which the HSI contains 

components that have been temporarily or permanently deactivated. The general approach suggested by NUREG-07 11 is to 

eliminate such HSI components and information. However, no specific guidance is available for determining when this 

approach is not desirable or needed. Similarly, guidance is sparse for cases where old and new HSI components with 

different methods of operation exist in the same HSI, as it is for instances in which personnel must adapt to a continually changing HSI. Lacking detailed guidance, such topics may be evaluated during integrated system validation. However, 

direct research should be undertaken to address these topics more directly. 

Personnel Acceptance of Upgrades 

Researchers have suggested that personnel acceptance of HSI changes can be enhanced by involving users in their design and evaluation (Medsker and Campion, 1997; Swanekamp, 1995). However, experience in industry suggests that this may 

not always be completely effective or sufficient. Thus, a better understanding is needed of the causes of poor operator 

acceptance, its impact on the plant's operation and safety, and appropriate countermeasures. These considerations are 

important both for the transition from traditional to computer-based technologies, and between subsequent more complex upgrades of the latter. The flexibility of computer-based systems and the rapid pace at which they become technologically obsolete may increase the rate at which HSIs are upgraded in the future. Thus, the user's acceptance of changes in the HSI 

will be a growing concern.

5-51 NUREG/CR-6637
6 DEVELOPMENT OF GUIDANCE 

As described in the Section 1.2, the review of upgrades is included in the Standard Review Plan (SRP) (NUREG-0800). 

NUREG-0800 provides guidance to NRC staff for reviewing NPPs and cites NUREG-07 I1 and Part 1 of NUREG-0700 for 

high-level design process criteria for reviewing overall HFE programmatic goals and objectives. Therefore, guidelines for 

HFE reviews of upgrades were developed according to the ten review elements of NUREG-071 1; they are presented in 

Section 9. They re organized into the following sections: 

General Guidance 

HFE Program Management 

Operating Experience Review 

Functional Requirements Analysis and Function Allocation 

* Task Analysis 

* Staffing 

Human Reliability Analysis 

Human-System Interface Design 

Procedure Development 

Training Program Development 

Human Factors Verification and Validation. 

Within each element, the guidance is further organized into the following four categories: 

Guidance for the Applicability of the NUREG-071I Element - This category includes guidance defining when and 

how an element of NUREG-0711 is applicable to reviewing an upgrade. It is found at the beginning of each 

section addressing a NUREG-071 1 element, and states the following: the conditions under which NUREG-071 I 

element is applicable to an upgrade, the section of that document which covers the element, and overall scope of 

the NUREG-0711 element as it pertains to upgrades. 

Guidance Tailoredfrom NUREG-0711for Upgrades - This category includes modified guidance from NUREG

0711 that focuses on characteristics and considerations relevant to upgrades. Because NUREG-07 lI's scope is 

broad, reviewers may have difficulty interpreting the guidelines in the context of upgrades. This category focuses 

the guidance of NUREG-0711 on specific aspects of upgrades that may be important to human performance and 

safety. However, this category does not repeat those guidelines from NUREG-071 1 that can be directly applied to 

upgrades without special interpretation. 

New Guidance for Upgrades Only - This category includes guidance specifically relevant to upgrades that does not 

currently appear in NUREG-0711.

6-1 NUREG/CR-6637
6 GUIDANCE DEVELOPMENT

New Guidance for the NUREG-0711 Element - The development of review guidance for upgrades identified some 

considerations with broad applicability. They are suggested as possible additions to the more general guidance of 

NUREG-071 1. 

When reviewing an upgrade, the NRC reviewer should first follow NUREG-0711 and then consider the guidance in 

Section 9 which focuses on special concerns for upgrades.

NUREG/CR-6637 6-2

I I
7 SUMMARY 

The objective of this study was to formulate HFE review guidance for NPP upgrades to plant systems and the HSI, based on 

a technically valid methodology for developing guidelines. Several tasks were performed, including the following: 

* A technical basis was established using human performance research and analyses of upgrades, 

* HFE review guidelines were developed in a format consistent with NUREG-071 1, and other NRC guidance, and 

* Remaining issues were identified for which research is insufficient to support the development of NRC review 

guidance. 

The status of each is briefly discussed below. 

Technical Basis Development 

The effects of upgrades on personnel performance were addressed by examining basic HFE literature, literature on complex 

human-machine systems, and industry experience gained from site visits, interviews, and industrial literature. The 

resulting technical basis was described in two sections. Section 4 reviewed human performance in complex human

machine systems, and discussed the types of knowledge and skills that must be adapted to a new work design after an 

upgrade. Section 5 was a more detailed discussion of these considerations within the context of the NUREG-07 11 review 

process. Specific considerations were identified for elements of the NUREG-07 11 review. 

HFE Review Guidelines 

Guidance for reviewing upgrades was developed to address the design process. The SRP specifies NUREG-0711 as the 

process used for HFE reviews of design processes. Therefore, guidance for upgrades was organized in Section 9 according 

to the ten review elements ofNUREG-071 1. Within each element, guidance was further organized into four sections. The 

first describes the conditions under which the particular NUREG-0711 element is relevant to the review of upgrades. The 

second category includes modified guidance from NUREG-0711 that focuses on characteristics and considerations for 

upgrades. The guidance in the third category is specifically relevant to upgrades, but does not appear in NUREG-0711. 

The fourth category has considerations that have potential applications beyond upgrades, and are possible additions to the 

more general guidance of NUREG-0711. 

Upgrade Issues Requiring Additional Research 

Several human performance issues associated with upgrades were identified in Section 5.6. They represent topics for which 

research is necessary before more guidance can be developed: 

The Effects of HSI Inconsistency on Alternating Use of HSI Components 

The Effects of HSI Design on Crew Coordination and Cooperation 

The Role of Training in HSI Skills 

The Effects of the Installation Process for HSI Upgrades on Personnel Performance 

Personnel Acceptance of Upgrades.

7-1 NUREG/CR-6637
8 REFERENCES 

Barnes, V. and Ryan, J. (1995). Advanced digital man-machine interface analysis. Maryland: Ryan Nuclear, Inc. 

Brown, W., Higgins, J., and O'Hara, J. (1994). Local control stations: Human engineering issues and insights 

(NUREG/CR-6146). Washington, DC: U.S. Nuclear Regulatory Commission. 

Card, S., Moran, T., and Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum. 

Childs, J. (1996). Training system evaluation. In T. O'Brien and S. Charlton (Eds.), Handbook of human factors testing 

and evaluation. Hillsdale, NJ: Erlbaum. 

Dien, Y. and Montmayeul, R. (1995). Taking account of human factors in control room design. In Proceedings of the 

Topical Meeting on Computer-Based Human Support Systems: Technology, Methods, and Future. LaGrange Park, IL: 

American Nuclear Society. 

Eberts, R. (1994). User interface design. Englewood Cliffs, NJ: Prentice Hall. 

EPRI (1993). Guideline on licensing digital upgrades (EPRI TR-102348). Palo Alto, CA: Electric Power Research 

Institute. 

Fenton, E. and Duckitt, W. (1991). The Darlington control room and operator interface. In Proceedings of the Human 

Factors Society 3 5ih Annual Meeting. Santa Monica, CA: Human Factors Society. 

Galletti, G. (1996). Human factors issues in digital system design and implementation. In Proceedings of the 1996 

American Nuclear Society International Topical Meeting on Nuclear Plant Instrumentation, Control, and Human-Machine 

Interface Technologies. La Grange Park, IL: American Nuclear Society. 

Goldstein, I. (1987). The relationship of training goals and training systems. In G. Salvendy (Ed.), Handbook of human 

factors and ergonomics (2nd ed.). New York: Wiley. 

Heslinga, G. and Herbert, M. (Eds.) (1993). Experiences with introduction of new systems for human-machine interaction 

from a human factors perspective. The Netherlands: KEMA. 

Higgins, J. and Pope, N. (1996). Human factors aspects of the major upgrade to the control systems at the LANL plutonium facility. In Proceedings of the IEEE Conference on Global Perspective of Human Factors in Power Generation. New 

York: IEEE. 

Holding, D. (1987). Concepts of training. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (2nd ed.). 

New York: Wiley. 

Hollnagel, E. (1992). The reliability of interactive systems: Simulation based assessment. In J. A. Wise, V. D. Hopkin, and 

P. Stager (Eds.), Verification and validation of human-machine systems. New York and Berlin: Springer-Verlag. 

Hutchins, E. (1990). The technology of team navigation. In J. Gallagher, R. Kraut, and C. Edido (Eds.), Intellectual 

teamwork: social and technological foundations for cooperative work. Hillsale, NJ: Erlbaum. 

Klein, G., Calderwood, R., and MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE 

Transactions on Systems, Man, and Cybernetics, 19, 462-472.

8-1 NUREG/CR-6637
8 REFERENCES

Lewis, C. and Norman, D. (1986). Designing for error. In D.A. Norman and S.W. Draper (Eds.), User-centered system 

design. Hillsdale, NJ: Erlbaum. 

Mackie, R. and Wyle, C. (1988). Factors influencing acceptance of computer-based innovations. In G. Salvendy (Ed.), 

Handbook of human-computer interaction. New York: Elsevier Science Publishers B. V. (North-Holland). 

Mann, T. and Hammer, J. (1986). Analysis of user procedural compliance in controlling a simulated process. IEEE 

Transactions on Systems, Man, and Cybernetics, 16, 505-5 10. 

Medsker, G. and Campion, M. (1997). Job and team design. In G. Salvendy (Ed.), Handbook of human factors and 

ergonomics (2nd ed.). New York: Wiley. 

Miller, G. (1956). The magical number seven, plus or minus two; some limits in our capacity for processing information. 

Psychological Review, 63, 81-97. 

Moray, N. (1992). Flexible interfaces can promote operator error. In H. Kragt (Ed.), Enhancing industrialperformance: 

Experiences of integrating the human factor. Washington, DC: Taylor and Francis. 

Mumaw, R. and Gabrys, G. (1996). Cognitive skills requirements for authorized operating personnel at CANDU nuclear 

generating stations (Tech. Report INFO-0664). Pittsburgh, PA: Westinghouse Science and Technology Center. 

Mumaw, R., Swatzler, D., Roth, E., and Thomas, W. (1994). Cognitive skill training for nuclear power plant operational 

decision making (NUREG/CR- 6126). Washington, DC: U.S. Nuclear Regulatory Commission. 

Nagel, D. (1988). Human error in aviation operations. In E. Wiener and D. Nagel (Eds.), Human factors in aviation. New 

York: Academic Press. 

Norman, D. (1988). The psychology of everyday things. New York: Basic Books. 

Norman, D. (1983). Design rules based on analyses of human error. Communications of the ACM, 26, 254-258. 

Norman, D. (1981). Categorization of action slips. Psychological Review, 88, 1-15. 

Noro, K. (1991). Concepts, methods, and people. In K. Noro and A. S. Imada (Eds.), Participatory ergonomics. London: 

Taylor and Francis. 

NRC (1998). An approach for using probabilistic risk assessment in risk-informed decisions on plant-specific changes to 

the licensing basis (Regulatory Guide 1.174). Washington, DC: U.S. Nuclear Regulatory Commission. 

NRC (1996). Standard review plan for the review of safety analysis reports for nuclear power plants (NUREG-0800). 

Washington, DC: U.S. Nuclear Regulatory Commission. 

NRC (1995a). Training and qualification effectiveness (NRC Inspection Procedure 41500). Washington, DC: U.S. Nuclear 

Regulatory Commission. 

NRC (1995b). Digital retrofits receiving prior approval (NRC Inspection Procedure 52001). Washington, DC: U.S. 

Nuclear Regulatory Commission. 

NUREG/CR-6637 8-2

I I
8 REFERENCES

NRC (1995c). Digital retrofits not receiving prior approval (NRC Inspection Procedure 52002). Washington, DC: U.S. 

Nuclear Regulatory Commission. 

NRC (1995d). Use of NUMARCIEPRI Report TR-102348, Guideline on licensing digital upgrades, determining the 

acceptability ofperforming analog-to-digital replacements under 10 CFR 50.59 (NRC Generic Letter 95-02). Washington, 

DC: U. S. Nuclear Regulatory Commission. 

NRC (1995e). Maintenance rule (NRC Inspection Procedure 62706). Washington, DC: U.S. Nuclear Regulatory 

Commission. 

O'Hara, J. (1994). Advanced human-system interface design review guideline (NUREG/CR-5908, Vol. 1). Washington, 

DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J. and Brown, W. (1994). Advanced alarm systems and human performance (Tech. Report No. A-3967-11/94). 

Upton, NY: Brookhaven National Laboratory. 

O'Hara, J., Brown, W., Higgins, J., and Stubler, W. (1994). Human factors engineering guidelines for the review of 

advanced alarm systems (NUREG/CR-6105). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Brown, W., and Nasta, K. (1996). Development of the human-system interface design review guideline (NUREG-0700, Revision 1) (Tech. Report No. L- 1317-2-12/96). Upton, NY: Brookhaven National Laboratory. 

O'Hara, J., Brown, W., Stubler, W., Wachtel. J., and Persensky, J. (1996). Human-system interface design review 

guideline (NUREG-0700, Rev. 1). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Higgins, J., and Kramer, J. (2000). Advanced information systems design: Technical basis and human factors 

review guidance (NUREG/CR-6633). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Higgins, J., Stubler, W. and Kramer, J. (2000). Computer-basedprocedures: Technical basis and human 

factors review guidance (NUREG/CR-6634). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Higgins, J., Stubler, W., Goodman, C., Eckenrode, R., Bongarra, J., and Galletti, G. (1994). Human factors engineering program review model (NUREG-07 11). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Stubler, W., and Higgins, J. (1996). Hybrid human-system interfaces: Human factors considerations (BNL 

Technical Report J60"12-T1-4/96). Upton, NY: Brookhaven National Laboratory. 

O'Hara, J., Stubler, W., Higgins, J., and Brown, W. (1997). Integrated system validation: Methodology and review criteria 

(NUREG/CR-6393). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Stubler, W., and Nasta, K. (1997). Human-system interface management: Effects on operator performance and 

issue identification (BNL Letter Report W-6546-1-1-7/96). Upton, NY: Brookhaven National Laboratory. 

Oser, R., McCallum, G., Salas, E., and Morgan, B. (1989). Toward a definition of team work.: An analysis of critical team 

behaviors (Technical Report 89-004). Human Factors Division, Naval Training Systems, Center. 

Osgood, C. (1949). The similarity paradox in human learning: A resolution. Psychological Review, 56, 132-143.

8-3 NUREG/CR-6637
8 REFERENCES

Pew, R., Miller, D., and Feeher, C. (1981). Evaluation ofproposed control room improvements through analysis of critical 

operator decision aids (EPRI NP- 1982). Palo Alto, CA: Electric Power Research Institute. 

Price, H. (1990). Conceptual system design and the human role. In H. R. Booher (Ed.), MANPRINT: An approach to 

systems integration. New York: Van Nostrand Reinhold. 

Ragheb, H. (1992). Backfitting of NPPs: Lessons learned from operational incidents, Presented at the eleventh annual 

meeting of CSNI principal working group no. 1. Paris: OECD Nuclear Energy Agency. 

Rasmussen, J. (1986). Information processing and human-machine interaction: an approach to cognitive engineering. 

New York: North-Holland. 

Rasmussen, J. and Goodstein, L. (1988). Information technology and work. In M. Helander (Ed.), Handbook of human

computer interaction. New York: Elsevier (North-Holland). 

Reason, J. (1990). Human error. New York: Press Syndicate of the University of Cambridge. 

Roth, E. and Mumaw, R. (1995). Using cognitive task analysis to define human interface requirements for first-of-a-kind 

systems. In Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Roth, E., Mumaw, R., and Lewis, P. (1994). An empirical investigation of operator performance in cognitively demanding 

emergencies (NUREG/CR-6208). Washington, DC: U. S. Nuclear Regulatory Commission. 

Roth, E. and O'Hara, J. (1998). Integrating digital and conventional HSls: Lessons learnedfrom a control room 

modernization program (BNL Letter Report J6012-T3-4-5/98). Upton, NY: Brookhaven National Laboratory. 

Roth, E. and Woods, D. (1988). Aiding human performance: I. cognitive analysis. Le Travail Humain, 51, 39-64. 

Roth, E., Woods, D., and Pople, H. (1992). Cognitive simulation as a tool for cognitive task analysis, Ergonomics, 35, 

1163-1198. 

Rouse, W. (1990). Designing for human error: concepts for error tolerant systems. In H. R. Booher (Ed.), MANPRINT: an 

approach to systems integration. New York: Van Nostrand Reinhold. 

Salas, E., Dickinson, T., Converse, S., and Tannenbaum, S. (1992). Toward an understanding of team performance and 

training. In R. W. Swezey and E. Salas (Eds.), Teams: Their training and performance. Norwood, NJ: Ablex. 

Sanquist, T., Lee, J., and McCallum, M. (1995). Methods for assessing training and qualifications needs for automated 

ships. In Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Sawyer, C., Pain, R., Van Cott, H., and Banks, W. (1982). Nuclear control room modifications and the role of transfer of 

training principles: A review of issues and research (NUREG/CR-2828). Washington, DC: U.S. Nuclear Regulatory 

Commission. 

NUREG/CR-6637 8-4

I I
8 REFERENCES

Schlager, M., Roth, C., Rolling, W., Mumaw, R., and Gagne, E. (1989). AAS transition training study (OPM work order 

8042-011). Alexandria, VA: HumRRO International, Inc. 

Schneider, W. (1985). Training high-performance skills: Fallacies and guidelines. Human Factors, 27, 285-300. 

Sen, T. (1987). Participative group techniques. In G. Salvendy (Ed.), Handbook of human factors and ergonomics. New 

York: Wiley. 

Stubler, W., Higgins, J., and Kramer, J. (2000). Maintenance of digital systems: Technical basis and human factors review 

guidance (NUREG/CR-6636). Washington, DC: U.S. Nuclear Regulatory Commission. 

Stubler, W., O'Hara, J., and Kramer, J. (2000). Soft controls: Technical basis and human factors review guidance 

(NUREG/CR-6635). Washington, DC: U.S. Nuclear Regulatory Commission. 

Stubler, W. and O'Hara, J. (1 996a). Proposed guidance development for hybrid human-system interface issues (BNL 

Technical Report J6012-T3-10/96). Upton, NY: Brookhaven National Laboratory. 

Stubler, W. and O'Hara, J. (I 996b). Group-view displays: Functional characteristics and review criteria (BNL Technical 

Report E2090-T4-4-12/96, RI). Upton, NY: Brookhaven National Laboratory. 

Stubler, W., Higgins, J., and O'Hara, J. (1996). Evaluation of the potential safety significance of hybrid human-system interface topics (BNL Technical Report J6012-T2-6/96). Upton, NY: Brookhaven National Laboratory. 

Swanekamp, R. (1995). Power plant automation. Power, 139 (2), 17-25. 

Swezey, R. and Llaneras, R. (1997). Models in training and instruction. In G. Salvendy (Ed.), Handbook of human factors 

and ergonomics. New York: Wiley. 

Tanaka, T., Eberts, R., and Salvendy, G. (1991). Consistency of human-computer interface design: Quantification and 

validation. Human Factors, 33, 653-676. 

US. Code of Federal Regulations, Part 50.59, Changes, tests, and experiments, Title 10, Energy (revised periodically). 

Washington, DC: U.S. Government Printing Office. 

Vicente, K. and Bumes, K. (1995). A field study of operator cognitive monitoring at the Pickering Nuclear Generation 

Station - B (Tech. Report CEL 95-04). Toronto, Canada: University of Toronto, Department of Industrial Engineering, 

Cognitive Engineering Laboratory. 

Vicente, K., Mumaw, R. and Roth, E. (1997). Cognitive functioning of control room operators -finalphase. (Report 

prepared for the Atomic Energy Control Board). Toronto: University of Toronto. 

Vicente, K. and Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions on 

Systems Man, and Cybernetics, 22, 589-606. 

Waugh, N. and Norman, D. (1965). Primary memory. Psychological Review, 72, 89-104.

8-5 NUREG/CR-6637
8 REFERENCES

Webb, J. and Kramer, A. (1990). Maps or analogies? A comparison of instructional aids for menu navigation. Human 

Factors, 32,251-266. 

Wickens, C. and Carswell, C. (1997). Information processing. In G. Salvendy (Ed.), Handbook of human factors and 

ergonomics (2nd ed.). New York: Wiley. 

Wilson, J. and Haines, H. (1997). Participatory ergonomics. In G. Salvendy (Ed.), Handbook of human factors and 

ergonomics (2nd ed.). New York: Wiley.

NUREG/CR-6637 8-6

I I
PART 2 

Review Guidelines: Hybrid HSI Design Process
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI 

The guidelines for the design process were developed to address important considerations identified in the literature and to 

provide a means whereby human performance issues may be assessed during a design review (see Section 6). The review 

guidelines were formatted to correspond to the NRC's general design process guidance in NUREG-07 11. They are 

organized into the following sections: 

* General Guidance 

- HFE Program Management 

* Operating Experience Review 

. Functional Requirements Analysis and Function Allocation 

• Task Analysis 

"* Staffing 

• Human Reliability Analysis 

. Human-System Interface Design 

• Procedure Development 

* Training Program Development 

* Human Factors Verification and Validation. 

Some guidelines specify that particular aspects of an upgrade should be "evaluated." General approaches to evaluation and 

review criteria are defined in NUREG- 0700; such evaluations are not described here. 

9.1 General Guidance 

(1) An HFE review should be conducted if the upgrade2 of a plant system or the HSI affects the role of personnel or the 

tasks by which their role is performed and is potentially significant to plant safety. Upgrades affect the role or 

tasks of personnel if they impose new or different demands on them to operate, maintain, or otherwise ensure 

safety. An upgrade may be considered potentially significant to plant safety if it 

* Constitutes an unreviewed safety question, as defined in 10 CFR 50.59, or 

* Is a modification of a structure, system, or component (SSC) that is safety related, or 

• Is a modification of a non-safety SSC that (a) mitigates accidents or transients, (b) is used in emergency 

operating procedures, or (c) could prevent a safety-related SSC from fulfilling its function. 

2The term upgrade is used generically to include any type of change, modification, or retrofit made to HSI 

components or plant systems that may influence personnel performance.

9-1 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

The goal of HFE reviews is to ensure plant safety. An upgrade may constitute an unreviewed safety question (10 

CFR 50.59, EPRI, 1993) if it may 

increase the probability of an accident evaluated in the Safety Analysis Report (SAR) 

increase the consequences of an accident evaluated in the SAR 

increase the probability of a malfunction of equipment important to safety evaluated in the SAR 

increase the consequences of a malfunction of equipment important to safety evaluated in the SAR 

create the possibility of an accident of a different type than any evaluated in the SAR 

create the possibility of a malfunction of equipment important to safety when such a malfunction differs 

from any evaluated previously in the SAR 

reduce the margin of safety as defined in the basis for any technical specification. 

Discussion: The criteria covering structures, systems, or components were adapted from the Maintenance Rule, 

NRC Inspection Procedure 62706 (NRC, 1995e). 

9.2 HFE Program Management 

Guidance for the Applicability of the NUREG-0711 Element 

(1) The HFE program management review should be conducted, in accordance with Section 2 of NUREG-07 11, for 

upgrades identified in Section 9.1. The licensee should prepare an HFE program, which is a technical plan 

describing how HFE considerations will be addressed in the design, development, and implementation of the 

upgrade. The scope of the HFE program should be consistent with the scope of the upgrade. 

Discussion: See NUREG-071 1, Section 2. 

Guidance Tailoredfrom NUREG-0711for Upgrades 

(2) The HFE program should ensure that the HFE requirements are specified in agreements with vendors or 

subcontractors, who will be providing upgrades, as described in Criterion 6 of Section 2.4.3 of NUREG-0711. 

These requirements should ensure that new components adhere to the principtes of good HFE design, including 

compatibility with the existing HSI. Criteria for selecting off-the-shelf equipment should include human factors 

requirements. If the upgrade will involve acquiring equipment that may contain human factors discrepancies, the 

HFE program should ensure that analyses and evaluations will identify potential problems, assess their potential 

importance to plant safety, and identify and implement solutions for ensuring plant safety. 

Discussion: This criterion is important for upgrades since add-ons or system modifications often are purchased as 

modular units designed to the standard specifications of the vendor, rather than the plant's specifications. In cases 

involving off-the-shelf components, factors such as the availability of vendors, technological compatibility, and 

constraints on proprietary information, may necessitate obtaining equipment that has human factors discrepancies; 

the HFE program should address them. 

NUREG/CR-6637 9-2

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

New Guidance for Upgrades Only 

(3) The HFE program should ensure that the knowledge and experience of plant personnel, who will use the upgrade, will be incorporated in developing and implementing the upgrade. The development may include designing new 

components or selecting off-the-shelf ones. The HFE program should involve plant personnel to ensure that the 

following are considered from a user's perspective in establishing upgrade requirements and evaluating the results 

of the design process: 

User's understanding (mental model) of how plant systems are structured and behave 

Task demands and constraints of the existing work environment 

Existing work processes 

Organizational goals that affect the implementation and use of the upgrade. 

Discussion: Plant personnel who will be the ultimate users of the upgrade can be valuable sources of domain

specific information that should be considered when developing and evaluating upgrades (e.g., the design requirements and evaluation criteria). Their involvement can help ensure that the final design of the upgrade is 

consistent with organizational concerns, such as reliability, maintainability, and supportability (Medsker and 

Campion, 1997; Price, 1990). (See Sections 5.1 and 5.5.) 

(4) The goals of the HFE program should address the need to consider the effects that the upgrade may have on the 

performance of personnel. The transition from the existing plant configuration to the upgrade configuration 

should be planned so that adapting to the change imposes minimal demands. The considerations should include 

the following: 

Planning the installation to minimize disruptions to ongoing operations 

Coordinating changes to training materials and procedures to ensure that they accurately reflect the 

systems being upgraded 

Conducting training to maximize personnel's knowledge and skill with the new design before its 

implementation. 

Discussion: Industrial experience showed that temporary HSI and plant configurations that occur when upgrades 

are established over a period can pose demands on human performance that differ from either the initial or final 

configurations. (See Section 5.4.) 

New Guidance for the NUREG-0711 Element 

(5) HFE considerations identified when implementing a new or upgraded design of plant systems or HSI components, 

or during its subsequent operation, should be recorded in an HFE issues tracking system as described in Section 

2.4.4 ofNUREG-0711. 

Discussion: HFE issues not detected during HFE verification and validation may become apparent during plant 

operation. The criteria in Section 2.4.4 ofNUREG-071 1 should be expanded to address subsequent issues.

9-3 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.3 Operating Experience Review 

Guidance for the Applicability of the NUREG-0711 Element 

(1) This review should be conducted in accordance with Section 3 of NUREG-0711 for all upgrades identified in 

Section 9.1. The scope of the operating experience review (OER) should emphasize the systems or HSI components 

that are being modified. 

Discussion: The OER should be focused to provide information relevant to the upgrade. 

Guidance Tailored from NUREG-0 71 ]for Upgrades 

(2) The OER, as described in Section 3.4.1 ofNUREG-071 1, should address the operating experience of the plant that 

will be upgraded, including experiences with the systems that will be upgraded and with HSI technologies that are 

similar to those under consideration for the upgrade. Also, when operators and maintenance personnel are 

unfamiliar with the proposed technology, attention should be paid to the operating experience of other plants that 

already have the technology. 

Discussion: This guideline is derived from the criteria of Section 3.4.1 of NUREG-07 11, and focuses on 

considerations related to upgrading existing plants. 

New Guidance for the NUREG-0711 Element 

(3) The OER should identify risk-important tasks that have been prone to errors. These tasks should receive special 

attention during the design of the user interface to lessen their probability. 

Discussion: When changing the user interface from analog to digital technology, the likelihood of some types of 

errors may be increased. Where their potential consequences are high, design approaches should be used to protect 

against them (Stubler, O'Hara, and Kramer, 2000). 

9.4 Functional Requirements Analysis and Function Allocation 

Guidance for the Applicability of the NUREG-0 711 Element 

(1) Functional requirements analyses should be reviewed in accordance with Section 4 of NUREG-07 11, for all 

upgrades identified in Section 9.1 that are likely to change existing functions that are important to safety, introduce 

new functions for systems supporting safety functions, or involve unclear functional requirements that may be 

important to safety. The functional requirements analysis should address new functions resulting from changes in 

the degree of integration between plant systems. For example, installing higher-level automation may bring 

systems that were formerly controlled separately under a single controller. Also, the upgrades may change the 

degree to which different plant systems share common resources (e.g., power sources, cooling water, and data

transmission buses). These may be important in diagnosing malfunctions or planning responses. The functional 

requirements analyses should be revised and updated to reflect requirements of the upgrade; the scope of the 

analyses may be restricted to functions related to the upgrade. 

Discussion: Upgrades, such as increased automation of systems, may change the role of personnel. For example, 

they may create new functions, such as detecting malfunctions of the new systems. (See Sections 5.2.1 and 4.4.) 

(2) Function allocation analyses should be reviewed, in accordance with Section 4 ofNUREG-071 1, for all upgrades 

identified in Section 9.1 that are likely to change the allocation between personnel and plant systems of functions 

NUREG/CR-6637 9-4

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

important to safety. The analyses should be revised and updated to reflect requirements for the upgrade; their 

scope may be restricted to functions involving the upgrade. 

Discussion: Upgrades, such as increased levels of automation in plant systems, may change the allocation of 

control functions between the plant's systems and operators. The operator's role in controlling systems manually 

may be more demanding when it is performed as a backup to an automated system than when the systems are 

always manually controlled. (See Sections 5.2.1 and 4.4.) 

Guidance Tailoredfrom NUREG-0711 for Upgrades 

(3) A change in an operator's role due to an upgrade should be examined within the context of its effects on the 

operator's overall responsibilities. Increases in certain task demands may affect the ability of the operator to carry 

out others that are risk important. 

Discussion: This guideline is adapted from NUREG-0711. 

9.5 Task Analysis 

Guidance for the Applicability of the NUREG-0711 Element 

(1) This review should be conducted, in accordance with Section 5 of NUREG-07 11, for all upgrades identified in 

Section 9.1 that are likely to affect tasks previously identified as risk important, cause existing ones to become risk 

important, or create new tasks that are risk important. The task analyses should be revised and updated to reflect 

requirements of the upgrade; the scope should include tasks involving the upgrade and its interactions with the rest 

of the plant, including those resulting from functions addressed in the analyses of functional requirements and 

function allocation. For maintenance, tests, inspections, and surveillances, attention should be given to risk

important tasks that are new or supported by new technologies (e.g., new capabilities for on-line maintenance). 

Discussion: A task analysis may not be necessary if people's tasks do not change. (See Section 5.2.2.) New 

capabilities for maintenance are described in Stubler, Higgins, and Kramer (2000). 

Guidance Tailoredfrom NUREG-0711 for Upgrades 

(2) Task analyses should identify user strategies for processing information and executing actions that are needed to 

perform successfully over the anticipated range of workload. Strategies identified in the existing HSI should be 

examined to understand task demands under the current design. These demands should be taken into account 

when developing design requirements for the upgrade. 

Discussion: Task analysis methods that only focus on observable actions may ignore cognitive skills important to 

skilled performance in highly demanding situations (e.g., high time pressure, high information content, and 

rapidly changing conditions) (Mumaw et al., 1994; Klein et al., 1989). The failure to consider cognitive skills 

required for a job may result in simplistic assumptions about personnel's tasks and the development of inadequate 

design requirements. (For more information, see Changes in Personnel Tasks and Cognitive Task Analysis in 

Section 5.2.2.) 

(3) Routine tasks, which may benefit from the ability to undertake actions without highly focused attention, and tasks 

requiring similar actions should be identified so they may be addressed consistently when designing the user 

interface. In this way, similar user interfaces may be provided for similar actions. 

Discussion: The ability to act without paying close attention can enhance a person's performance by freeing mental resources for higher-level activities (Schneider, 1985; Mumaw and Gabrys, 1996; Schlager et al., 1989).

9-5 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

Overall performance may suffer if the new upgrade does not support automaticity for low-level tasks but, instead, 

increases task demands. (For additional information, see Performing Tasks without Highly Focused Attention in 

Section 5.2.2 and Automaticity of Action in Section 4.3.2.) 

New Guidance for Upgrades Only 

(4) The task analysis should identify the design characteristics of the existing HSI that support the superior 

performance of experienced personnel (e.g., support high levels of performance during demanding situations). 

Such characteristics, which may include the spatial arrangement of control and display devices and the ability to 

adjust controls and displays to deal with special tasks, should be considered in developing new design 

requirements. The new design should have features performing similar functions, or should eliminate the need for 

them by performing these functions differently. In addition, the task analysis should identify and examine 

adjustments made to the HSI by users, such as notes and external memory aids, which suggest the users' needs may 

not be fully met by its current design. All task demands should be adequately addressed by the new design 

requirements. Design features identified during OERs should be considered in these analyses. 

Discussion: HSI adjustments made by users to fulfil task demands may highlight aspects of their tasks that are 

particularly challenging and may not be fully supported by the design of the HSI (Schlager et al., 1989). (For 

additional information, see HSI Tailoring in Section 5.2.2.) 

New Guidance for the NUREG-0711 Element 

(5) The task analysis should identify tasks especially prone to errors. This analysis should include tasks identified 

during the OER as being associated with errors. Error prone tasks should be designated as requiring special 

attention during the design of the user interface to protect against errors. 

Discussion: Computer-based user interfaces may be especially prone to certain types of errors. When changing 

the user interface to a computer-based technology, the likelihood of some types of errors may increase. Where the 

potential consequences of such errors are high, design approaches should be used to protect against them (Stubler, 

O'Hara, and Kramer, 2000). 

9.6 Staffing 

Guidance for the Applicability of the NUREG-0711 Element 

(1) This review should be conducted in accordance with Section 6 of NUREG-0711 for all upgrades identified in 

Section 9.1 that are likely to change staffing requirements. The scope of the analysis should address staffing 

demands that result from the upgrade and its interactions with the rest of the plant. An upgrade may be likely to 

change staffing requirements if it does any of the following: 

Changes the roles of operators, as determined by the function analysis 

Decreases the availability of operators (e.g., increases demands for tasks outside of the CR) 

Decreases the ability to coordinate work 

Increases demands for operators to coordinate their work 

NUREG/CR-6637 9-6

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

Decreases the availability or accessibility of information needed by operators 

Changes the levels of knowledge, skill, and experience from those specified by current qualification 

requirements. 

Discussion: Changes in plant systems or the HSI may impose new requirements for interactions between CR 

personnel or for personnel knowledge and skills which exceed the assumptions used to establish current staffing levels (Hutchins, 1990). (For additional information, see Section 5.2.3.) 

9.7 Human Reliability Analysis 

Guidance for the Applicability of the NUREG-0711 Element 

(1) This review should be conducted in accordance with Section 7 of NUREG-07 11 for all upgrades identified in 

Section 9.1 that may affect tasks previously identified as risk important, cause existing tasks to become risk 

important, or create new ones that are risk important. The scope of the human reliability analysis should address 

personnel actions resulting from the upgrade and its interactions with the rest of the plant. Discussion: Changes in the HSI or plant systems may change the risk importance of tasks, so that conclusions 

drawn from previous HRAs may be inaccurate. 

Guidance Tailored from NUREG-0 71 ]for Upgrades 

(2) When upgrading plant systems or the HSI, consideration should be given to the following effects of these 

modifications on the existing HRA: 

Whether the original HRA assumptions are valid for the upgraded design 

Whether the human errors analyzed in the existing HRA are still relevant 

Whether the probability of errors by operators and maintenance personnel may change 

Whether errors may be introduced that are not modeled by the existing HRA and PRA 

Whether the consequences of errors, established in the existing HRA, may change. 

Discussion: HRAs often are based on assumptions about operators' ability to detect a fault and begin a manual 

response within a specific time. Changes to plant systems may alter the bases of these assumptions, and affect each 

of the considerations listed above. For example, introducing new levels of automation may make the operator's role of detecting and responding to malfunctions more challenging. Changes to the HSI may lessen the operators' ability to access information and controls quickly and accurately (O'Hara, Stubler and Nasta, 1997). In addition, 

changes to systems and the HSI may affect their ability to carry out maintenance and operations correctly (Stubler, 

O'Hara, and Kramer, 2000; Stubler, Higgins, and Kramer, 1998). 

(3) If the upgrade involves new or existing human actions that are risk important, then these actions should be re

examined as described in Criterion 3 of Section 7.4 of NUREG-071 1, and addressed accordingly. 

Discussion: Section 7.4 ofNUREG-071 I gives criteria for reviewing HRAs.

9-7 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.8 Human-System Interface Design 

Guidance for the Applicability of the NUREG-0711 Element 

(1) All upgrades identified in Section 9.1 should be reviewed, in accordance with Section 8 ofNNUREG-0711 when 

they change existing HSI components used for risk-important tasks, or add new HSI components to risk-important 

tasks. The scope, specified in Section 8.4.1.1 NUREG-07 11, may be limited to the HSI upgrade and its 

interactions with the other components of the HSI. Inputs to the design process specified in Section 8.4.1.2 may be 

limited to those from the HSI design process analyses that involve the upgrade. The HSI design procedures, 

specified in Section 8.4.1.3, may be limited to those analyses that apply to the upgrade. 

Discussion: An HSI design review should address risk-important aspects of the upgrade (NUREG-07 11). 

9.8.1 HSI Design Methodology 

9.8.1.1 Development of HSI Design Guidance 

Guidance Tailored from NUREG-071 ]for Upgrades 

(2) The development of guidance for interface design should include upgrades. This guidance should specifically 

address consistency in design across the HSI. As far as possible, it should specify standard formats for controls and 

displays that are used for categories of commonly occurring control actions. The failure to develop such standard 

formats should be justified. 

Discussion: This guideline is derived from the guidance for tailoring the HFE design guidelines in Section 8.4.2.1 

ofNUREG-071 1, and also the High-Level Design Review Principle of Consistency (see Appendix A). 

Investigations of computer-based HSIs have shown that many types of errors may be prevented or eliminated by 

employing design guidance that encourages the consistent use of sound HFE principles across the HSI (i.e., 

Stubler, O'Hara, and Kramer, 2000). 

9.8.1.2 HSI Detailed Design and Integration 

New Guidance for Upgrades Only 

(3) HSI upgrades should be designed, to the extent possible, to be consistent with users' existing strategies for 

gathering and processing information and executing actions, as identified in the task analysis. Consistency with 

existing strategies can reduce the learning required for personnel to become proficient in using the upgrade. 

Introducing designs that require personnel to develop new strategies should be avoided, unless the new upgrade has 

significant benefits (i.e., the users should not be required to abandon existing skills and acquire new ones unless 

the new methods provide distinct benefits, such as allowing the user to reach a higher level of performance). 

Discussion: The design of conventional (e.g., analog, hardwired) HSIs supports high levels of personnel 

performance in ways that are not always obvious; experienced operators often have special strategies for extracting 

information from them (Schlager et al., 1989). The failure to consider the strategies and HSI characteristics used 

in the existing environment may result in simplistic or inadequate design requirements for an upgrade. (For 

additional information, see Design for Consistency Between New HSI Components and the Rest of the HSI, in 

Section 5.3.1, Basic Properties of Human Information Processing and Skilled Performance in Section 4.3, and 

Generic Cognitive Tasks in Section 4.4.) 

NUREG/CR-6637 9-8

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

(12) Design requirements for computer-based HSI upgrades should include requirements for crew coordination and 

define design characteristics for supporting it. The requirements should be derived from function and task analyses and should include communication and flexible allocation of tasks between crew members. If the proposed design 

may limit crew coordination, the design requirements should include features for overcoming these potential problems. Design characteristics that may limit crew coordination include features that limit the ability of 

personnel to have a shared view of plant information (e.g., decision aids and display devices that can only be 

accessed by one individual), maintain an awareness of others' actions, and communicate effectively with others 

from anticipated work locations. The design requirements should ensure that the HSI is compatible with the 

organizational structure of the crew, and produces manageable workload while ensuring plant safety. Discussion: Introducing computer-based HSI technologies may entail complex interactions with crew coordination 

and have negative effects (Hutchins, 1990; Stubler and O'Hara, 1996b; O'Hara, Higgins, Stubler, and Kramer, 

2000). This guideline is an application of the High-Level Design Review Principle of Task Compatibility (see Appendix A). (For additional information, see Changes in Personnel Tasks at the Crew Level in Section 5.2.2 and 

Crew Performance and Team Skills in Section 4.4.2. Additional organizational factors are discussed in Section 

5.5.) 

(5) If the degree of integration between plant systems is changed, then design requirements should be developed to 

ensure that the HSI supports personnel in controlling these systems. For example, higher-level automation may bring together under a single controller systems that were formerly controlled separately. Also, system upgrades 

may change the degree to which different systems share common resources (e.g., power sources, cooling water, and 

data-transmission buses). In addition, the design requirements should ensure that the HSI does not suggest a 

degree of integration between plant systems that does not actually exist (e.g., the ability to access a set of plant variables from the same control or display device may suggest to the operator that they are functionally related 

when they are not). The design requirements of the HSI should ensure that the relationships between plant systems 

are clearly and accurately depicted. 

Discussion: Industrial experience has shown that the design of computer-based HSIs can lead to 

misunderstandings about the integration between plant systems. This guideline is an application of the High-Level Design Review Principle of Task Compatibility (see Appendix A). 

New Guidance for the NUREG-0711 Element 

(6) The detailed design of the HSI should be consistent with the interface design guidance, such as in its use of 

standard formats. Any discrepancies should be justified based on HFE design process tests and evaluations. 

Discussion: A purpose of the HSI design guidance described in Section 8.4.2.1 ofNUREG-0711 is to ensure 

adherence to sound HFE principles, including consistency of design characteristics across the HSI. Therefore, 

deviations from this guidance should be justified. 

9.8.2 HSI Design Process Tests and Evaluations 

New Guidance for Upgrades Only 

(7) When an HSI component is being replaced with an upgrade, evaluations should be performed to assess the 

consistency between the methods of operating the new and the old HSI component. New methods should be 

considered unacceptable if prior experience with former methods suggest they will increase the likelihood of errors 

or increase mental workload required to avoid them. New HSI components that are unacceptable should be

9-9 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

rectified in the HSI design and implementation process through modifications to improve HSI compatibility, 

personnel training, and procedure development. 

Discussion: This guideline addresses Section 8.4.3.2 of NUREG-0711 and is consistent with the High-Level 

Design Review Principle of Response Workload (see Appendix A). It is also consistent with the principle of 

transfer of training (Holding, 1987; Swezey and Llaneras, 1997; Sawyer, Pain, Van Cott, and Banks, 1982). 

(8) Assessments should be made of the level of workload and the degree of consistency between the ways of operating 

the new HSI components, and between the new HSI components and the other components of the HSI. These 

evaluations should include the alternating use of different interfaces, particularly where the same user is expected 

to use different devices with overlapping methods of operation. New methods should be considered unacceptable if 

alternating use may increase the likelihood of errors or increase the mental workload required to avoid them. In 

addition, the level of workload associated with using the upgraded HSI should be investigated through 

performance-based measures and interviews, which should identify strategies users employ in managing workload. 

For example, complex strategies may indicate that the HSI design is not highly compatible with human 

capabilities. Unacceptable new components should be addressed through the HSI design and implementation 

process to improve their compatibility, or the likelihood of human problems reduced through other means, such as 

training and procedure development. 

Discussion: This guideline addresses Section 8.4.3.2 of NUREG-071 1. Empirical studies showed that human 

performance may show a greater decrement when different devices have overlapping methods of operation than 

when they are very different (Tanaka et al., 1991). This guideline is consistent with the High-Level Design Review 

Principle of Response Workload (see Appendix A). 

(9) Assessments should be made of the effects of HSI upgrades upon the crew's interactions and coordination, as 

defined in function and task analyses. The evaluations should ensure that the HSI design supports communication 

and flexible allocation of tasks between crew members, is compatible with the crew's organizational structure, and 

supports manageable workloads while ensuring plant safety. These tests should especially consider 

Decision aids that provide special information to an individual but not to all crew members (e.g., the user 

may experience difficulty communicating complicated concepts to the rest of the crew because they do not 

share a common view of plant conditions) 

Characteristics of display systems that interfere with the ability of personnel to share a view of the plant or 

to maintain awareness of other operators' actions 

Physical characteristics of HSI upgrades that may impede face-to-face communications, such as 

components that interfere with the line-of-sight between operators or equipment that requires them to 

spend more time away from the main control panel. 

* Other physical characteristics of the HSI that may impair crew coordination, such as background noise 

that can affect the intelligibility of speech. 

Discussion: Industrial experience showed that introducing computer-based HSI technologies may affect crew 

coordination in complex ways (Hutchins, 1990; Stubler and O'Hara, 1996b; O'Hara, Higgins, Stubler, and 

Kramer, 2000). This guideline is an application of the High-Level Design Review Principle of Task Compatibility 

(see Appendix A). (For additional information see Crew Performance and Team Skills in Section 4.4.2.) 

NUREG/CR-6637 9- 10

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.9 Procedure Development 

Guidance for the Applicability of the NUREG-0711 Element 

(1) For all upgrades identified in Section 9.1, a review of procedure development should be conducted, in accordance 

with Section 9 of NUREG-07 11. Procedure development should address all personnel tasks, described in Section 9 

of NUREG-07 11, that are affected by the upgrade or its interactions with the rest of the plant. 

Discussion: Because procedures are an essential component of HSI design, they should be kept current with the 

design of the plant and the HSI. This guideline extends the criteria of Section 9 ofNUREG-071 Ito upgrades. 

(For additional information, see Section 5.3.2.) 

Guidance Tailoredfrom NUREG-O711for Upgrades 

(2) Procedures should be developed or modified to reflect the characteristics and functions of the upgrade. 

Discussion: This guideline is an extension of the criteria of Section 9 of NUREG-07 11 to upgrades. (For 

additional information see Section 5.3.2.) 

(3) Procedural modifications should be integrated across the full set of procedures; alterations in particular parts of the 

procedures should not conflict nor be inconsistent with other parts. 

Discussion: Conflicts and inconsistencies may confuse operators and lower the usability of the procedures. This 

guideline extends the criteria of Section 9 of NUREG-071 1 to upgrades. (For additional information see Section 

5.3.2.) 

(4) All procedures should be verified as denoted in Criterion 6 of Section 9.4 ofNUREG-0711 to ensure their adequate content, format, and integration. The procedures also should be assessed through validation if an upgrade 

substantially changes personnel tasks that are significant to plant safety. The validation should ensure that the 

procedures correctly reflect the characteristics of the upgraded plant and can be carried out effectively to restore the 

plant. 

Discussion: Procedures should be evaluated similarly to other portions of the HSI because they are an essential 

component of its design. This guideline extends to upgrades the criteria of Section 9 ofNUREG-071 1. (For 

additional information see Section 5.3.2.) 

New Guidance for Upgrades Only 

(5) Procedures should be developed for temporary configurations of HSI components and plant systems that are used 

by operations or maintenance personnel when the plant is not shut down. 

Discussion: Because procedures are an essential component of HSI design, they should be current with the design of the plant and the HSI. This guideline is an extension of the criteria of Section 9 of NUREG-071 1 to the case of 

upgrades. (For additional information see Section 5.3.2.)

9-11 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.10 Training Program Development 

Guidance for the Applicability of the NUREG-0711 Element 

(1) This review should be conducted, in accordance with Section 10 of NUREG-07 11, for all upgrades identified in 

Section 9.1. Development of the training program should address all personnel tasks, described in Section 10, 

NUREG-071 1, that are affected by the upgrade or its interactions with the rest of the plant. 

Discussion: Training is an important factor for ensuring the plant's safe, reliable operation (NUREG-071 1). 

New Guidance for Upgrades Only 

(2) If upgrades are to be implemented over time so that temporary configurations of plant equipment or HSI 

components will be created, then training should be provided for each configuration that personnel will use. 

Discussion: Industrial experience has shown that temporary HSI and plant configurations due to upgrading the 

plant over time can pose demands on human performance that differ from either the initial or final configurations. 

(See Section 5.4 for additional information.) 

(3) Team training should be addressed when the upgrade significantly affects personnel's interaction (e.g., increasing 

the need for personnel to interact or changing the means available to do so), and if these interactions are important 

to the plant's safety. This training should cover teamwork skills if the upgrade requires them. Groups of people 

who work together should be specifically trained in team-related tasks, including how individual roles are related, 

how the team's performance depends upon individual performances, and what makes the team's task different 

from the sum of the individual ones. 

Discussion: The development of team skills is recognized as being critical to safe, effective performance in 

complex human-machine systems (Mumaw et al., 1994; Swezey and Llaneras, 1997). (For additional information 

see Sections 5.3.3.5, 4.3, and 4.4.) 

(4) Operators' training for upgrades should address the following points: 

The plant goals and subgoals affected by the upgrade 

Rules for decision making relating to the upgrade, such as its operation and recognition of faults 

Effective strategies for accessing and processing information presented in the upgraded HSI and for taking 

actions. 

This information should be presented in a way consistent with skills that personnel already possess. They should 

be instructed in how their existing skills may be applied to the upgraded HSI. 

Discussion: Goal assessment, effective use of decision-making rules, and strategies for gathering and processing 

information are important elements of skilled performance (Mumaw et al., 1994). Instructing personnel in using 

these skills can take them to higher levels of performance faster than if they must develop and learn these skills by 

themselves (Mumaw et al., 1994). Industry experience shows that learning and job performance are enhanced 

when personnel can transfer their skills from the old to the new HSI design (Schlager et al., 1989). (For additional 

information see Sections 5.3.3.5, 4.3, and 4.4.) 

NUREG/CR-6637 9- 12

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

(5) If older HSI components are to be maintained as backups to newer HSI components that require different skills, 

then the training program should ensure that personnel can maintain adequate levels of skill for both. This 

training should also provide skills for transitioning between the old and new components. 

Discussion: Skills for using the older HSI component may deteriorate if adequate training is not given (Roth and 

O'Hara, 1998). (For additional information see Section 5.3.3.5, Team Training.) 

New Guidance for the NUREG-0711 Element 

(6) Leaming objectives for personnel training should address the knowledge and skill requirements associated with all 

relevant dimensions of the trainee's job, such as interactions with the plant, the HSI, and other personnel. Table 

9.1, below, shows these dimensions. 

Discussion: Experiences in the nuclear power industry and other process-control industries have indicated that 

personnel have not always obtained appropriate training for new technologies because the analyses of training 

needs were incomplete or inadequate. A systematic approach to identifying training needs is specified in Section 

10.4 ofNUREG-071 1, 10 CFR 55.4 , and NUREG-0800. (These dimensions are described further in Section 

5.3.3.1.) 

Table 9.1 Some Knowledge and Skill Dimensions for Training-Needs Assessment 

Topic Knowledge Skill 

Plant Understanding of plant processes, Skills associated with monitoring 

Interactions systems, operational constraints, and detection, situation awareness, 

and failure modes response planning and 

implementation 

HSI Understanding of HSI structure, Skills associated with interface

Interactions functions, failure modes, and management tasks 

interface management tasks 

(actions, errors, and recovery 

strategies) 

Personnel Understanding information Skills associated with crew's 

Interactions requirements of others, how interactions (i.e., teamwork) 

(In the CR and in the plant) actions must be coordinated with 

others, policies and constraints on 

crews' interaction 

(7) Factual knowledge should be taught within the context of actual tasks so that personnel learn to apply it in the 

work environment. The context of the job should be defined, and it should be represented meaningfully to help 

trainees to link the knowledge to the job's requirements. Training that addresses theory should be integrated with 

training in the use of procedures. 

Discussion: Inert knowledge (Mumaw et al. 1994; Mann and Hammer, 19$6) is knowledge that can be applied in 

a training setting but cannot be used effectively in the actual task setting. It fails to become part of the usable store 

of the trainee's knowledge because it is not tied into the context of task performance. (For additional information, 

see Section 5.3.3.2, Approaches to Training Cognitive Skills.)

9-13 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

(8) Training should address the relationships between the plant's goals and subgoals affected by the upgrade. It should 

support personnel in determining the goals most relevant to the current state of the plant, assessing possible effects 

that alternative paths may have on other goals, selecting paths for achieving the most relevant goals, and ensuring 

that actions specified by procedures are consistent with those goals and paths. 

Discussion: Goal assessment is an important element of skilled problem solving and decision making (Mumaw et 

al., 1994). (For additional information, see Section 5.3.3.2, Approaches to Training Cognitive Skills, and Section 

5.3.3.3, Training for Conceptual Knowledge.) 

(9) Training should address rules for decision making related to plant systems and the HSI. It should include rules for 

accessing and interpreting information provided by HSI components, and rules for interpreting symptoms of 

failures of systems or the HSI. This training should cover acquiring new decision-making rules and eliminating 

existing ones that are not appropriate to the plant and HSI design. 

Discussion: Decision-making rules are an important aspect of skilled performance. Instructing personnel using 

correct and efficient rules can bring them to high levels of performance more quickly than if they themselves must 

develop and learn the rules (Mumaw et al., 1994). (For additional information, see Section 5.3.3.2, Approaches to 

Training Cognitive Skills, and Section 5.3.3.3, Training for Conceptual Knowledge.) 

(10) Operators should be trained in the use of effective strategies for accessing and processing information from the 

upgraded HSI. This training should support personnel in acquiring new strategies, modifying existing ones, and 

eliminating inappropriate strategies. 

Discussion: Strategies for gathering and processing information are important for achieving high levels of skilled 

performance. Instructing personnel in using them correctly and efficiently can enable personnel to achieve high 

levels of performance more quickly than if they must develop and learn them by themselves (Mumaw et al., 1994). 

(For additional information see Section 5.3.3.2, Approaches to Training Cognitive Skills.) 

(11) Training programs for developing skills should be structured so that the training environment is consistent with 

the level of skill being taught. It should support skill acquisition by allowing trainees to manage cognitive 

demands. For example, trainees should not be placed in environments teaching high-level skills, such as 

coordinating control actions among crew members, before they have mastered requisite, low-level skills, such as 

how to manipulate control devices. 

Discussion: The effective management of cognitive resources can support the acquisition of skills, making 

training more effective and enhancing performance in the work environment (Mumaw et al., 1994). (For further 

information, see Section 5.3.3.4, Training for Skill Automaticity and Section 5.3.3.6, Evaluation of Training 

Systems.) 

9.11 Human Factors Verification and Validation 

9.11.1 General Criteria 

Guidance for the Applicability of the NUREG-0711 Element 

(1) Verification and validation should be conducted in accordance with the general criteria in Section 11.4.1 of 

NUREG-07 11. The general scope of the V&V activities should include all items, defined in Criterion 1 of Section 

11.4.1 ofNUREG-071 1, that are applicable to the upgrade. Plans for the V&V and reports of the results should be 

documented as described in Section 11.3 of NUREG-07 11. The applicability of individual verification and 

validation evaluations is described in the three subsections, below. 

NUREG/CR-6637 9- 14

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

Discussion: This guideline was derived from Section 11.4. lof NUREG-071 1. 

9.11.2 HSI Task Support Verification 

Guidance for the Applicability of the NUREG-0711 Element 

(2) A review of HSI task support should be conducted, in accordance with Section 11.4.2 NUREG-07 11, for all 

upgrades identified in Section 9.1. The HSI verification should address all HSI aspects described in Section 11.4.2 

of NUREG-07 11 that are relevant to the upgrade. For upgrades that do not include modifications to the HSI, task 

support verification should identify any new demands for monitoring and control, and determine whether they are 

adequately addressed by the existing HSI design. 

Discussion: System upgrades sometimes impose new monitoring and control demands on personnel. An HSI task 

support verification can identify and evaluate these demands. (For additional information, see Sections 5.4.1 and 

5.4.4.) 

New Guidance for Upgrades Only 

(3) Task support verification should address upgrade configurations in which old HSI components are permanently 

deactivated, but not removed (e.g., abandoned in place). Criterion 2 of Section 11.4.2 states that the HSI should 

not contain any information, displays, or controls that do not support the operator's tasks. This verification should 

identify deactivated HSI components that may have potentially negative effects on personnel's performance, such 

as obstructing the view of important information or adding visual clutter which may interfere with monitoring. 

Deactivated HSI components requiring further evaluation through HIFE design verification or integrated system 

validation should be identified. 

Discussion: For additional information, see Sections 5.4.1 and 5.4.4. 

(4) Task support verification should address temporary configurations of the HSI and plant systems that may be 

created during implementation of the upgrade, and used by operations and maintenance personnel when the plant 

is not shut down. These configurations may include: 

The use of HSI components that differ from the intended final design 

Combinations of HSI components and system configurations that differ from both the original and the 

intended final designs. 

For each temporary HSI configuration, task requirements should be identified and compared to the information and 

control capabilities provided. For example, if a temporary configuration of plant systems introduces special 

monitoring requirements, then the HSI should provide the necessary information. 

Discussion: The operation and maintenance of temporary configurations of plant systems or HSI components can 

impose unique demands on human capabilities. (For additional information, see Sections 5.4.1 and 5.4.4.)

9-15 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.11.3 HFE Design Verification 

Guidance for the Applicability of the NUREG-0711 Element 

(5) HFE design verification should be reviewed, in accordance with Section 11.4.3 ofNUREG-071 1, for all upgrades 

of the HSI identified in Section 9.1. Its scope may be restricted to the upgraded HSI components and their 

interactions with the rest of the HSI. 

Discussion: HFE design verification is only performed on the HSI, not on upgrades of plant systems. (NUREG

0711). 

New Guidance for Upgrades Only 

(6) When both old and new versions of similar HSI components are permanently present in the HSI, this verification 

should ensure that their means of presentation and methods of operation are compatible, such that personnel 

performance will not be impaired when using them alternately. 

Discussion: Incompatibilities may result in decrements in human performance when their use is alternated. (For 

additional information, see Sections 5.4.2 and 5.4.4.) 

(7) Temporary configurations of the HSI and plant systems, which may be used by operations and maintenance 

personnel when the plant is not shut down, should be reviewed to ensure that their design is consistent with the 

principles of good HFE design, including consistency with the rest of the HSI. 

Discussion: Industry experience indicates that when the HSI or plant systems are upgraded, plant personnel may 

encounter temporary configurations which they must operate and maintain. These may impose demands that differ 

from the initial and final configurations. (For additional information see Sections 5.4.2 and 5.4.4.) 

9.11.4 Integrated System Validation 

Guidance for the Applicability of the NUREG-0711 Element 

(8) The integrated system validation should be reviewed, in accordance with Section 11.4.4 NUREG-071 1, for all 

upgrades identified in Section 9.1 that may (1) change personnel's tasks; (2) change task demands, such as 

changing a task's dynamics, complexity, or workload; or (3) interact with or affect other HSI components in ways 

that may degrade performance. Integrated system validation may not be needed when an upgrade results in minor 

changes to personnel tasks such that they may reasonably be expected to have little or no overall effect on workload 

and the likelihood of error. The scope of the integrated system validation should include the upgrade and its 

interactions with the rest of the plant. 

Discussion: Integrated system validation is the process by which an integrated system design (i.e., hardware, 

software, and personnel elements) is evaluated to determine whether it acceptably supports safe operation of the 

plant. It is intended to evaluate the acceptability of those aspects of the design that cannot be determined though 

such analytical means as HSI task support verification and HFE design verification. For upgrades that change 

plant systems but do not modify the HSI, validation can provide evidence about the adequacy of the existing HSI 

for supporting personnel performance. (For additional information see Sections 5.4.3 and 5.4.4.) 

NUREGICR-6637 9- 16

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.11.4.1 Test Objectives 

Guidance Tailoredfrom NUREG-0711 for Upgrades 

(9) Section 11.4.4.2 of NUREG-071 1 states that the detailed test objectives should identify aspects of the integrated 

system, including staffing, communications, and training, that may negatively affect integrated system 

performance. This scope may be limited to HSI components included in the upgrade, other HSI components used 

with the upgrade, and the potential interactions between the two. 

Discussion: Integrated system validation should address aspects of integrated system performance that are related 

to the upgrade, including interactions between HSI components. For example, a new CRT-based display system 

may be negatively affected by the existing lighting system of the CR. A new HSI component that emits high levels 

of background noise may degrade the effectiveness of the existing audible alarm system in the CR. (For additional 

information see Sections 5.4.3 and 5.4.4.) 

(10) The test objectives and scenarios should be developed to exercise dimensions of personnel performance that are 

addressed by the design of the upgrade, including functions and tasks affected by the upgrade. 

Discussion: Integrated system validation should focus on aspects of integrated system performance related to the 

upgrade. This guideline extends the criteria of Section 11.4.4 of NUREG-07 11. (For additional information, see 

Sections 5.4.3 and 5.4.4.) 

9.11.4.2 Validation Testbeds 

Guidance Tailored from NUREG-071 ] for Upgrades 

(11) The validation testbeds should include procedures that have been modified for the upgrades. 

Discussion: Procedures are an important component of the HSI (NUREG-071 1). 

9.11.4.2.1 Main Control Room 

Guidance Tailored from NUREG-0711for Upgrades 

(12) If a training simulator is used as the validation testbed for the main control room, then 

The user interfaces of the training simulator should be modified to accurately represent characteristics of 

the HSI upgrade. 

• The simulation of the plant's response should be modified to reflect the behavior of the upgrade. 

Discussion: This guideline is an extension to upgrades of the guidance of Section 11.4.4.3.1 of NUREG-071 1. 

(13) If a training simulator is not used as the validation testbed for the main control room, then other suitable testbeds 

should be developed, such as part-task simulators and mockups. These testbeds should 

Represent the HSI components of the upgrade 

Represent interactions with HSI components and plant systems that are related to the upgrade

9-17 NUREG/CR-6637
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

Represent aspects of the task environment that are relevant to the human performance considerations 

identified early in the HFE design review 

Be consistent with the dimensions of testbed fidelity described in Section 11.4.4.3.1 of NUREG-0711. 

Discussion: This guideline extends the guidance of Section 11.4.4.3.1 ofNUREG-071 1 to upgrades. 

9.11.4.2.2 Representation of Facilities Remote from the Main Control Room 

New Guidance for the NUREG-0711 Element 

(14) When simulations or mockups are used, the testbed should represent all aspects of the environment that are 

relevant to the human performance factors identified in the earlier stages of the HFE design review, including the 

following: 

Static and dynamic characteristics of the HSI, including manual and automatic operating features (e.g., 

automatic mode changes) and faults 

Static and dynamic characteristics of the plant's systems, including their behavior and fault indications 

Communications with personnel located in the control room or elsewhere in the plant 

Task-environment factors that may affect personnel's performance (lighting, noise, heating and 

ventilation, and protective clothing and equipment). 

Discussion: This guideline is a revision Criterion 2 of Section 11.4.4.3.2 ofNUREG-071 1. Operational and 

maintenance tasks performed at local control stations and local maintenance interfaces can affect the plant's safety 

(Stubler, Higgins, and Kramer, 2000). 

9.11.4.3 Plant Personnel 

New Guidance for the NUREG-0711 Element 

(15) The selection of participants for particular validation trials should be consistent with the test's objectives. For 

trials requiring CR operating shifts, consideration should be given to the assembly of operating crews (e.g., shift 

supervisors, reactor operators, and shift technical advisors) that will participate in the tests. For trials that address 

maintenance tasks, maintenance personnel should be selected with expertise in the appropriate area (e.g., 

mechanical, electrical, or instrumentation and control. 

Discussion: This guideline is a revision of Criterion 3 of Section 11.4.4.4 of NUREG-07 11. It was expanded to 

include some considerations related to upgrades and new plant designs. Integrated system validation should 

address risk-important maintenance tasks, such as those undertaken while the plant is on-line. (For additional 

information, see Sections 5.4.3 and 5.4.4.) 

NUREG/CR-6637 9-18

I I
9 DESIGN PROCESS REVIEW GUIDANCE FOR HYBRID HSI

9.11.4.4 Operational Conditions 

Guidance Tailoredfrom NUREG-O711for Upgrades 

(16) The personnel tasks used during the validation trials should reflect tasks and HSI aspects that involve the upgrade, 

rather than the entire range of topics discussed in Criterion 3 of Section 11.4.4.5. 

Discussion: The guideline extends the criteria of Section 11.4.4.5 ofNUREG-071 I to validating upgrades. 

(17) The operational conditions used during the validation trials should address the ability of personnel to comprehend 

the limits of integration between components of the upgrade, and between the upgrade and the rest of the plant. For example, operators should be able to determine whether similar plant variables, presented via different display 

devices or display pages, are identical, or whether their data originate from different sources. 

Discussion: Industrial experience suggests that the displays of computer-based systems can obscure boundaries 

between functionally separate systems or suggest levels of integration that do not exist. (See Sections 5.3.1, 5.4.3 

and 5.4.4 for additional information.) 

New Guidance for Upgrades Only 

(18) The operational conditions used during the validation trials should address the effects of transfer of learning effects 

on personnel's performance when an upgrade replaces an older HSI component. These trials should determine 

whether the training, procedures, and HSI design are sufficient to prevent negative transfer of learning which may 

affect plant safety. 

Discussion: Negative transfer of learning effects may occur when the new and old components are different and 

impose different demands on personnel using them. (See Sections 5.3.1, 5.4.3 and 5.4.4 for additional 

information.) 

(19) When both old and new versions of the same HSI components with different means of presentation and methods of 

operation are permanently present in the HSI, integrated validation should ensure that personnel can alternate their 

use of these HSI components without degrading their performance. 

Discussion: Incompatibilities between the methods of operating HSI components may cause decrements in human 

performance when their use is alternated. (See Sections 5.4.3 and 5.4.4 for additional information.) 

(20) If HSI task support verification identifies potential human factors discrepancies associated with old HSI 

components that are to be deactivated and left in place in the HSI, then validation tests should be developed for 

these problems. 

Discussion: The presence of deactivated HSI components may have negative effects on personnel, such as causing 

visual clutter that interferes with people's ability to locate and use other HSI components. (See Sections 5.4.3 and 

5.4.4 for additional information.)

9- 19 NUREG/CR-6637
Appendix A

High-Level Design Review Principles 

From NUREG-0700, Rev. 1

NUREG/CR-6637
NUREG-0700 HIGH-LEVEL DESIGN REVIEW PRINCIPLES

The design of human-system interfaces (HSIs) should support the operating personnel's primary task of monitoring and 

controlling the plant, without imposing an excessive workload associated with using them (manipulating windows, 

selecting display selection, and navigating, for example). The HSI also should support the recognition, tolerance, and 

recovery from human errors. Guidelines for reviewing human factors engineering design help to ensure that these goals are 

achieved. As part of the guidance development for NUREG-0700, Rev. 1, a set of "high-level" design review principles 

were established representing the generic HSI characteristics necessary to support personnel's performance. They were 

used to draft many detailed review guidelines in Part 2 of NUREG-0700 (see O'Hara, Brown, and Nasta, 1996 for a 

discussion of their use). The high-level principles also were used in the formulating guidelines for computer-based 

procedures. 

The 18 principles are divided into four categories: general principles, primary task design, secondary task control, and task 

support. The categories and the principles that comprise them are described below. 

General Principles 

These principles ensure that the HSI design supports personnel safety, and is compatible with people's general cognitive 

and physiological capabilities. 

Personnel Safety - The design should minimize the potential for injury and exposure to harmful materials. 

Cognitive Compatibility - The operators' role should consist of purposeful, meaningful tasks that enable them to 

remain familiar with the plant, and maintain a level of workload that is not so high as to negatively affect 

performance, but sufficient to maintain vigilance. 

Physiological Compatibility - The design of the interface should reflect physiological characteristics, including 

visual and auditory perception, biomechanics (reach and motion), motor control, and anthropometry. 

Simplicity of Design - The HSI should represent the simplest design consistent with functional and task 

requirements. 

Consistency - There should be a high degree of consistency between the HSI, the procedures, and the training 

systems. At the HSI, the way the system functions and appears to the operating crew always should reflect a high 

degree of standardization, and consistency with procedures and training. 

Primary Task Design 

These principles support the operator's primary task of process monitoring, decision making, and control to maintain safe 

operation. 

Situation Awareness - The information presented to the users by the HSI should be correct, rapidly recognized, and 

easily understood (e.g., "direct perception" or "status-at-a-glance" displays) and support the higher-level goal of 

user's awareness of the system's status. 

Task Compatibility - The system should meet the requirements of users to perform their tasks (including operation, 

safe shutdown, inspection, maintenance, and repair). The forms and formats of data should be appropriate to the

A-1I NUREG/CR-6637
APPENDIX A

task (including the need to access confirmatory data or raw data in the case of higher-level displays), and control 

options should encompass the range of potential actions. No unnecessary information or control options should be 

present. 

User Model Compatibility - All aspects of the system should be consistent with the users' mental models 

(understanding and expectations about how the system behaves, learned through training, using procedures, and 

experience). All aspects of the system also should be consistent with established conventions (i.e., expressed in 

customary, commonplace, useful, and functional terms, rather than abstract, unusual or arbitrary forms, or in forms 

requiring interpretation). 

Organization of HSI Elements - The organization of all aspects of the HSI (from the elements in individual 

displays, to individual workstations, to the entire control room) should be based on the user's requirements and 

should reflect the general principles of organization by importance, frequency, and order of use. Critical safety 

function information should be available to the entire operating crew in dedicated locations to ensure its 

recognition, and to minimize data search and response. 

Logical/Explicit Structure - All aspects of the system (formats, terminology, sequencing, grouping, and the 

operator's decision-support aids) should reflect an obvious logic based on task requirements or some other non

arbitrary rationale. There should be a clear relationship of each display, control, and data-processing aid to the 

overall task or function. The structure of the interface and its associated navigation aids should make it easy for 

users to recognize where they are in the data space, and enable them to rapidly access data not currently visible 

(e.g., on other display pages). The way the system works and is structured should be clear to the user. 

Timeliness - The system's design should take into account users' cognitive processing capabilities as well as 

process-related time constraints to ensure that tasks can be performed within the time required. Information-flow 

rates and control-performance requirements that are too fast or too slow could diminish performance. 

Controls/Displays Compatibility - Displays should be compatible with the requirements for data entry and control. 

Feedback - The system should provide useful information on its status, permissible operations, errors and error 

recovery, dangerous operations, and validity of data. 

Secondary Task Control 

These principles minimize secondary tasks, i.e., tasks personnel must perform when working with the system that are not 

directed to the primary task. Secondary tasks include managing the interface, such as navigating through displays, 

manipulating windows, and accessing data. Performing secondary tasks detracts from the crew's primary tasks, so their 

demands must be controlled. 

Cognitive Workload - The information presented by the system should be rapidly recognized and understood; 

therefore, its design should minimize the need for mental calculations or transformations, and use of recall 

memory (recalling lengthy lists of codes, complex command strings, information from one display to another, or

NUREG/CR-6637 A-2
APPENDIX A

lengthy action sequences). Raw data should be processed into a directly usable form (although raw data still 

should be accessible for confirmation). 

Response Workload - The system should require a minimum number of steps to accomplish an action, e.g., single

versus command-keying, menu selection versus multiple-command entry, single input mode (keyboard, mouse) 

versus mixed mode. In addition, it should not require the entry of redundant data, nor the re-entry of information 

already present, or information the system can generate from data already resident. 

Task Support 

These principles address the characteristics of the HSI that support its use by personnel, such as providing (1) HSI 

flexibility so tasks can be accomplished in more than one way, (2) guidance for users, and (3) mitigation of errors. 

Flexibility - The system should give the user multiple means to carry out actions (and verify automatic actions), 

and permit displays and controls to be configured in the most convenient way. However, flexibility should be 

limited to situations where it is advantageous in task performance (e.g., in accommodating different levels of 

experience of the users); flexibility should not be provided for its own sake because there is a tradeoff between 

consistency and the imposition of interface management workload (which detracts from monitoring and 

operations). 

User Guidance and Support - The system should provide an effective "help" function; i.e., informative, easy-to

use, and relevant guidance should be provided on-line and off-line to help the user understand and operate the 

system. 

Error Tolerance and Control - A fail-safe design should be provided wherever failure can damage equipment, 

injure personnel, or inadvertently operate critical equipment. Therefore, the system should generally be designed 

so that a user's error will not have serious consequences. The negative effects of errors should be controlled and 

minimized. The system should offer simple, comprehensible notification of the error, and simple, effective 

methods for recovery.

A-3 NUREG/CR-6637
GLOSSARY 

Abilities: Usually cognitive capabilities that require applying some knowledge base, and are necessary to perform a job 

function. 

Automaticity: The ability to perform specific sets of actions without committing extensive cognitive resources. 

Capture error: An error of execution (slip) that occurs when an infrequent action requires a sequence of operations, some 

of which are the same as, or similar to, those of a frequent action. In attempting the infrequent action, the more frequent 

one is performed instead. For example, an operator intends to perform task 1, which is composed of operations A, B, C, 

and D, but instead executes the more-frequently performed task 2, composed of operations A, B, C, and E. 

Data-driven monitoring: Monitoring that depends on the relative and absolute levels of salience. The strength of the 

stimuli must exceed some threshold to draw attention, above which personnel tend to attend to the stimuli with the greatest 

salience. (Contrasts with model-driven monitoring.) 

Description error: An error of execution (slip) that involves performing the wrong set of well-practiced actions. 

Description errors occur when the information that activates or triggers the action is either ambiguous or undetected. 

Detection: Recognition that something is not operating correctly, and that an abnormality exists. 

Emergent feature: A "high-level," global perceptual feature produced by the interactions among individual parts or 

graphical elements of a display (e.g., lines, contours, and shapes). 

Knowledge: An organized body of information, usually factual or procedural, the application of which allows adequate job 

performance. It is the foundation upon which abilities and skills are built. 

Long-term memory: A mental repository of things that have been learned. Its capacity for storing information is large 

(i.e., considered by some to be virtually unlimited). 

Loss-of-activation error: An error (slip) that occurs when an intended action is not carried out due to a failure of memory 

(i.e., the intention has partially or completely decayed from memory). A special case of loss-of-activation errors involves 

forgetting part of an intended act while remembering the rest (e.g., retrieving a display while not being able to remember 

why it is needed). 

Mental model: The operator's internal representation of physical and functional characteristics of the plant and its 

operation. 

Misordered components of an action sequence: An error (slip) involving skipped, reversed, or repeated steps. 

Mistake: An error in intention formation, such as forming a plan inappropriate for the situation. Mistakes are related to 

incorrect assessments of the situation, or inadequate planning of a response. (Contrasts with slip.)

G-1I NUREG/CR-6637
GLOSSARY

Mode error: An operation that is appropriate for one mode when the device is in another mode. Mode errors occur when 

the user believes the device is in one mode when it is in another, and consequently, performs an action that is inappropriate 

for the actual mode. 

Model-driven monitoring: Monitoring that is directed by the individual's knowledge and expectations. It can be viewed 

as active monitoring in the sense that personnel deliberately direct attention to areas they expect to provide useful 

information, rather than merely attending to stimuli based on their relative salience. (Contrasts with data-driven 

monitoring.) 

Monitoring: Checking the state of the plant to determine whether the systems and equipment are operating correctly. This 

may include checking parameters indicated on the control room panels, monitoring parameters displayed by the process 

computer, and obtaining verbal reports from personnel. 

Primary tasks: Tasks performed by operators as part of their functional role of supervising the plant. Primary tasks 

require knowledge of the plant and cognitive skills for making decisions and taking actions. 

Response implementation: Performing actions identified during response planning. 

Response planning: Developing an approach for achieving a goal. The planned response may be as simple as deciding to 

access a particular parameter from a particular human-system interface component, or it may involve selecting a 

complicated course of action guided by a procedure. 

Secondary tasks: Tasks that operators must perform but which are not directly related to a primary task. Secondary tasks 

involving management of the human-system interface include navigating through an information system, manipulating 

windows on a display, and manipulating performance aids. 

Selective attention: The ability to shift attention to important things and ignore irrelevant ones. Selective attention is 

driven by both bottom-up and top-down processes. The former are based on characteristics of the stimuli that affect 

salience, such as their size, loudness, and brightness, and results in attention to those with highly salient features (see data

driven monitoring). Top-down processes are developed from past experiences and are based on a person's understanding of 

the environment (i.e., mental model) and expectations. Top-down processing results in attention to stimuli that have 

special significance (i.e., see model-driven monitoring). 

Situation awareness: The cognitive activities involved in processing stimuli to construct an understanding of the situation. 

Two important ones are interpreting the current state of the plant, and determining its implications. 

Situation model: An understanding of the specific current situation (i.e., the current state of the plant), based on factors 

known or hypothesized to be affecting the situation at a given time. 

Slip: An error in executing an intention (i.e., the user intends to do one thing but does another). (Contrasts with mistake.) 

NUREG/CR-6637 G-2

I I
GLOSSARY

Skills: The capability to perform a job with ease and precision. The term is often applied to both psychomotor and 

cognitive activities. For example, a psychomotor skill may entail precisely operating a particular control device. A 

cognitive skill may entail using strategies for rapidly finding and accessing information. 

Sustained attention: The ability to attend to a single information source for an extended period. It is also called focused 

attention. 

Team: A set of two or more people who interact dynamically, interdependently, and adaptively toward a common valued 

goal, objective, or mission; each having specific roles or functions to perform. 

Transfer of training: A framework for describing the effects of prior leaming upon performance in a new (transfer) 

condition. Positive and negative transfer, respectively, refer to the facilitative and inhibitory effects of earlier learning. 

Unintentional activation: A slip that occurs when a set of actions that is not part of a current action sequence becomes 

activated for extraneous reasons, and then becomes triggered; this can lead to the unintended actuation of an input device. 

Working memory: A mental repository containing things that are currently in consciousness (i.e., things that one is 

considering at the moment). Unlike long-term memory, the number of items that can be stored at any time in working 

memory is severely limited. (Working memory is sometimes referred to as active memory.)

G-3 NUREG/CR-6637
NRC FORM 335 U.S. NUCLEAR REGULATORY COMMISSION 1. REPORT NUMBER (2-89) (Assigned by NRC, Add Vol., Supp., Rev., NRCM 1102, and Addendum Numbers, if any.) 3201. 3202 

BIBLIOGRAPHIC DATA SHEET NUREG/CR-6637 

(See instructions on the reverse) BNL-NUREG-52567 

2. TITLE AND SUBTITLE 

Human-System Interface and Plant Modernization Process: Technical Basis and 3. DATE REPORT PUBUSHED 

Human Factors Review Guidance MONTH YEAR 

March 2000 

4. FIN OR GRANT NUMBER J-6012 

5. AUTHOR(S) 6. TYPE OF REPORT 

William F. Stubler, John M. O'Hara, James C. Higgins, and Joel Kramer Technical 

7. PERIOD COVERED (inclusive Dates) 

8. PERFORMING ORGANIZATION - NAME AND ADDRESS (if NRC, provide Division, Office or Region, U.S. Nuclear Regulatory Commission, and mailing address; if contractor, provide name and mailing address.) 

Department of Advanced Technology 

Brookhaven National Laboratory 

Upton, NY 11973-5000 

9. SPONSORING ORGANIZATION - NAME AND ADDRESS (if NRC, type 'Same as above; if contractor, provide NRC Division, Office or Region, U.S. Nuclear Regulatory Commission, and mailing address.) 

Division of'Systems Analysis and Regulatory Effectiveness 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

10. SUPPLEMENTARY NOTES 

3. Kramer, NRC Project Manager

11. ABSi'AC.I (200uwordsor les Hybrid human-system interfaces (HSIs) result from the combination of new (e.g., digital) and traditional technologies. New 

demands may be imposed on personnel in operating and maintaining these systems. These demands may result from many factors 

including the characteristics of the new technologies, the characteristics of the mixture of new and traditional technologies, the 

process by which the hybrid HSI is developed and used, and the way in which personnel are prepared to use the hybrid HSI. The 

objective of this study was to develop human factors review guidance on the processes by which hybrid HSIs are developed, implemented, and integrated into plant operations. A characterization framework was developed for describing the key characteristics of hybrid HSIs that are important to human factors engineering (HFE) reviews. Then, the research studies, FIFE 

processes, and guidance related to system development and modernization were reviewed. This information was used as the 

technical basis upon which we developed the design review guidelines. This guidance applies to general work that should be 

undertaken and factors that should be considered in designing and implementing hybrid HSIs, particularly in upgrading existing HSIs. The establishment and analysis of design requirements, interface design, and the evaluation of the final system are 

addressed. Issues for further research were identified for process consideration for which the technical basis was insufficient to 

support guidance development.

12 KEY WORDS/DESCRIPTORS (List words or phrases that will assist researchers in locating the report.) 13. AVAILABILITY STATEMENT 

Control rooms, human factors engineering, human-system interface, man-machine systems, Unlimited 

reactor safety, reactor operators, test and evaluation, human-factors review criteria 14. SECURITY CLASSIFICATION 

(This Page) Unclassified 

(This Report) Unclassified 

15. NUMBER OF PAGES 

16. PRICE 

NRC FORM 335 t2-.891 N

RC FORM 335 (2-89)
Federal Recycling Program
UNITED STATES 

NUCLEAR REGULATORY COMMISSION 

WASHINGTON, D.C. 20555-0001

SPECIAL STANDARD MAIL 

POSTAGE AND FEES PAID 

USNRC 

PERMIT NO. G-67