NUREG/CR-6633 

BNL-NUREG-52563

Advanced Information 

Systems Design: Technical 

Basis and Human Factors 

Review Guidance 

Brookhaven National Laboratory 

U.S. Nuclear Regulatory Commission 

Office of Nuclear Regulatory Research 

Washington, DC 20555-0001 ON
AVAILABILITY NOTICE 

Availability of Reference Materials Cited in NRC Publications

NRC publications in the NUREG series, NRC regu

lations, and Title 10, Energy, of the Code of Federal 

Regulations, may be purchased from one of the fol

lowing sources: 

1. The Superintendent of Documents 

U.S. Government Printing Office 

PO. Box 37082 

Washington, DC 20402-9328 

<http://www.access.gpo.gov/sudocs> 

202-512-1800 

2. The National Technical Information Service 

Springfield, VA 22161-0002 

<http://www.ntis.gov> 

1-800-553-6847 or locally 703-605-6000 

The NUREG series comprises (1) brochures 

(NUREG/BR-XXXX), (2) proceedings of confer

ences (NUREG/CP-XXXX), (3) reports resulting 

from international agreements (NUREG/IA-)XXXX, 

(4) technical and administrative reports and books 

[(NUREG-XXXX) or (NUREG/CR-)000q], and (5) 

compilations of legal decisions and orders of the 

Commission and Atomic and Safety Licensing 

Boards and of Office Directors' decisions under 

Section 2.206 of NRC's regulations (NUREG

X)0(q. 

A single copy of each NRC draft report for com

ment is available free, to the extent of supply, upon 

written request as follows: 

Address: Office of the Chief Information Officer 

Reproduction and Distribution 

Services Section 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

E-mail: < DISTRIBUTION@nrc.gov> 

Facsimile: 301-415-2289 

A portion of NRC regulatory and technical informa

tion is available at NRC's World Wide Web site:

<http://www.nrc.gov> 

After January 1,2000, the public may electronically 

access NUREG-series publications and other NRC 

records in NRC's Agencywide Document Access 

and Management System (ADAMS), through the 

Public Electronic Reading Room (PERR), link 

<http://www.nrc.gov/NRC/ADAMS/index.html>. 

Publicly released documents include, to name a 

few, NUREG-series reports; Federal Register no

tices; applicant, licensee, and vendor documents 

and correspondence; NRC correspondence and 

internal memoranda; bulletins and information no

tices; inspection and investigation reports; licens

ee event reports; and Commission papers and 

their attachments. 

Documents available from public and special tech

nical libraries include all open literature items, such 

as books, journal articles, and transactions, Feder

al Register notices, Federal and State legislation, 

and congressional reports. Such documents as 

theses, dissertations, foreign reports and transla

tions, and non-NRC conference proceedings may 

be purchased from their sponsoring organization. 

Copies of industry codes and standards used in a 

substantive manner in the NRC regulatory process 

are maintained at the NRC Library, Two White Flint 

North, 11545 Rockville Pike, Rockville, MD 

20852-2738. These standards are available in the 

library for reference use by the public. Codes and 

standards are usually copyrighted and may be 

purchased from the originating organization or, if 

they are American National Standards, from

American National Standards Institute 

11 West 42nd Street 

New York, NY 10036-8002 

<http://www.ansi.org> 

212-642-4900

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United 

States Government nor any agency thereof, nor any of their em ployees, makes any warranty, expressed or implied, or assumes

any legal liability or responsibility for any third party's use, or the 

results of such use, of any infonration, apparatus, product, or process disclosed in this report, or represents that its use by 

such third party would not infringe privately owned rights.

I I
NUREG/CR-6633 

BNL-NUREG-52563 

Advanced Information 

Systems Design: Technical 

Basis and Human Factors 

Review Guidance 

Manuscript Completed: January 2000 

Date Published: March 2000 

Prepared by 

J. M. O'Hara, J. C. Higgins/BNL 

J. Kramer/NRC 

Brookhaven National Laboratory 

Upton, NY 11973 

J. Kramer, NRC Project Manager 

Prepared for 

Division of Systems Analysis and Regulatory Effectiveness 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

NRC Job Code J6012
NUREG/CR-6633 has been reproduced 

from the best available copy.

I I
ABSTRACT 

Plant information systems provide operators with information supporting situation assessment, monitoring and 

detection of disturbances, response planning, and response implementation. The importance of the design of 

information systems for human performance and reliability has long been recognized. Recent advances in design 

go beyond the 'one sensor - one indicator' display systems of conventional plants. Computer-based systems provide a variety of ways to process and present data. These characteristics of design represent a trend toward 

making displays more immediately meaningful to personnel by mapping display representations to important 

aspects of plant processes and to underlying cognitive mechanisms, such as perceptual processes and mental 

models. The objective of this study was to develop guidance for human factors review of advanced information 

systems based on a technically valid methodology for developing guidance. To support this objective, we 

developed a characterization framework for describing key design characteristics of information systems. The 

characterization includes the following major components: information requirements, representation systems, 

interface management functions, and display devices. Representation systems were further hierarchically divided 

into display elements (basic building blocks of displays, such as axes, alphanumeric elements, and icons), display 

formats (such as mimic displays, configural displays, and novel graphics), display pages (a defined set of 

information that is intended to be displayed as a single unit), and display networks (the organization of display 

pages). Then, we examined research in the following areas: (1) generic cognitive tasks that an information system 

must support, (2) information requirements analysis, (3) information representation, and (4) information 

organization. This research was used to provide the technical basis on which guidelines for review of design were 

developed. These guidelines address both the design process and the implementation of advanced information 

systems. However, there were aspects of information systems for which the technical basis was insufficient to 

support the development of guidance. These were identified as issues to be addressed in future research.

iii NUREG/CR-6633
CONTENTS 

Paiie 

A BSTRA CT ... ......... ................................................................. ix 

LIST OF FIGURES ............. ............................................................. ix 

LIST OF TABLES ..... .............................................................. xi 

EXECUTIVE SUMMARY....................................................... xiii 

PREFA CE .................................................................... xix 

ACKNOWLEDGMENTS ..................................................................... xxi 

ACRONYMS...................................................................... xi 

PART 1: Guidance Development and Technical Basis 

1 INTRODU CTION ................................................................... 1-1 

1.1 Background ................................................................. 1-1 

1.2 Scope of the Research ......................................................... 1-3 

1.3 Organization of the Report ..................................................... 1-4 

2 O BJECTIV E ....................................................................... 2-I 

3 M ETHODOLOGY .................................................................. 3-1 

3.1 O verview ................................................................... 3 1 

3.2 Characterization of the Information System ........................................ 3-2 

3.3 Development of the Technical Basis .............................................. 3-2 

3A Development of Guidelines and Documentation ..................................... 3-6 

3.5 Identification of Issues ........................................................ 3-6 

3.6 Peer Review ................................................................. 3-6 

4 CHARACTERIZATION OF INFORMATION SYSTEMS .................................. 4-1 

4.1 Information Requirements ...................................................... 4-1 

4.2 Representation System ........................................................ 4-1 

4.3 M anagement Functions ........................................................ 4-6 

4.4 D isplay D evices ............................................................. 4-6 

4.5 Relationship to Other Aspects of HSIs ............................................ 4-6 

5 DEVELOPMENT OF TECHNICAL BASIS .............................................. 5-1 

5.1 Existing NRC Guidance for Information Systems ................................... 5-1 

5.2 Cognitive Tasks Supported by Information Systems ................................. 5-7 

5.2.1 Situation Assessm ent .................................................. 5-7 

5.2.2 M onitoring and Detection .............................................. 5-11 

v NUREG/CR-6633
CONTENTS (Continued) 

Pane 

5.2.3 Response Planning ................................................... 5-15 

5.2.4 Response Implementation .............................................. 5-16 

5.2.5 Implications for Information System Design ............................... 5-17 

5.3 Analysis of Information Requirements ........................................... 5-18 

5.3.1 The EID Approach to the Analysis of Information Requirements ............... 5-18 

5.3.2 Research on the Effectiveness of EID Displays ............................. 5-23 

5.3.2.1 DURESS EID Display ......................................... 5-23 

5.3.2.2 Rankine Cycle EID Display ..................................... 5-31 

5.3.2.3 Toshiba Advanced BWR HSI .................................... 5-41 

5.3.2.4 Other EID Studies ............................................. 5-4 5 

5.3.3 Conclusions ......................................................... 5-4 8 

5.4 Research on Information Representation ......................................... 5-53 

5.4.1 Theory of Representational Aiding ....................................... 5-54 

5.4.2 Research on the Effectiveness of Representational Aiding .................... 5-59 

5.4.3 Conclusions ......................................................... 5-63 

5.5 Organization of Displays ...................................................... 5-64 

5.5.1 Organization of Display Pages .......................................... 5-65 

5.5.2 Organization of Networks .............................................. 5-67 

5.5.3 Conclusions ......................................................... 5-68 

5.6 General Summary and Issues ................................................... 5-68 

5.6.1 Information System Design and Operator Performance ....................... 5-68 

5.6.2 Issues ........................................................... 5-71 

5.6.2.1 Technical Basis Issues .......................................... 5-71 

5.6.2.2 Design Review Issues ......................................... 5-72 

5.6.2.3 Operator-Related Issues ........................................ 5-74 

6 DEVELOPMENT OF GUIDANCE ..................................................... 6-1 

6.1 Guidance for Reviewing the Information Systems Design Process ...................... 6-1 

6.2 Guidelines for HFE Design Review .............................................. 6-1 

7 GENERAL SUM MARY ............................................................ 7-1 

8 REFEREN CES ..................................................................... 8-1 

NUREG/CR-6633 vi
CONTENTS (Continued) 

Page 

PART 2: Review Guidelines for Information System Design 

9 REVIEW GUIDANCE FOR THE INFORMATION SYSTEM DESIGN PROCESS .............. 9-1 

9.1 Operating Experience Review (OER) ............................................. 9-1 

9.2 Function and Task Analysis .................................................... 9-1 

9.3 Human-System Interface Design ................................................. 9-2 

9.4 Training Program Development ... ............................................. 9-4 

10 REVIEW GUIDELINES FOR THE INFORMATION SYSTEM HIFE DESIGN ................. 10-1 

10.1 General Display Guidelines .................................................... 10-1 

10.2 D isplay Form at ........................................................... 10-4 

10.2.1 Integral Form ats ..................................................... 10-4 

10.2.2 Configural Display Formats ............................................ 10-4 

G LO SSA RY ............................................................................. G -1

NUREG/CR-6633 vii
LIST OF FIGURES 

Page 

3.1 Major Steps in Developing NUREG-0700 Guidance ........................................ 3-1 

3.2 Process for Developing Technical Basis and Guidance ...................................... 3-3 

4.1 Display System Characterization ........................................................ 4-2 

4.2 The Relationship Between Surface Structure and Sentence Meaning ........................... 4-3 

4.3 The Relationship Between Surface Structure and Semantics in Display from a Loss of Coolant 

Accident (LOCA) in a Pressurized Water Reactor (PWR) .................................... 4-3 

4.4 An Example of More Complex Display Semantics from a Loss of Coolant Accident (LOCA) in a 

Pressurized W ater Reactor (PW R) ...................................................... 4-5 

5.1 Generic Primary Tasks of a Supervisory Controller ......................................... 5-8 

5.2• Diagnostic Strategies Used in Troubleshooting ............................................ 5-21 

5.3 Electrical Production Abstraction Hierarchy .............................................. 5-22 

5.4 Physical Interface (P) for DURESS ..................................................... 5-25 

5.5 Physical + Function (P+F) Interface for DURESS ......................................... 5-26 

5.6 EID P+F Interface with Analytical Redundancy ........................................... 5-29 

5.7 Boiling W ater Reactor (BW R) ......... .............................................. 5-32 

5.8 Pressurized W ater Reactor (PW R) ..................................................... 5-33 

5.9 Thermodynamic Diagram of Water and Steam ............................................ 5-34 

5.10 Rankine Cycle ..................................................................... 5-35 

5.11 Rankine Cycle Display .............................................................. 5-36 

5.12 Simulated Analog M eter Display ....................................................... 5-38 

5.13 Rankine Cycle Display of Abnormalities ................................................ 5-39 

5.14 EID Production Goal and Display ...................................................... 5-43 

5.15 Toshiba EID Display ................................................................ 5-44 

5.16 Edlund and Lewis Displays ........................................................... 5-4 6 

5.17 Separable Display With and Without Emergent Features .................................... 5-57

ix NUREG/CR-6633
LIST OF TABLES

Page

Abstraction-Aggregation (A-A) M atrix .................................................. 5-20 

M ean Steady-State Times ........................................................... 5-30 

Operator Cognitive Activity ........................................................... 5-42

NUREG/CR-6633

5.1 

5.2 

5.3

xi
EXECUTIVE SUMMARY 

The Human-System Interface Design Review Guideline, NUREG-0700, Rev. I (O'Hara, et al., 1996), was 

developed to provide guidance on human factors engineering (HFE) for the U.S. Nuclear Regulatory Commission 

(NRC). The NRC staff uses NUREG-0700 for (1) reviewing submittals of human-system interface (HSI) designs prepared by licensees or applicants for a license or design certification of a commercial nuclear power plant (NPP), 

and (2) undertaking HSI reviews that could be included in an inspection or other types of regulatory review of HSI 

designs, or incidents involving human performance. It describes those aspects of the HSI design review process 

that are important to identifying and resolving human engineering discrepancies that could adversely affect plant 

safety. NUREG-0700 also has detailed HFE guidelines for assessing the implementation of HSI designs. 

In generating NUREG-0700, Rev I, several topics were identified as "gaps" because there was an insufficient 

technical basis upon which to formulate guidance. One such topic is the integration of advanced HSI technology 

into conventional NPPs. The NRC is currently sponsoring research at Brookhaven National Laboratory (BNL) to 

(1) better define the effects of changes in HSIs brought about by incorporating digital technology on personnel 

performance and plant safety, and (2) develop FIFE guidance to support safety reviews, should a review of plant 

modifications or HSIs be necessary. 

Based upon the literature, interviews, and site visits, O'Hara et al., (1996) identified changes in HSI technology 

and their potential effects on personnel performance. The topics were then evaluated for their potential safety 

significance (Stubler et al., 1996); information systems was one HSI technology that was found to be potentially 

safety significant. 

The importance of information system design for the performance and reliability of operators has long been 

recognized (Patrick, 1987). In conventional control' rooms (CRs), the large spatially fixed arrangement of controls, 

displays, and alarms supports rapid scanning and pattern-recognition approaches to plant diagnosis. Information 

is primarily presented by detailed parameter indicators on a variety of analog devices, such as gauges and linear 

scales. 

Computer-based systems provide a variety of ways to process and present data. Information may be presented as 

individual parameter displays (as in conventional systems) or in processed form; e.g., raw data parameters are 

integrated to provide higher levels of information. These design trends are toward making displays more 

immediately meaningful to personnel by mapping the presentation of information to underlying cognitive 

mechanisms, such as perceptual processes and mental models. However, this places a significant burden on the 

designer both to anticipate the information needs and map them onto appropriate display formats that correspond 

to the operator's mental models. This is because designs that appeal to engineers are not necessarily well suited to 

plant operations and maintenance personnel. If mistakes are made in the design, then their use may impair rather 

than support performance. This may lead to the initiation of an event or complicate operators' responses to an 

event by causing them to take improper actions or omit necessary actions. 

In general, poor implementation of information systems can lead to human performance problems. Some human 

performance issues associated with display design include the following: Information overload; limited display 

area; lack of overview of plant status, appropriateness of display formats and page arrangements to operator tasks; 

and improper integration and presentation of numerical data. 

We focused our efforts on those aspects of information system design that are most relevant to the nuclear industry, where the research is most active, and where the results are reflected in actual NPP design efforts: information 

requirements, representation, and organization. Information requirements refer to what information operators 

need to safely and efficiently operate the plant. This is a vital step in design of information systems and recent 

approaches to its analysis, such as ecological interface design approaches that largely stem from the nuclear and

NUREG/CR-6633 °..i
process control industries, promise to enhance the analysis of information needs (e.g., Vicente and Rasmussen, 

1992). 

Information representation refers to how the needed information is displayed in the control room. The 

development of novel graphical means to represent information in a format that supports immediate and accurate 

understanding of plant conditions has a long history in the nuclear industry stemming from the use of simple 

polargraphic displays for safety information (e.g., Woods, Wise, and Hanes, 1981) to more recent efforts to 

represent the thermodynamic state of the plant using graphic displays based on the Rankine cycle (Beltracchi, 

1995). 

Finally, structure is imposed on the information representations by organizing them into display pages and the 

pages into an entire display network. This is a significant issue in process control facilities, because the networks 

may contain hundreds or even thousands of display pages (O'Hara, Stubler, and Higgins, 1996). While this aspect 

of information system design has been generally neglected, recent research suggests that organization is significant 

to operator performance, and some common organizational schemes can make it difficult for operators to perform 

their tasks (Heslinga and Herbert, 1995). 

The objective of this study was to develop HFE review guidance for information systems based on a technically 

valid methodology. To support this objective, the following tasks were undertaken: 

* Development of a framework for characterizing key design features of information systems 

* Development of a technical basis using research and analyses on human performance relevant to information 

systems 

• Development of HFE review guidelines for information systems in a format that is consistent with NUREG

0700, Rev. 1, and NUREG-07 11 

Identification of remaining information systems issues for which research was insufficient to support our 

development of NRC review guidance 

The status of each will be briefly addressed below. 

Characterization Framework for Information Systems 

Existing information systems were reviewed to identify the dimensions and characteristics along which 

information systems can be defined. Characterization was important because it provided a structure within which 

the reviewer could request information about a system, and with which to structure the guidance. 

The characterization of information systems included the following major components: information requirements, 

representation systems, management functions, and display devices. Representation systems were further 

hierarchically broken down into display elements (basic building blocks of displays, such as axes, alphanumeric 

elements, and icons), display format (such as piping and instrumentation diagram (P&ID) displays and trend 

graphs), display pages (a defined set of information that is intended to be displayed as a single unit), and display 

networks (the organization of display pages). 

Technical Basis Development 

The development of detailed review guidelines began by collecting technical information on which to base the 

guidance. Research in the following areas was examined: (1) operators' generic cognitive tasks that an 

information system must support, (2) analyses of information requirements, (3) representation of information, and 

(4) organization of information. 

NUREG/CR-6633 xiv

I I
The plant's information systems are the basis for operators to perform their role. The general tasks that the 

information system must support include situation assessment, monitoring and detection, response planning, and 

response implementation. Considerations of the information processing basis needed for the operators' cognitive 

tasks have to be reflected in design review guidance for the information system. The way in which these needs are 

reflected in designing information systems is in analyses of information requirements, the representation of 

information, and the organization of information. 

In NPP design, a systems-engineering approach was recommended for identifying information requirements. Plant 

functions are broken down based on performance requirements and allocated to humans and machines. Human 

functional requirements are further analyzed to better define the information requirements of task performance. 

EID was identified as a potential additional approach that addressed following: 

* Specification of information requirements 

• Organization of information among the levels of an abstraction-hierarchy 

* Development of innovative displays to show functionality 

* Addition of analytical redundancy 

Each of these aspects of EID may have some performance-enhancing features. However, the associated research 

had methodological weaknesses that limited the conclusions that could be drawn. 

Once information requirements are identified, they must be represented in the information system; that is, display 

formats are developed to communicate the information to the operators. The success of representational aids was 

discussed as being a function of the characteristics of the plant, the operators, and the representation through 

which the operators view and act on the plant. All three are essential parts of the overall system. Approaches to 

representation include separable displays (individual parameters displayed in unrelated form), integral displays 

(individual parameters integrated into object display where the individual ones are not visible), and configural 

displays (individual parameters are displayed, but emergent perceptual features are created by the arrangement and 

organization of parameters). 

In general, some research support was found for the representation principles; however, most of the studies could 

not be generalized due to the use of simple systems, tasks, and inexperienced participants. Thus, extending the 

research findings to complex systems, such as NPPs must be done with caution. 

While the organization of the display page and network are important, there were few data on organizing large, 

complex information systems. Such studies are complicated due to the interactions between informational 

organization and other design considerations, such as the display area available and interface management 

functions. Further, some research seemed to contradict current HFE guidance and current nuclear practice; that is, 

very dense displays were found to lead to better performance, were preferred by operators, and the industry's 

common practice of function-system network organization was found to be difficult for operators to use when 

performing certain tasks. 

HFE Review Guidelines 

Once the technical information was assembled, a draft set of guidelines was developed. The guidelines were 

organized and specified in a standard format. In general, guidelines were only developed for those aspects of 

display design that, in our interpretation, are supported by the literature. Many research studies were either weak 

in experimental methodology or limited in generalizability to complex systems. This situation constrains the 

development of new HFE guidelines.

XV NUREG/CR-6633
While there was a sufficient technical basis to develop a detailed design-implementation guidance, as is typical in 

NUREG-0700, several limitations in the technical basis were identified, and so, many issues remain for which 

typical NUREG-0700 guidance could not be developed (the specific issues are listed in the next section). Until 

sufficient information is available to support guidance development in these areas, they can be addressed for 

specific information systems case-by-case using design process review guidance. Thus two types of guidance was 

developed: design process review guidance and HFE design review guidance. 

The HFE design process guidelines- were designed in the standard format adopted in NUREG-07 11 (O'Hara, et al., 

1994). The guidelines were organized into the following sections: 

* Operating Experience Review 

* Function and Task Analysis 

* Human-System Interface Design 

* Training Program Development 

The HFE design guidelines were developed in the standard format adopted in NUREG-0700, Rev. 1. The 

guidelines were organized into the following sections: 

* Guidelines for General Display 

* Display Format 

- Integral Formats 

- Configural Display Formats 

The guidance was peer reviewed and revised. The new guidance will be integrated into the existing guidance on 

information systems in NUREG-0700, Rev. 1. 

Information Systems Issues 

Where there was insufficient information for the technical basis upon which to develop valid design review 

guidance, an issue was defined. There were several human performance issues associated with information 

systems. The issues were organized into three topic areas: 

* Technical Basis Issues 

- Lack of a Well-Defined EID Process 

- Lack of Specific Representation Guidance 

- Evaluation of Operating Experience 

- Critical Testing and Evaluation of EID Concepts 

- Evaluation of Displays 

* Design Review Issues 

- Task and Temporal Considerations 

- Volume of Information 

- Density of Display Information 

- Operator Use of a Large Span and Variety of Displays 

- System Complexity and Emergent Features 

- Perceptual Resolution 

- Elements of Configural Display 

- Effect of Instrumentation Failures 

- Organization of Information 

- Integration of EID Displays into Remainder of HSI 

NUREG/CR-6633 xvi

I I
* Operator-Related Issues 

- Training and Qualification Implications 

- Operators' Acceptance 

- Internal vs. External Mental Models

NUREG/CR-6633 xvii
PREFACE 

This report was prepared by Brookhaven National Laboratory for the Division of Systems Technology of the U.S. 

Nuclear Regulatory Commission's (NRC's) Office of Nuclear Regulatory Research. It is submitted as part of the 

requirements of the project Human Factors Topics Associated with Hybrid Human-System Interfaces (NRC JCN J

6012), specifically, as part of Task 3, "Develop Review Guidance." The NRC Project Manager is Joel Kramer and 

the BNL Principal Investigator is John O'Hara.

xix NUREG/CR-6633
ACKNOWLEDGMENTS 

The authors wish to express their sincere gratitude to our colleagues Lew Hanes; Mike Fineberg and his review 

team at the Crew Systems Ergonomics Information Analysis Center (CSERIAC); and Bill Ruland, Greg Galletti, 

Jim Bongarra, Clare Goodman, Jim Stewart, and Rich Correia of the U.S. NRC for their review of the report. 

These reviewers provided insightful comments and perspectives on the issues addressed in the report, and their 

knowledge and understanding significantly contributed to the study. 

We wish to thank Carl Edlund, John Lee, Neville Moray, Jens Rasmussen, Kim Vicente, and David Woods for 

allowing us to use their graphics; and the American Nuclear Society, Elsevier Science, Human Factors and 

Ergonomics Society, and Valley Litho for granting permission to reproduce copyrighted material. 

We also wish to thank Barbara Roland, Mary Anne Corwin, and Katherine Vivirito for their preparation and 

careful technical editing of the report.

xxi NUREG/CR-6633
ACRONYMS 

AA Abstraction - aggregation 

ABWR Advanced boiling water reactor 

AECB Atomic Energy Control Board (Canadian) 

ALWR Advanced Light Water Reactor 

BNL Brookhaven National Laboratory 

BWR Boiling water reactor 

CBP Computer-based procedure 

CR Control room 

DOE U.S. Department of Energy 

DPI Direct perception interface 

DURESS DUal REservoir System Simulation 

EBR Experimental Breeder Reactor 

EID Ecological interface design 

EOP Emergency operating procedure 

EPRI Electric Power Research Institute 

HFE Human factors engineering 

HRA Human reliability analysis 

HSI Human-system interface 

I&C Instrumentation and control 

IAC Information access cost 

IEC International Electrotechnical Commission 

IEEE Institute of Electrical and Electronics Engineers 

ISLOCA Interfacing systems loss of coolant accident 

LOCA Loss of coolant accident 

NAS National Academy of Science 

NPP Nuclear power plant 

NRC U.S. Nuclear Regulatory Commission 

OER Operating experience review 

PCP Proximity-compatibility principle 

P&ID Piping and instrumentation diagram 

PRZR Pressurizer 

PWR Pressurized water reactor 

RCS Reactor coolant system 

RHR Residual heat removal 

RV Reactor vessel 

SA Situation assessment 

SDT Signal detection theory 

SME Subject matter expert 

SPDS Safety parameter display system 

SRK Skill-based, rule-based and knowledge-based 

TMI Three Mile Island nuclear power station 

URD Utility Requirement Document 

V&V Verification and validation 

VDU Video display unit

xxiii NUREG/CR-6633
PART 1 

Guidance Developmentand Technical Basis

NUREG/CR-6633
1 INTRODUCTION 

1.1 Background 

The Human-System Interface Design Review Guideline, NUREG-0700, Rev. I (O'Hara et al. 1996), was developed 

to provide guidance on human factors engineering (HFE) to the U.S. Nuclear Regulatory Commission (NRC). The 

NRC staff uses NUREG-0700 for (1) reviewing the submittals of human-system interface (HSI) designs prepared 

by licensees or applicants for a license or design certification of a commercial nuclear power plant (NPP), and (2) 

conducting reviews of HSIs that could be undertaken as part of an inspection or other type of regulatory review 

involving HSI design or incidents involving operator's performance. It describes those aspects of the HSI design 

review that are important to identifying and resolving discrepancies in human engineering that could adversely 

affect plant safety. NUREG-0700 also details HFE guidelines for assessing implementations of HSI design. 

In developing NUREG-0700, Rev. 1, several topics were identified as "gaps," because there was an insufficient 

technical basis upon which to develop guidance (O'Hara, 1994; O'Hara, Brown, and Nasta, 1996). One such topic is the integration of advanced HSI technology (such as advanced information systems) into conventional NPPs. 

The NRC is currently sponsoring research at Brookhaven National Laboratory (BNL) to (1) better define the effects 

of changes to HSis brought about by applying digital technology on personnel's performance and plant safety, and 

(2) develop guidance on HFE to support safety reviews if a review of plant modifications or HSIs is necessary. 

This guidance will be integrated into NUREG-0700 and will be used to provide the NRC staff with the technical 

basis to ensure that the modifications or HSI designs do not compromise safety. 

The results of this project will contribute to satisfying the NRC's goals of (1) maintaining safety, (2) increasing 

public confidence, (3) increasing regulatory efficiency and effectiveness, and (4) reducing unnecessary burden. 

Based upon literature, interviews, and site visits, HSI technology trends and their potential effects on personnel 

performance were identified (O'Hara, Stubler, and Higgins, 1996). One trend was Information Design and 

Organization. The importance of information system design for the performance and reliability of operators has 

long been recognized (Patrick, 1987). In conventional control rooms (CRs), the large spatially fixed arrangement 

of controls, displays, and alarms supports rapid scanning and pattern-recognition approaches to plant diagnosis. 

Information is primarily presented by detailed parameter indicators on a variety of analog devices, such as gauges 

and linear scales. 

Computer-based systems provide a variety of ways to process and present data. Information may be presented as 

individual parameter displays (as in conventional systems) or in processed form; e.g., raw data parameters are 

integrated to provide higher levels of information. Several potential benefits of computer-based information 

systems were identified (Lorenzo and Brune, 1985): 

* Ability to perform calculations that show the status of plant parameters and calculated variables 

(e.g., subcooling margin and temperature mismatch) 

* Digital readouts of values (e.g., from trend plots) 

Ability to graphically represent status of equipment (e.g., the actual open or closed status of a valve presented 

along with the normal or desired status) 

* Improved presentation of trend information 

* Ability to integrate the many and diverse indications into a compact form (e.g., a single display page may 

contain indications that would be distributed over several feet of a conventional control panel)

I-1I NUREG/CR-6633
1 INTRODUCTION

These design characteristics represent a trend toward making displays more immediately meaningful to personnel 

by mapping the presentation of information to underlying cognitive mechanisms, such as perceptual processes and 

mental models. However, this places a significant burden on the designer both to anticipate the information needs 

and map them onto appropriate display formats that correspond to the operator's mental models. This is because 

designs that appeal to engineers are not necessarily well suited to plant operations and maintenance personnel. If 

mistakes are made in the design, then their use may impair rather than support performance. This may lead to the 

initiation of an event or complicate operators' responses to an event by causing them to take improper actions or 

omit necessary actions. 

In general, poor implementation of information systems can create human performance problems, as shown by the 

NRC's experience with safety parameter display systems (SPDSs) (Eckenrode, 1991; Liner and DeBor, 1987). 

Modem computer-based information systems are considerably more complex than the SPDSs of the 1980s. Some 

human performance issues associated with display design include the following: 

Information Overload- The vast quantity of information that is available through the display system may impose 

high levels of mental workload. For example, demands on the operator's attention may be high (e.g., the operator 

may need to view multiple sets of information at the same time or in rapid succession). Also, the need to consider 

the relevance of a vast set of information to current task requirements may impose high demands on the operator's 

ability to analyze information. 

Appropriateness of Display Formats and Page Arrangements to Operator Tasks - Modem information systems typically access and present a very large quantity of data and present information in a variety of formats from 

digital to advanced graphic formats. Graphic display formats can represent information related to plant functions, 

processes, and systems in ways that are different from conventional CR instrumentation and earlier computer

based display systems. For example, graphic displays contain many more elements (e.g., abbreviations, labels, 

icons, symbols, coding, and highlighting) than conventional ones. As displays convey more information at 

multiple levels of abstraction, the complexity of the elements becomes greater. The cognitive principles that 

support the effectiveness of these formats, and their relationship to operators' tasks, are becoming better 

understood. A greater understanding is emerging about the effectiveness of designs of display pages, including 

how display formats are combined to produce effective pages, how different formats might interact within a display 

page, and how complex information should be divided to form individual pages. Thus, guidelines on human 

factors that were developed for simple displays may not be appropriate for the complex ones that are being developed for computer-based display systems. If properly implemented, graphical displays can enhance detection, 

recognition, and comprehension of important information. However, improper implementation may impair an 

operator's performance. For example, visual clutter may be increased, which may inhibit an operator from 

detecting important information. New display formats may increase mental workload associated with recognizing 

and comprehending information. Inefficient distribution of information among display pages may result in high demands on an operator's short-term memory (e.g., remembering values from previous displays) or a loss of 

understanding of plant status. 

Limited Display Area - A limited display area in visual display unit (VDU)-based systems has been found to be a 

problem. Further, compact control consoles and computer-based display devices, which contain multiple levels of 

information, may not be consistent with the scanning and recognition strategies of operators (NAS, 1995). When. 

the information needed by an operator is contained on more than one display screen and only one screen can be 

displayed at one time, the operator may be required to make rapid transitions between screens, try to remember 

values, or write them on paper. Compared to the sweeping wall-to-wall panels of older conventional CRs, CRs that 

use these computer-based displays provide a more restricted view, even when multiple display devices are 

NUREG/CR-6633 1 -2

I I
I INTRODUCTION

provided. Further, since operators can only monitor what they can see, they may concentrate excessively on 

selected areas while ignoring others. 

Lack of Overview - A vast amount of information may be available at different levels of abstraction (e.g., 

functional versus physical characteristics of the plant) and level of detail. This information may be distributed 

among separate display pages. As a result of the quality, type, and distribution of information in the display 

system, operators may experience difficulty in obtaining a rapid, overall assessment of the plant's status. The lack 

of an adequate overall assessment may impair the operator's situation awareness and ability to respond to 

transients and accidents. 

Improper Integration and Presentation of Numerical Data - Digital displays can summarize and combine 

numerical data from a wide range of sources and present them in new ways, such as calculating simple, weighted, 

and running time-interval averages; summed values; and differences between values. If properly implemented, 

these displays can support an operator's understanding of the plant. However, the improper presentation of 

integrated data from the plant may confuse or mislead operators. Operators may not fully understand the nature of 

the data that are contained in the display, the data used in calculations, or the calculations. These factors may also 

interfere with the operator's ability to verify displayed values against other data from the plant. For example, a 

digital SPDS display may present the average of a set of redundant wide-range measurement instruments, rather 

than presenting the individual values. If the average is carried out to one or more decimal places, it may appear to 

have the accuracy of a narrow-range measurement instrument and, thus, may mislead the operator. As another 

example, a polar-graphic display (e.g., octagon) may plot the current value of a variable against a reference value. 

However, the basis for determining the reference value may not be apparent to the operator. 

Once the trends in HSI design and the potential human performance issues associated with them were identified, 

each trend (O'Hara, Stubler, and Higgins, 1996), was evaluated for its potential safety significance. The 

evaluation indicated that display design and organization was potentially safety significant and that further 

research was needed (Stubler, Higgins, and O'Hara, 1996). That research is the subject of this report. 

1.2 Scope of the Research 

The literature addressing the design of information systems is extensive. Reviewing all of it was well beyond the 

scope of the project. Therefore, we focused our efforts on those aspects of information system design that are most 

relevant to the nuclear industry, where the research is most active, and where the results are reflected in actual 

NPP design efforts: information requirements, representation, and organization. Information requirements refer 

to what information operators need to safely and efficiently operate the plant. This is a vital step in design of 

information systems and recent approaches to its analysis, such as ecological interface design approaches that 

largely stem from the nuclear and process control industries, promise to enhance the analysis of information needs 

(e.g., Vicente and Rasmussen, 1992). 

Information representation refers to how the needed information is displayed in the control room. The 

development of novel graphical means to represent information in a format that supports immediate and accurate 

understanding of plant conditions has a long history in the nuclear industry stemming from the use of simple 

polargraphic displays for safety information (e.g., Woods, Wise, and Hanes, 1981) to more recent efforts to 

represent the thermodynamic state of the plant using graphic displays based on the Rankine cycle (Beltracchi, 

1995). 

Finally, structure is imposed on the information representations by organizing them into display pages and the 

pages into an entire display network. This is a significant issue in process control facilities, because the networks

1-3 NUREG/CR-6633
1 INTRODUCTION

may contain hundreds or even thousands of display pages (O'Hara, Stubler, and Higgins, 1996). While this aspect 

of information system design has been generally neglected, recent research suggests that organization is significant 

to operator performance, and some common organizational schemes can make it difficult for operators to perform 

their tasks (Heslinga and Herbert, 1995). 

The specific information determined to be needed by operators, its representations, and its organization into an 

entire information network can significantly impact operator performance and nuclear safety. 

1.3 Organization of the Report 

The report is divided into two parts. Part I describes the guidance development methodology and basis. The 

objectives of the study are described in Section 2, and the guidance development methodology is described in 

Section 3. Section 4 presents a characterization of information systems. Section 5 discusses the literature and 

information that served as the technical basis upon which the review guidance was developed. The actual use of 

the technical information to develop guidance is discussed in Section 6. Section 7 summarizes the guidance 

development. References are in Section 8. 

Part 2 contains the results of the process for formulating the guidance, and is presented in two sections. Section 9 

discusses the design process considerations for reviewing information systems, and Section 10 contains the HFE 

design guidelines for the review of an implemented information system design.

NUREG/CR-6633 1-4

I I
2 OBJECTIVE 

The objective of this study was to develop HFE review guidance for information systems - specifically with respect 

to information requirements analysis, information representation, and information organization - based on a 

technically valid guidance development methodology. To support this objective, the following tasks were 

undertaken: 

• Development of a characterization framework for describing key design characteristics of information systems 

* Development of a technical basis using research and analyses of human performance that are relevant to 

information systems 

* Development of HFE review guidelines for information systems in a format that is consistent with NUREG

0700, Rev. 1, and other NRC review guidance 

* Identification of remaining information system issues for which research results were insufficient to support 

developing NRC review guidance

2-1 NUREG/CR-6633
3 METHODOLOGY 

3.1 Overview 

Figure 3.1 shows the methodology for the overall guidance development for NUREG-0700. The process for this 

development is discussed in detail elsewhere (O'Hara, Brown and Nasta, 1996). The methodology that applies to 

this report and project is boxed in the figure. This section describes the rationale for general guidance 

development.

Figure 3.1 Major Steps in Developing NUREG-0700 Guidance 

The methodology was guided by the following objectives: 

• Establish a process that will result in valid, technically defensible, review criteria. 

• Establish a generalizable process that can be applied to any aspect of HSI technology for which review 

guidance is needed. 

* Establish a process that optimally uses available resources, i.e., develop a cost-effective methodology. 

The methodology places a high priority on establishing the validity of the guidelines. Validity is defined along two 

dimensions: internal and external validity. Internal validity is the degree to which the individual guidelines are 

based on an auditable technical basis. The technical basis is the information upon which the guideline is 

established and justified. The technical bases vary for individual guidelines. Some guidelines may be based on 

technical conclusions from a study of empirical research, some on a consensus of existing standards, while others 

are based on judgement that a guideline represents good practices based on the information reviewed. Maintaining 

an audit trail from each guideline to its technical basis serves several purposes by enabling the following: 

• Technical merit of the guideline to be evaluated by others 

* A more informed application of the guideline since its basis is available to users 

• Deviations or exceptions to the guideline to be evaluated

3-1 NUREG/CR-6633
3 METHODOLOGY

External validity is the degree to which the guidelines are independently peer reviewed. Peer review is a good 

method of screening guidelines for conformance to accepted HFE practices and for comparing guidelines to the 

practical operational experience of HSIs in real systems. 

For individual guidelines, these forms of validity can be inherited from the source documents that form their 

technical basis. Some HFE standards and guidance documents, for example, already have good internal and 

external validity. If validity is not inherited, however, it should be established as part of the process for guidance 

development. Methodology was established to provide validity both inherited from its technical basis and through 

developing and evaluating guidance. 

Figure 3.2 shows the process used to develop the technical basis and guidance. The process emphasizes 

information sources that have the highest degree of internal and external validity for developing the technical 

basis. Thus, primary and secondary source documents were sought as sources of guidance first, followed by 

tertiary source documents, basic literature, industry experience, and other sources. From these, we identified 

design principles and lessons from industry experience. Using this technical basis as a foundation, the guidance 

was developed. For specific aspects of the topic, in which the technical basis was inadequate for developing 

guidance, we defined unresolved research issues. Thus, the technicil basis led to the development of both guidance 

and issues. The resulting guidance documentation includes HFE guidelines, technical basis, the development 

methodology, and unresolved research issues. 

Each of the steps of this research - topic characterization, development of technical basis, guidance development 

and documentation, identification of issues, and peer review - is discussed in greater detail in the sections that 

follow. 

3.2 Characterization of the Information System 

The first step in developing guidance was to identify the areas for which it was needed. We reviewed existing 

information systems to identify the dimensions and characteristics along which information systems can be 

defined. The characterization was important because it provided a structure within which the reviewer could 

request information about a system and with which to structure the guidance for design review. The 

characterization of information systems is presented in Section 4. 

3.3 Development of the Technical Basis 

We began to formulate detailed review guidelines by collecting technical information on which guidance would be 

based (Figure 3.2). The process was designed to formulate valid guidance cost effectively. First, primary source 

documents, HFE standards and guidance documents possessing internal and external validity, were sought. That 

is, the primary source documents generally provided their own research bases, and the document developers 

considered the available research and operational experience. Using their knowledge and expertise, they developed 

HFE guidelines. These primary source documents were often extensively peer reviewed. The documents added 

tremendous value to individual research reports. They were developed by experts who consider research in terms 

of its applicability and generalizability to real systems, include knowledge and expertise gained through 

operational experience and guidance application, and modify the guidance based on extensive peer review. 

However, we found little information that addressed advanced display design. While several documents address 

display design in nuclear plants, their applicability and guidance for advanced display systems is limited. These 

documents are 

NUREG/CR-6633 3-2

I I
3 METHODOLOGY

I Lieratre Basc IDevelopment of 

usiLteryGuidance and 

Sufficient Yes Documentation 

e.g., scientific, technical, and HFE guidelines 

trade journals Technical basis Development methodology 

Industryufien Experience No . 

Se.g., interviews with operators, 

designers, and researchers 

Original Research 

Sufficient 

e.g., studies conducted 

specifically to develop guidance 

No 

Identification of 

Unresolved 

Issues 

Figure 3.2 Process for Developing Technical Basis and Guidance 

Electric Power Research Institute's Advanced Light Water Reactor Utility Requirements Document (URD) 

(EPRI, 1993) 

International Electrotechnical Commission's (IEC) Design for Control Rooms of Nuclear Power Plants (IEC

964) (IEC, 1989) 

3-3 NIIREG/CR-66"

Development of Technical Basis 

Primary and Secondary 

Source Documents Yes 

Sufficient 

e.g., HFE standards 

and guidelines 

No 

Tertiary Source 

Documents Yes 

Sufficient 

e.g., HFE handbooks 

lNo

33
3 METHODOLOGY

* Institute of Electrical and Electronics Engineers' (IEEE) Guide for the Application of Human Factors 

Engineering in the Design of Computer-Based Monitoring and Control Displays for Nuclear Generating 

Stations (Standard P1289) (IEEE, 1996) 

Section 10.4.4, Displays, of the EPRI URD addresses display requirements in advanced NPPs. Requirements are 

identified for trend displays, integrating displays and mimics (similar to group view displays discussed in Stubler 

and O'Hara, 1996), and electronic displays at workstations. However, the URD does not address specific formats 

or how displays should be organized. It states that the detailed design of displays should be tested using 

simulation. 

Section 4.6, Information Systems, of IEC-964 discusses displays in general and refers to Appendix A Section 

A.4.6, Information Systems, for more details. However, neither section gives more than high-level principles, such 

as the need for consistency, and they do not address specific requirements for individual display formats, or the 

organization of information within the network. 

IEC-964 also contains a supplement on "Visual Display Unit Application to Main Control Room in Nuclear Power 

Plants." The supplement gives applications, advantages, and disadvantages of alphanumerical displays, bar 

graphs, trend curves, system status displays, and logic displays. However, there is no detailed guidance for any 

formats, and more advanced displays and organization are not addressed. IEC-964, like the URD, requires 

simulator testing as part of the verification and validation (V&V) for advanced VDU-based systems. 

Section 6, Typical Display Format Types, of IEEE-P1289 addresses graphs, mimics, logic displays, faceplate 

displays, alphanumeric displays, and integrated displays and discusses the application, advantages and 

disadvantages of each. The integrated display refers to a display that integrates several types of displays within a 

single one and should not be confused with an integral display. 

Thus, the scope of these documents is addressed in NUREG-0700, Rev. 1 (Section 5.1 addresses the scope of 

NUREG-0700 guidance), and therefore, guidance for more advanced information systems, the focus of our current 

research, is not addressed. Consequently, while developing a technical basis for information systems we 

considered the other sources shown in Figure 3.2. Secondary sources were documents for which either internal or 

external validity was established. Documents for which neither type of validity was established were considered 

tertiary sources. The preference was to use documents for which validity had been established. 

This information includes the results from basic literature that we analyzed. While the information pertaining to 

advanced information systems was limited in current standards and guidance documents, the situation is 

completely reversed in the basic literature. The literature is extensive; therefore, we focused on using published 

literature reviews where available, and supplementing them with more recent published accounts of individual 

studies. 

When guidance was based on basic literature, engineering judgement was required to generalize from the unique 

aspects of individual experiments and studies to actual applications in the workplace. This is because individual 

experiments have unique constraints that limit their generalizability (such as their unique participants, types of 

tasks performed, and types of equipment used). For example, laboratory experiments often do not involve tasks of 

the complexity of NPP operations, and most experiments do not examine tasks under the same performance 

shaping factors (such as rotating shifts, stress, and fatigue) that exist in a work environment. While information 

from research is a valuable part of developing guidance, it usually cannot be blindly adopted. Thus, the results 

must be interpreted in the context of real-world tasks and systems, which involves judgement based on professional 

and operational experience. 

NUREG/CR-6633 3-4

I I
3 METHODOLOGY

Industry experience also was used, such as published case studies and surveys and interviews with knowledgeable 

domain experts. Site visits and interviews with operators, trainers, and designers were conducted as part of an 

earlier phase of this project, and pertinent information from that effort was used. Although such information may 

lack a rigorous experimental basis (and thus, a measure of validity), it is highly relevant. 

Finally, some information was identified in original research. A full account of the research is published elsewhere 

(Roth and O'Hara, 1998). Original research has the advantage of enabling a study to be focused on the specific 

issues that need to be addressed in guidance development. However, because of the time and resources required to 

conduct original research, it is only used when important information is needed that cannot be obtained through 

other means. 

Empirical evaluations of information systems and characteristics provided a substantial portion of the technical 

basis upon which to develop design review guidance. The general criteria by which such studies were evaluated 

will be briefly described. One of the most important considerations is that the information system studies provided 

a basis from which conclusions can be generalized beyond the specific individual study. To make this evaluation, 

the studies were considered within the context of validation studies where generalization (external validity) is a 

primary consideration. 

O'Hara et al. (1997) discussed the detailed methodological considerations for validation of complex human

machine systems. A conceptual approach to validation was developed that identified important validation 

principles and their relationships. The general concepts are concerned with (1) establishing the requirements for 

making a logical and defensible inference from validation tests to predicted integrated system performance under 

actual operating conditions, and (2) identifying the aspects of validation methodology that are important to the 

inference process. The technical basis for inference in validation is based upon four general forms of validity: 

system representation, performance representation, test design, and statistical conclusion. 

System representation validity refers to the degree to which the tests include those aspects of the system that are 

important to real-world conditions. Specifically, this validity is based on the representativeness of the system 

model, human-system interface, personnel, and operational events. The inference process is supported to the 

extent that important aspects of the integrated system are represented with high fidelity, and to the extent to which 

important contributors to potential variability in system performance have been adequately sampled. 

Performance representation validity refers to the completeness and representativeness of the performance measures. 

A comprehensive, hierarchal approach to evaluation guided by supervisory control theory may be used to specify 

important aspects of performance ranging from operator cognitive processes to system functions. In general, the 

effects can be related to (1) personnel role - a change in functions and responsibilities of plant personnel; 

(2) primary tasks - a change in the way that personnel perform their primary tasks, such as process monitoring, 

situation assessment, response planning, and response implementation and control; (3) secondary tasks - a change 

in the tasks the operator must perform when interacting with the information system, such as navigating through 

displays and searching for data; (4) cognitive factors - such as a change in cognitive workload; and (5) personnel 

factors - a change in the required qualifications or training of plant personnel. Performance measures must be 

used that can address these effects. Failure to include measures of all important performance variables, poor 

measurement properties, and poorly specified criteria weaken this validity and the ability to generalize the results. 

Test design validity addresses the experimental design and test procedures. Experimental design considerations 

address the suitability of the definitions of independent variables. Inappropriate test procedures can bias the 

relationship between the observations of performance and the integrated system, and thus undermine their causal

3-5 NUREG/CR-6633
linkage. When factors introduced by the test methodology weaken the ability to interpret the system-performance 

correlation, validity is compromised. 

Finally, statistical conclusion validity addresses the relationship between the observed data and established criteria 

of performance. 

While the studies reviewed in Section 5 were evaluated for these types of validity and their associated 

methodological considerations, such an analysis is limited by the extent to which the studies are documented. 

3.4 Development of Guidelines and Documentation 

Once the technical information was assembled, a draft set of guidelines, which are in Part 2, was developed from the information in the technical basis. The guidelines were organized and specified in a standard format (which is 

discussed in Section 6). 

3.5 Identification of Issues 

Where there was insufficient information to provide a technical basis upon which to develop valid design review 

guidance, an issue was defined. Section 7 describes these issues. 

From a research standpoint, issues reflect aspects of the design and use of information systems that will require additional investigation to resolve. From a design review standpoint, issues reflect aspects of design and use that will have to be addressed on a case-by-case basis. For example, an issue can be addressed as part of design-specific 

tests and evaluations. 

3.6 Peer Review 

The resulting technical basis and guidance was submitted for review by individuals with knowledge and expertise related to the review topic. Included were reviews by personnel from the U.S. NRC with expertise in FIFE and engineering fields directly related to the topic. Human factors specialists who are external to the NRC and have 

expertise in human performance in complex systems, such as NPPs and aviation, conducted additional reviews. 

These external reviews include evaluations of the topic characterization along the criteria of clarity, accuracy, and completeness, and the technical basis along the criteria of organization, necessity, sufficiency, resolution, and 

basis. Comments from the peer reviews were incorporated into the current version of this document.

NUREG/CR-6633 3-6
4 CHARACTERIZATION OF INFORMATION SYSTEMS 

The plant information system refers to those aspects of the HSI that give information to the operator about the 

plant state. NUREG-0700, Rev. 1 provided a preliminary characterization of information system components. In 

this section, that characterization is made more definitive and expanded to accommodate the results of this 

research. 

An information system has the following major components: information requirements, representation system, 

management functions, and display devices. Each is briefly described below. This section gives a general 

characterization of information systems, whether they are advanced or not. The term advanced, as used in this 

report, refers to information systems that use novel graphic forms, such as a Rankine Cycle display, and go beyond 

conventional forms, such as bar charts and trend graphs. This general characterization is consistent with other 

approaches to define the hierarchical organization of information and display systems (e.g., Woods et al., 1991). 

This report addresses the first two aspects of the characterization: information requirements and representation 

systems. The topic of management functions is being addressed in another NRC research project because of its 

general applicability to human-computer interfaces, such as soft controls and computer-based procedures, in 

addition to information systems. The topic of display devices is hardware related and is being considered 

separately. 

4.1 Information Requirements 

Information requirements refer to the information operators need to monitor and control the plant. Information is 

at the center of human functions in any complex system. The operators' information needs are the requirements 

that must be addressed in the design of the information system. The determination of what information is needed 

is referred to as information requirements analysis. New approaches to information analyses conducted to define 

the information requirements of display systems have emerged and will be discussed in Section 5.3. 

4.2 Representation System 

While the information requirements identify what information operators' need, the way in which that information 

is presented to operators is called the representation system. Information is given to the operators via some sensory 

channel. In a nuclear plant, it is predominantly the operator's visual and auditory channels that are used to convey information about the plant. The representation system is the means by which information is communicated to 

operators; it contains the following components: elements, formats, pages, and networks. Each is discussed below. 

Figure 4.1 shows the representation system (along with management functions and display devices). 

Display Elements 

Display elements refer to the basic building blocks that are used to make up display formats, such as abbreviations, 

labels, icons, symbols, coding, and highlighting. 

Display Format 

Display formats refer to the general class of information presentation. Examples of general classes are continuous 

text (such as a procedure display), mimics and piping and instrumentation diagram (P&ID) displays, trend graphs, 

and flowcharts. This aspect of representation systems has been most affected by recent trends in design of 

information systems. The ability of computer graphics to portray an essentially limitless set of novel graphic forms 

has offered great possibilities to provide operators with enhanced representations of the plant. Woods et al. (1991) 

4- 1 NUREG/CR-6633
4 CHARACTERIZATION OF INFORMATION SYSTEMS

Display Display Display Device Page Network 

A. 

Safety Function Display / \ 

vv \ 

Ix "-" Display 

"Elements 

Zoom Dlsolay 

S. L- ,, I 

V 

Management Display 

Functions Formats 

Figure 4.1 Display System Characterization 

(Information requirements not shown.) 

refer to this level as "Graphic Forms" and consider it the "fundamental hinge in the series of levels." The same can 

be said of display formats in the current characterization. Below this level, the characterization seeks to define the 

building blocks of display formats. Above this level, formats are successively organized into pages and then into 

networks. Thus, display formats, which are discussed in Section 5.4, are the most significant "unit of analysis" of 

the information system. 

There is a grammatical aspect of display formats similar to human language. In both cases the grammar describes 

the relationship between the physical form (e.g., sentence or visual aspects of a graphic display) and its meaning. 

The link between physical form and meaning is made using transformation rules. To illustrate this point, an 

example from linguistics is described, followed by an example from graphic displays. Figure 4.2 shows the link 

between a sentence and its meaning through transformational rules. Panel (a) in Figure 4.2 shows the case where 

the physical form (sentence) is unambiguously linked to meaning (i.e., the sentence form has only one grammatical 

meaning). However, some sentence structures are ambiguous; i.e., the same physical form (sentence) can lead to 

more than one meaning (see panel (b) Figure 4.2). The left sentence is about apple eaters, while the right describes 

a type of apple that is suitable for eating. 

Just as the same sentence can have two distinct meanings, two very different sentence structures can have the same 

meaning. Thus, there is not always a one-to-one mapping between the physical form and semantics. 

Similar relationships can exist between a graphic display and its meaning(s). "Display semantics" pertains to the 

meaning of the display form for the plant (see Figure 4.3). The semantics must consider the instrumentation and 

the data processing that are used to drive its format. Thus, the display's grammar describes the correspondence 

between the display and the aspects.of the system it represents by relating its physical form and functions to its 

meaning to the plant's functions and states. There are syntactical rules that relate the display form and its 

semantics. 

NUREG/CR-6633 4-2

I I
4 CHARACTERIZATION OF INFORMATION SYSTEMS

Sentence 

Noun phrase verb noun phrase SI I 

The ball hit the boy. 

(a) Unambiguous Sentence 

Sentence Sentence 

Ver hrseVerb hrase 

Noun phrase verb noun phrase Noun phrase verb noun phrase I I I I 

They are eating apples. They are eating apples. 

(b) Ambiguous Sentence 

Figure 4.2 The Relationship Between Surface Structure and Sentence Meaning 

(Adapted from Reynolds and Flagg, 1977) 

CORE EXIT CORE EXIT 

618/618*F 6181618*F 

33 SUBCOOL 33 SUBCOOL 

WID RCS PRESS STARTUP WID RCS PRESS STARTUP 

2235/2235 PSIG 0 0 DPM 2235t2235 PSIG 0 0 DPM 

PRZR LEV CNTMT PRESS PRZR LEV CNTMT PRESS 60160% 0 PSIG 40160% \ 0 PSIG 

RV LEV RAD CMITNT RV LEV ,RAD CMTNT 100/100% SEC 100/100% SEC 

OTHER OTHER WID SG LEV WID SG LEV 

50150% 50150% 

(a) (b) 

Figure 4.3 The Relationship Between Surface Structure and Semantics in Display from a 

Loss of Coolant Accident (LOCA) in a Pressurized Water Reactor (PWR) 

(Source: Reprinted with permission from Woods, D., Wise, J., and Hanes, L., "An Evaluation of Nuclear 

Power Plant Safety Parameter Display Systems," Proceedings of the Human Factors Society 25th Annual 

Meeting. Copyright 1981 by the Human Factors and Ergonomics Society. All rights reserved.)

4-3 NUREG/CR-6633
4 CHARACTERIZATION OF INFORMATION SYSTEMS

In the display's design, it is important to have a unique meaning for each form. Just as the surface structure of 

language can be ambiguous with regard to its semantics, so can a display's form. If a display can lead to more 

than one interpretation, it is ambiguous and can be more easily misunderstood. The meaning of graphic format in 

Figure 4.3 (from Woods et al., 1981) shows this. Figure 4.3 (a) is an example of a polargraphic, iconic display 

developed to provide a NPP operator with a high-level overview of the status of plant safety functions. The display 

is driven by eight parameters (that correlate to the plant safety functions). Each parameter is represented by a 

spoke on the format. The full scale of the parameters' values are displayed over the length of the spoke, between 

the two tick marks. Points representing normal conditions are plotted as fixed points equidistant from the center of 

the figure. Connecting the points results in a regular geometric pattern, an octagon. The reference points are 

dynamically scaled, so that the reference octagon remains regular. Parameters that are higher than normal are 

plotted further away from the center, and those that are lower are plotted closer to the center. 

When a disturbance occurs, the individual parameter values being monitored as representative of safety functions 

change. This changes the points on the spokes, and hence, a new "distorted" polygon is drawn. Figure 4.3 (b) 

shows a case where the pressurizer level decreased from the normal value of 60% to 40%. However, this situation 

can have more than one interpretation. For example, this decrease may be due to a leak and subsequent loss of 

water from the primary system, a failure of the normal pressurizer level control system, or a failure of the 

pressurizer level instrumentation. Thus, the same physical form can lead to distinctly different meanings. 

This type of grammatical relationship does not only apply to graphic displays, although the rules for graphic 

displays may generally be more complicated. One of the lessons from the Three Mile Island (TMI) accident was 

that the display's grammar can be significant even for simple displays. The control board at TMI contained a light 

indicating the position of a relief valve on the pressurizer. Operators thought that the valve was physically open 

when the light was on and closed when it was off. However, the true meaning of the display was that the valve 

solenoid operator was either energized or not. During the accident, the relief valve was actually stuck open and not 

closed as the operators thought, even though the indicator light had gone out. There was a disconnect between the 

display form, the status indicator, and its meaning (solenoid energized) that contributed to a significant 

misunderstanding about the plant. 

A polargraphic display similar to that shown in Figure 4.3 also can be useful for analysis during major transients 

and accidents. In this case, the semantics of the display are more involved. Figure 4.4 shows the figure resulting 

from a significant loss of coolant accident (LOCA) in a pressurized water reactor (PWR). In a LOCA, a major 

pipe break occurred in the primary system, resulting in a large loss of radioactive reactor coolant to the 

containment's atmosphere. In Figure 4.4, pressurizer (PRZR) level, reactor vessel (RV) level, and reactor coolant 

system (RCS) pressure decreased due to the water leak. Core exit temperature increased from the loss of water and 

resultant lack of cooling to the reactor fuel. Containment pressure also increased due to the high pressure/high 

temperature steam and water mix flowing into the containment from the RCS. The radiation level in the 

containment also increased due to radioactive water and steam in containment. As shown in Figure 4.3 (b), as the 

values of the individual parameter change, the polygon distorts. For the LOCA situation, the amount of distortion 

indicates a large deviation in many parameters from normal (still indicated by the dotted regular octagon). 

However, the operator cannot draw any specific conclusions from the shape of the polygon, but rather must think 

about the meaning of each parameter's deviation and then about what they mean in the aggregate. This requires 

substantial training and is aided by detailed emergency procedures. 

For any information display format, it is important to clearly understand all three components of the display's 

grammar: the format itself, its semantics or meaning, and the rules that relate the two. 

NUREG/CR-6633 4-4

I I
4 CHARACTERIZATION OF INFORMATION SYSTEMS

CORE EXIT 

579/4220 F 

0 SUBCOOL 

WID RCS PRESS STARTUP 

1265/2235 PSIG 0 0 DPM 

768 MCP 

/I 

PRZR LEV / CNTMT PRESS 

0/41% 15 PSIG 

RV LEV RAD CMTNT 

70/100% SEC 

OTHER 

WID SG LEV 

39/50% 

Figure 4.4 An Example of More Complex Display Semantics from a Loss of Coolant 

Accident (LOCA) in a Pressurized Water Reactor (PWR) 

(Source: Reprinted with permission from Woods, D., Wise, J., and Hanes, L., "An Evaluation of 

Nuclear Power Plant Safety Parameter Display Systems," Proceedings of the Human Factors Society 

25th Annual Meeting. Copyright 1981 by the Human Factors and Ergonomics Society.. All rights 

reserved.) 

Display Pages 

Display page refers to a defined set of information that is intended to be displayed as a single unit. Typical display 

pages of NPPs may combine several different formats on a single VDU screen, e.g., combining bar charts and 

digital displays in a graphic P&ID format. 

What goes into one display page, i.e., the integration of formats that make up the page, is typically meant to show 

some aspect of the process. For example, one page may provide a high-level status overview of the primary system. 

Display pages typically have a label and designation within the computer system so they can be accessed by 

operators as a single "display." Section 5.5 addresses display pages. 

Display Networks 

Complex systems, such as NPPs, are not represented by a single graphic display. In fact, for many recently built 

plants, the numbers of display pages is more typically in the hundreds or thousands (O'Hara, Stubler, and Higgins, 

1996). Display networks refer to an entire set of display pages within an information system. 

To perform their functions and tasks, operators must access these pages. When the number of pages is large, 

knowing where information is located is difficult. Thus, the organizational structure of the network within the 

information system is an important consideration. A common type of organizational structure, for example, is by 

plant system. Section 5.5 addresses display networks.

4-5 NUREG/CR-6633
4 CHARACTERIZATION OF INFORMATION SYSTEMS

4.3 Management Functions 

As discussed above, display networks can include hundreds or more pages. Further, the information may be visible 

within a limited display area provided by video display units (VDUs). Thus, facilities, such as navigating within 

the display network to retrieve displays, to manipulating display pages (such as sizing or moving windows), and 

moving within a single display page (such as scrolling and zooming), are needed so that operators can manage the 

information. Generically, these are called interface management functions. While they are important 

considerations in information system design, they will not be addressed in this report. Interface management is 

being addressed in a separate project (O'Hara, Stubler, and Nasta, 1997). 

4.4 Display Devices 

Display devices are the media used to present information to plant personnel. Display devices include meters, 

gauges, VDUs, and hard-copy display devices.(printers and plotters). NUREG-0700, Rev. 1 contains HFE 

guidance for the characteristics of these devices, such as resolution and viewing angle, that are important to crew 

performance. However, technology or new display devices, such as flat panel displays, is emerging and their 

characteristics are discussed separately (O'Hara, Stubler, and Higgins, 1996). Thus, like interface management, 

display devices are an important aspect of information systems, but they will not be addressed here. 

4.5 Relationship to Other Aspects of HSIs 

The relationship between the design features and functions of the information system and that of the other aspects 

of the HSI should be addressed. Modem information systems may be integrated with other HSI subsystems, such 

as plant controls. For example, controls may use on-screen representations that provide information through 

which operators can control the plant. These types of controls are referred to as soft controls. Features and 

functions of soft control and their impact on the design of information system is addressed elsewhere (Stubler, 

O'Hara, and Kramer, 2000). 

Information systems also may be integrated with plant procedure systems, especially when the procedure systems 

are computer-based. Procedure systems that are computer-based may be fully integrated with the information 

system. For example, data comparisons required for evaluating procedure-step logic may be presented using trend 

graphs provided by the information system for that and other purposes as well. In addition, the procedure system 

may share the same display devices and interface management functions. Procedure systems that are computer

based and their impact on the design of information systems also is addressed elsewhere (O'Hara, Higgins, Stubler, 

and Kramer, 2000). 

NUREG/CR-6633 4-6

I I
5 DEVELOPMENT OF TECHNICAL BASIS 

The purpose of this section is to establish a technical basis upon which to develop review guidance for information 

systems and to identify human performance issues where the technical basis is insufficient for developing 

guidance. The review begins with a discussion of the NRC's current approach for reviewing the design of 

information systems (Section 5.1). This discussion provides a baseline against which new research can be 

compared to determine whether new guidance is necessary based on recent advances in these areas. 

In Section 5.2, the generic cognitive tasks of operators are discussed to define the general scope of activities that an 

information system must support. Section 5.3 addresses the recent approaches to analyzing information 

requirements. These approaches are evaluated for their research support as well as their contribution to the NRC's 

existing guidance on design review. Information representation (Section 5.4) and organization (Section 5.5) are 

addressed next. Section 5.6 summarizes the results and conclusions. 

5.1 Existing NRC Guidance for Information Systems 

The NRC's approach to the review of information system requirements analysis, representation design, and 

evaluation will be discussed to form the basis against which to evaluate recent approaches to these topics. 

Nuclear plants, like other complex human-machine systems, are generally governed by applied general systems 

theory (Bailey, 1982; DeGreen, 1970; Gagne and Melton, 1988; Van Cott and Kinkade, 1972; Woodson, 1981). 

Systems engineering has been defined as "the management function which controls the total system development 

effort for the purpose of achieving an optimum balance of all system elements. It is a process which transforms an 

operational need into a description of system parameters and integrates those parameters to optimize the overall 

system effectiveness" (Kockler et al., 1990). Developing a complex system begins with the mission or purpose of 

the system and the requirements needed to achieve the mission's objectives. The effective integration of HFE 

considerations into the design is accomplished by providing (1) a structured top-down approach to system 

development that is iterative, integrative, interdisciplinary, and requirements-driven, and (2) a management 

structure that details the HFE considerations in each step of the overall process. 

The systems engineering approach has been used as the basis for many HFE design approaches developed by the 

NPP industry (EPRI, 1992; IEC, 1989; IEEE, 1988). This approach forms the basis of the NRC's review HFE 

methodology as well, e.g., NUREG-0711 (O'Hara, Higgins, et al., 1994) and NUREG-0700, Rev. 1. The key elements of this approach will be briefly described to understand the approach to the development of information 

systems. (While the key elements are essentially the same across the different documents, some of the specific 

details are slightly different. See Welch and O'Hara (1996) for a detailed discussion of systems engineering as it 

relates to NUREG-071 1). The systems approach addresses all aspects of system design; however, there are six that 

are important to developing information systems: operating experience review, functional requirements analysis, 

task analysis, human reliability analysis, HSI design, and verification and validation. 

Operating Experience Review 

The objective of an operating experience review (OER) is to ensure that HFE-related problems and issues 

encountered in previous designs that are similar to the current design were identified and analyzed, so that they are 

avoided in developing the current design, or in the case of positive features, to ensure that they are retained. 

An effective OER conducted on plants and systems that are similar to one under design will identify information 

requirements needed for the operators to properly perform their tasks. In particular, an OER can identify problem 

areas with current designs that led to incidents, and which can be addressed at the design stage.

5-1 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Functional Requirements Analysis and Function Allocation 

Functional requirements analysis consists of identifying functions that must be undertaken to satisfy plant 

productivity and safety objectives, e.g., to prevent or mitigate the consequences of postulated accidents that could 

cause undue risk to the health and safety of the public. Function allocation is the analysis of the requirements for 

plant control and the assignment of control functions to personnel, system elements, and combinations of personnel 

and system elements. 

The term function may be used in different ways by plant designers. It can refer to very high-level plant functions, 

such as critical safety functions, or the functioning of an individual piece of equipment, such as a valve or a wall

panel information display system. The former is meant here. Therefore, a function is an action that is required to 

achieve a desired goal. A high-level objective, such as preventing the release of radioactive material to the 

environment, is one that designers strive to achieve through implementing other more specific, high-level 

functions. Designers will strive to achieve these high-level functions through the design of the plant, and plant 

operators will strive to achieve them through the proper operation of the plant. Examples of these functions are 

reactivity control, containment integrity and control of RCS water mass inventory. 

The high-level function may be described without reference to specific plant systems or to the level of human or 

machine intervention that is required to satisfy the function. High-level functions are usually accomplished 

through some combination of lower-level system actuations, such as reactor trip, safety injection, or accumulators. 

Manipulating lower-level systems to satisfy a higher-level function is defined here as a control function. Often 

plant systems are used in combination to achieve a higher-level function. 

The requirements of these functions are analyzed to (1) determine the objectives, performance requirements, and 

constraints of the design; (2) define the high-level functions that must be accomplished to meet the objectives and 

required performance; (3) define the relationships between high-level functions and plant systems (e.g., plant 

configurations or success paths) responsible for performing the function; (4) provide a framework for 

understanding the role of controllers (whether personnel or system) for controlling the plant. Some of the relevant 

NUREG-0711 considerations for function analysis and decomposition include the following: 

The functional requirements and allocation analysis should be kept current throughout the development of the 

design and held until decommissioning so that it can be used as a design basis when modifications are 

considered. Control functions should be reallocated iteratively, in response to developing design specifics, 

operating experience, and the outcomes of ongoing analyses and trade-off studies. 

The functions and systems should be described. For each safety function, the set of plant system 

configurations or success paths that are responsible for or capable of carrying out the function should be 

clearly defined. Functions should be diagramed at several levels, starting at "top-level" functions, where a 

very general picture of major functions is described, and continuing to lower levels until a specific critical end

item requirement emerges (e.g., a piece of equipment, software, or an operator). The functional decomposition 

should address the following levels: (1) high-level functions (e.g., maintain reactor coolant system [RCS] 

integrity) and critical safety functions (e.g., maintain pressure control of the RCS) and (2) specific plant 

systems and components. 

NUREG/CR-6633 5-2

I I
5 DEVELOPMENT OF TECHNICAL BASIS

• Each high-level function should be described; the descriptions should include: 

- Purpose of the function 

- Conditions that indicate that the function is required 

- Parameters that indicate that the function is available 

- Parameters that indicate that the function is operating (e.g., flow indication) 

- Parameters that indicate that the function is achieving its purpose (e.g., reactor vessel level returning to 

normal) 

- Parameters that indicate that operation of the function can or should be terminated 

Some of the functions are allocated completely or in part to plant personnel. The functions undergo additional 

analyses to determine human task requirements. 

Task Analysis 

Task analysis is the evaluation of the performance demands on plant personnel to identify the task requirements for 

accomplishing the functions allocated to them (Drury et al., 1987). It is a very important activity because it defines 

the HSI requirements for accomplishing tasks, including information requirements. Although there is no precise definition of a task with respect to the level of abstraction, a task is a group of related activities that have a 

common objective or goal. 

Again, as in functional requirements analysis, decomposition into ever increasing levels of detail is necessary. .Task analyses should begin on a gross level and involve developing detailed descriptions of what personnel must 

do. The nature of the input, process, and output required by and of personnel should be defined. For information 

requirements analysis, the detailed description of tasks should include the following: 

• Gathering Information 

- Information required (e.g., parameters, units, precision, accuracy) 

- Information source (e.g., alarm, displays, verbal communication) 

Requirements for Decision Making 

- Description of the decisions to be made (relative, absolute, probabilistic) 

- Evaluations to be conducted 

- Decisions that are probable based on the evaluation (opportunities for cognitive errors, such as capture 

error, will be identified and carefully analyzed)

5-3 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

* Requirements for Response 

- Action to be taken 

- Overlap of task requirements (serial vs. parallel task elements) 

- Frequency 

- Time available for operator response based on characteristics of plant response 

- Temporal constraints (task ordering) 

- Tolerance and accuracy 

- Operational limits of personnel performance 

- Operational limits of machine and software 

- Body movements required by action taken 

* Requirements for Feedback 

- Feedback required to indicate adequacy of actions taken 

The task analysis should be iterative and become progressively more detailed over the design cycle. Sufficient 

detail is necessary to identify information and control requirements to enable specification of detailed requirements 

for HSI resources, such as alarms, displays, data processing, and controls. 

Based on its use of functional decomposition and analysis of cognitive requirements (information gathering, 

decision making, response, and feedback), NUREG-071 I is consistent with recent approaches to cognitive task 

analysis (e.g., Roth and Mumaw, 1995). 

Human Reliability Analysis 

The objective of the human reliability analysis (HRA) of the HFE design is to ensure that potential effects of 

human error on plant safety and reliability were analyzed, that human actions important to plant risk were 

identified, and that human error mechanisms were addressed in the design of the HSIs, to minimize the likelihood 

of personnel error and to provide for error detection and recovery capability. 

The HRA element of NUREG-07 11 that is especially pertinent to developing information requirements is the area 

of risk-significant human actions. The HRA will develop those operator actions that are especially significant to 

risk. These actions are often associated with low-probability and high-consequence sequences of events. These 

actions may relate to activities that only are needed once certain failures and transients have already occurred. As 

a result, they would not typically be a part of task analyses that are done during the usual design process. Thus, 

consideration of these risk-significant human actions will often add more controls, displays, or alarms that would 

not have otherwise been included. 

NUREG/CR-6633 5-4

I I
5 DEVELOPMENT OF TECHNICAL BASIS

HSI Design 

The HSI design process, as the term is used here, is the translation of function and task requirements into a 

detailed HSI design; that is, the specific design of the HSI resources identified above as well as their integration in 

CRs and other workplaces. This translation should be designed using a structured methodology that guides 

designers in the identification of what information and controls are required, the identification and selection of 

candidate HSI approaches, and the detailed design of the HSIs. Constraints imposed by the overall instrumentation 

and control (I&C) system should be considered throughout the design process. 

The HSI design methodology addresses specific design activities, including: 

• Development and use of guidelines for HFE design 

* Functional requirement specification 

* HSI concept design 

* Detailed design and integration of the HSI 

* Tests and evaluations of the HSI design process 

* Documentation of final design 

At this point the information system requirements have been defined and the representational aspect of the 

information system begins to be developed. Display formats are developed to meet the functional requirements 

with a design that reflects a combination of past operational experience (what types of formats were successful or 

unsuccessful), HFE guidelines, and prototype testing and evaluation. 

Section 1, Information Display of Part 2 of NUREG-0700, Rev. I provides guidance for information system design review. It deals primarily with the review of visual displays, both text and graphics based. Guidance is provided 

in top-down fashion beginning with general display guidelines (Section 1. 1), and then finer levels of display 

details. Section 1.2, Display Formats, addresses common display forms: 

* Continuous Text Displays 

* Tables and Lists 

* Data Forms and Fields 

* Bar Charts and Histograms 

* Graphs 

* Pie Charts 

* Flowcharts 

* Mimics and Diagrams

5-5 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

* Maps 

• Graphic Instrument Panels 

• Speech Displays 

In addition to the display guidance in NUREG-0700, guidance for group-view displays, such as is in advanced CR 

wall panel displays, is available (Stubler and O'Hara, 1996). 

A display format can be decomposed into a set of basic elements, such as alphanumeric characters, icons, symbols, 

color, highlighting, abbreviations, labels, coding, and presentation of numeric data. These are addressed in 

Section 1.3, Display Elements. Section 1.4 contains guidelines for reviewing the quality of data and display update 

rate. Section 1.5, Display Devices, addresses the hardware aspects of displays. Review guidelines are given for 

display devices, such as video display units, large panel displays, and hard-copy display devices (printers and 

plotters). 

There are two significant shortcomings in the treatment of information representation: display formats do not 

correspond to these generic formats, and the organization of displays in a display network. For novel formats, i.e., 

formats not addressed in NUREG-0700, it was recommended that they be evaluated using the general display 

guidelines and the guidelines in Section 1.3, Display Elements. While this provides some support for design 

review, a review limited to these guidelines would fail to address important aspects of many novel designs, such as 

the relationship between their dynamic graphic elements and the underlying process dynamics that they are 

intended to represent. The organization of displays in an information network was not addressed at all. 

HSI designs should be tested and evaluated throughout their development. This is especially important for designs 

that are not well defined by HFE guidance. NUREG-071 1 addresses the methodology used for testing. 

Verification and Validation (V&V) 

V&V of HSI designs should be conducted to ensure the acceptability of the final design and to address any 

remaining issues. Because no one method is likely to be sufficiently comprehensive, a series of analyses may be 

done. 

HSI Task Support Verification. This evaluation verifies that the HSI supports all identified personnel task 

requirements. This is determined by comparing the HSI design elements to the criteria defined by function and 

task analysis requirements. Problems are identified for (1) personnel task requirements that are not fully supported 

by the HSI, and (2) the presence of HSI components, which may not be needed to support personnel tasks. 

HFE Design Verification. This evaluation verifies that the HSI is designed and implemented to account for 

capabilities and limitations of humans. This is determined by comparing the detailed HSI design to the criteria 

defined by HFE guidelines. Problems are identified if the design is inconsistent with HFE guidelines. 

Integrated System Validation. This evaluation validates that the integrated HSI design enables tasks required for 

safe operation to be performed without imposing excessive workload. The performance criteria are derived from 

task, risk, and engineering analyses. Problems are identified if performance criteria are not achieved, or if the HSI 

imposes a high workload on plant personnel. 

NUREG/CR-6633 5-6

I I
5 DEVELOPMENT OF TECHNICAL BASIS

5.2 Cognitive Tasks Supported by Information Systems 

The plant information systems provide the basis for operators to perform their role in the plant. In this section, we 

consider the general tasks that the information system must support. This is important in scoping out the range of 

cognitive activities that must be addressed in the design of information systems. This is not only important when 

considering requirements for design, but also for evaluating the results of research on displays for generalizability 

to the full range of NPP operations. 

The operator's role in a NPP is that of a supervisory controller, i.e., the plant's performance is the result of the 

interaction of human and automatic control. Reason (1990) called this a complex multiple-dynamic configuration; 

in such settings, situation assessment and response planning can be difficult when things go wrong. In addition to 

failures of plant process, the automatic control systems and HSI also can fail. Thus, personnel must respond to 

failures in the plant and to the interfaces that communicate failures. One of the most significant aspects of the HSI 

in responding to process failures is the plant information system. 

The operator's impact on plant functions, processes, systems, and components is mediated by a causal chain from 

the operator's physiological and cognitive processes, to operator task performance, and ultimately to plant 

performance through the operator's manipulation of the plant's HSI. Design of HSI, including displays, impacts plant's performance through personnel tasks in support of plant operations. The operator's role involves two types 

of tasks, primary tasks and secondary tasks. Primary tasks are those the operator performs as part of the functional 

role of supervising the plant. Operators may be required to take an action in support of the plant's performance of 

a higher-level function. Even when operators are not required to take an explicit action, they must monitor the 

performance of automatic systems and intervene when they fail or function unacceptably. 

Primary tasks involve several generic cognitive tasks, i.e., situation assessment, monitoring and detection, response 

planning, and response implementation (Figure 5.1). When addressing primary tasks, these generic cognitive 

tasks are discussed rather than the detailed specific tasks, such as monitoring steam flow, starting pumps, and 

aligning valves. 

Secondary tasks are those the operator must perform when interfacing with the HSIs or job performance aids, but 

which are not directed to the primary task (O'Hara, Stubler, and Nasta, 1997). Secondary tasks include navigating 

through an information system and manipulating windows on a VDU. To adequately perform their tasks, both 

primary and secondary, operators use their information processing resources, such as attention, reasoning, and 

memory. 

The design of the information system affects each of the operator's generic cognitive primary tasks and the 

secondary tasks. In this section, we address situation assessment, monitoring, response planning, and response 

implementation. 

5.2.1 Situation Assessment 

When faced with an abnormal occurrence, operators actively try to construct a coherent, logical explanation to 

account for their observations. This cognitive activity, situation assessment, involves two related concepts: the 

situation model and the mental model. Operators develop and update a mental representation of the factors known, 

or hypothesized, to be affecting the plant's state at a given point in time. The mental representation resulting from 

situation assessment may be referred to as a situation model, the person's understanding of the specific current 

situation. The situation model is constantly updated as new information is received.

5-7 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Figure 5.1 Generic Primary Tasks of a Supervisory Controller 

To construct a situation model, operators use their general knowledge and understanding about the plant and how 

it operates to interpret the information they observe and understand its implications. Limitations in knowledge 

may result in incomplete or inaccurate situation models. The general knowledge governing the performance of 

highly experienced individuals may be referred to as a mental model, which constitutes the operator's internal 

representation of the physical and functional characteristics of the system and its operation. Mental models may 

not always be fully accurate or complete (Woods et al., 1994). 

The mental model is built up through formal education, system-specific training, and operational experience. It is 

represented in the knowledge bases of long-term memory. An accurate mental model is generally considered a 

defining characteristic of expert performance (e.g., Wickens, 1984; Bainbridge, 1986; Moray et al., 1986; 

Rasmussen, 1983; Sheridan, 1976) and is extremely important to many aspects of information processing 

(discussed below). The mental model is thought to drive skill-based processing, control rule-based activity through 

the mediation of the operator's conscious effort in working memory, and provide the substantive capability to 

reason about and predict future plant states, which is required of knowledge-based processing (Rasmussen, 1983). 

The distinctions between the mental and situation models reflect their cognitive underpinnings in long-term and 

working memory. The mental model is relatively permanent. By contrast, an operator's situation model is the 

current interpretation of the plant's status, and therefore, can be changed. 

NUREG/CR-6633 5-8

I I
5 DEVELOPMENT OF TECHNICAL BASIS

When the operator's situation model is an accurate reflection of the plant's actual state, an operator is said to have 

good situation awareness. Thus, the accuracy of situation awareness is a function of the degree of correlation 

between the operator's situation model and the actual plant conditions at any given time. An operator can have a 

good mental model (e.g., knowledge of how the plant functions), but poor situation awareness because the situation 

model does not match the current plant conditions. The process of situation assessment has been identified as the 

single most important factor in improving effectiveness of the crew in complex systems (Endsley, 1988). 

For an experienced, well-trained operator, when the HSI can provide information that readily maps to knowledge in the operator's mental model, an accurate situation model is easily developed. To the extent that an easy match 

cannot be made between plant information and the situations defined in the mental model, situation assessment 

requires more working memory and attention, and cognitive workload will be high (Endsley, 1993, 1995; Fraker, 

1988). However, in addition to supporting situation assessment, working memory also must support other 

activities, such as the selection and implementation of operator actions. Accordingly, if other tasks place high 

demands on working memory, situation awareness may suffer. On the other hand, the demands placed on working 

memory in maintaining situation awareness can be lessened if the situation is familiar (based on training or 

experience), and therefore, can be identified with a representation stored in long-term memory. In this case, it 

would not be necessary for the operator to maintain in working memory each detail of the situation. 

Situation awareness and cognitive workload may vary inversely under complex, somewhat ambiguous situations. 

For example, under unfamiliar or otherwise difficult conditions, high cognitive workload may be associated with 

decreased situation awareness This may be due to a lack of available attentional resources for analyzing the 

situation. However, as Endsley (1993) points out, situation awareness and cognitive workload, while interrelated, 

may vary independently. For example, a task may be intensive, but readily recognizable. Situation awareness 

requires the expenditure of cognitive resources that contribute to workload, but it is not the only cognitive activity 

requiring such resources. 

Thus, mental models enable operators to engage in situation assessment and to establish situation models. Another 

important aspect of mental models is that they enable operators to make predictions and form expectations that, in 

turn, guide monitoring and affect the interpretation of information. This is a general characteristic of human 

information processing; it is a synthesis of "bottom-up" processing (what an operator perceives from the 

environment) and "top-down" processing (what an operator expects) (Neisser, 1967). An example of bottom-up 

processing occurs during a disturbance when an operator monitors the HSI and processes data from the interface to 

determine what is wrong. At the same time, these data are used to formulate hypotheses or expectations about the 

status of the plant, which serve to structure the perceptual process and data gathering occurring at lower levels. 

This is top-down processing. Both contribute to the operator's interpretation of the situation. 

The ability to make predictions using a mental model that is based on the current situation model enables the 

operator's performance to become more "open-loop" (Moray, 1986). "Open-loop" in this context means that 

behavior becomes less driven by feedback and more governed by the operator's prediction of future system behavior 

and the desired goal state. A NPP mental model includes such knowledge as the physical interconnections among 

plant systems to predict flow paths (e.g., considering piping and valve interconnections to figure out how water 

from one system could get into another) and knowledge of mass and energy changes in one system to predict the 

effect on a second system (e.g., predicting the effect that changes in secondary side steam generator levels and 

temperatures will have on primary system cooldown). While mental models provide the principles upon which 

predictions can be made, the situation model provides the starting point and becomes the basis from which 

expectations are developed about events that should be happening at the same time, how events should evolve over 

time, and effects that may occur in the future.

5-9 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

The operator's expectations of the near-term future state of the plant are used to guide the sampling of indicators to 

confirm the inference (Bainbridge, 1974). Expectations are used to search for evidence to confirm the current 

situation model and to explain observed symptoms. If a new symptom is observed that is consistent with operator 

expectations, a ready explanation for the finding will be developed, yielding greater confidence in the situation 

model. 

While the mental model allows prediction and expectancy to guide control responses, expectancy also can make 

detecting subtle system failures difficult (Wickens and Kessel, 1981). When a new symptom is inconsistent with 

an operator's expectation, the operator may discount or misinterpret it to make it consistent with the expectations 

derived from the current situation model. For example, an operator may fail to detect key signals, or detect them 

but misinterpret or discount them, because of an inappropriate understanding of the situation and the expectations 

derived from that understanding. That is, operators tend to ignore or discount symptoms that are not consistent 

with their situation model. However, if the new symptom is recognized as an unexpected plant behavior, the need 

to revise the situation model will become apparent. In that case, the symptom may trigger situation assessment 

activity to search for a better explanation of the current observations. In turn, situation assessment may involve 

developing a hypothesis for what might be occurring, and then searching for confirmatory evidence in the 

environment. Thus, situation assessment activities can result in detecting abnormal plant behavior that might not 

otherwise have been observed, detecting plant symptoms and alarms that may have otherwise been missed, and 

identifying problems, such as sensor failures or plant malfunctions. 

The situation model is constantly updated as new information is received and a person's understanding of a 

situation changes. In NPP applications, maintaining and updating a situation model entails keeping track of the 

changing factors that influence plant processes, including faults, operator actions, and automatic system responses. 

The importance of mental and situation models, and the expectations that are based on them, cannot be 

overemphasized. They not only govern situation assessment, but also play an important role in guiding 

monitoring, using procedures and formulating response plans, and implementing responses. 

On the basis of the above discussion, several factors can be identified that make the process of situation assessment 

difficult: unanticipated events, a poor mental model, a poorly designed information system, and divided attention. 

Unanticipated events - Mental models are established through training and experience. Situation assessment 

involves matching current information to those models. Therefore, situations or events that were not analyzed 

by designers, for which operators were not trained and have not experienced, will be difficult to assess. 

Vicente and Rasmussen (1992) suggested that a significant design challenge is to develop a HSI to support 

operators in handling unanticipated events. Since responses to NPP abnormal and emergency events are 

analyzed carefully in advance to design systems to handle the disturbance (e.g., engineered safety features) and 

operator responses (e.g., emergency operating procedures [EOPs]), the unplanned and unanticipated events 

pose the greatest challenge to the human-machine system. As Vicente and Rasmussen (1992) noted, "The 

inescapable conclusion is that unanticipated events can and do happen in large-scale industrial systems. In 

fact, they are the major cause of life-threatening accidents" (p. 589). 

* Poor mental model- Even when events are anticipated, failures in training or inadequate experience can 

result in mental models that are not adequate to support situation assessment. 

* Poor information system design - Even when events occur that are familiar and well understood by operators, 

they must recognize them. As noted above, this is accomplished by a match between information provided by 

the HSI and the operator's mental model. Information systems may not support such a mapping very 

NUREGICR-6633 5 - 10

I I
5 DEVELOPMENT OF TECHNICAL BASIS

effectively. For example, relevant information may be lost in an information-dense display or spread out 

across many displays. Even when all the relevant information is available, it may be presented in a way, such 

as individual parameters, that operators must mentally process to get it to a level where it is able to be mapped 

to the mental model. 

Divided attention - Maintenance of situation awareness is disrupted by shifting attention between different 

tasks (Gaba and Howard, 1995). Attention is directed away from situation assessment to engage in other 

primary tasks, such as communications with coworkers, or secondary tasks, such as retrieving a display. 

5.2.2 Monitoring and Detection 

Monitoring and detection refer to the activities involved in extracting information from the environment. 

Monitoring is checking the state of the plant to determine whether the systems are operating correctly, including 

checking parameters indicated on the CR panels, monitoring parameters displayed by the process computer, 

obtaining verbal reports from operators in the plant areas, and sending operators to areas of the plant to check on 

equipment. Detection is the operator's recognition that something is not operating correctly and that an 

abnormality exists. 

In a highly automated plant, much of what supervisory controllers do involves monitoring. For example, operators 

must monitor 

* Normal conditions to determine that what is expected to happen does 

* Results of actions (feedback) 

* Normal and safety indications for changes or disturbances 

* Performance of automated systems 

* Problematic equipment 

* Activities of coworkers (test and maintenance) 

Ironically, despite the fact that monitoring is such a central cognitive activity, the monitoring by supervisory 

controllers of computer-based systems is often "unstructured" (Thurman, 1997). Operators are advised to monitor 

everything and given little guidance on how to accomplish this. Thus, monitoring lacks goal direction and 

important information can be missed. Further, display designs often fail to support monitoring because designers 

simply take all available status information and develop a set of displays structured around a physical 

representation of the system (Thurman, 1997). The cognitive basis for monitoring is discussed below. 

Monitoring and detection are influenced by two factors: the characteristics of the environment and the operator's 

knowledge and expectations. These factors lead to two types of monitoring: data-driven and model-driven. 

Monitoring that is driven by characteristics of the environment is often referred to as data-driven monitoring. 

Data-driven monitoring is affected by the salience of the information's presentation (e.g., size, color, and 

loudness). For example, alarm systems are basically automated monitors that are designed to influence data-driven 

monitoring by using aspects of physical salience to direct attention. Auditory alerts, flashing, and color coding are 

examples of physical characteristics that enable operators to quickly identify an important new alarm. Data-driven 

monitoring also is influenced by the behavior of the information being monitored, such as the bandwidth and rate

5- 11 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

of change of the information signal. For example, observers more frequently monitor a signal that is rapidly 

changing. 

Model-driven monitoring is initiated by operators based on their knowledge and expectations about the most 

important sources of information. This type of monitoring also is referred to as knowledge-driven monitoring. 

Knowledge-driven monitoring can be viewed as active monitoring in that the operator is not merely responding to 

characteristics of the environment that "shout out" as an alarm system does, but is deliberately directing attention 

to areas of the environment that are expected to provide specific information. 

Model-driven monitoring may be initiated by several factors. First, it may be guided by operating procedures or 

standard practice (e.g., control panel walk-downs that accompany shift turnovers). Second, it can be triggered by 

situation assessment or response planning activities, and therefore is strongly influenced by a person's current 

situation model. The situation model allows the operator to direct attention and focus monitoring effectively. 

However, such a monitoring strategy also can lead operators to miss important information. For example, an 

incorrect situation model may lead an operator to focus attention in the wrong place, to fail to observe a critical 

finding, or to misinterpret or discount an indication. 

An operator is faced with an environment containing more variables than can be realistically monitored. 

Monitoring is challenging because there are a large number of potentially relevant things to attend to at any point 

in time and that the operator must determine what information is worth pursuing within a constantly changing 

environment (Vicente et al., 1997). In this situation, monitoring requires the operator to decide what to monitor 

and when to shift attention elsewhere. These decisions are strongly influenced by an operator's current situation 

model, which guides the allocation of attentional resources to sampling data from the environment based on its 

statistical properties; i.e., expected probability and correlation. The operator's ability to develop and effectively use 

knowledge to guide monitoring relies on an understanding of the current state of the process. As cognitive 

workload increases, monitoring strategies become less thorough and the capability to detect particular failures 

decreases (Ephrath and Young, 1981). 

As discussed above, under normal conditions, situation assessment is accomplished by mapping the information 

obtained in monitoring to elements in the situation model. For experienced operators, this comparison is relatively 

effortless and requires little attention. During unfamiliar conditions, however, the process is considerably more 

complex. The first step in realizing that the current plant conditions are not consistent with the situation model is 

to detect a discrepancy between information representing the current situation and information derived from 

monitoring. This process is facilitated by the alarm system, which helps to direct the attention of a plant operator 

to an off-normal situation. 

When determining whether or not a signal is significant and worth further investigation, operators examine the 

signal in the context of their current situation model. They must judge whether the anomaly indicates a real 

abnormality or an instrumentation failure. They will then assess the likely cause of the abnormality and evaluate 

the importance of the signal in determining their next action. 

Monitoring and detection were described in terms of signal detection theory (SDT) (Green and Swets, 1988). 

Process control operators are in a monitoring environment that was described in SDT terms as an alerted-monitor 

system (Sorkin et al., 1985; Sorkin and Woods, 1988). Such a system is composed of an automated monitor and a 

human monitor, an operator. The automated monitor is the alarm system which monitors the system to detect 

off-normal conditions. When a plant parameter exceeds the criterion of the automated monitor, the operator is 

alerted and must then detect, analyze, and interpret the signal as a false alarm, or a true indication of a plant upset. 

The operator also can assess plant parameters independently of the automated monitor (the alarm system). Both 

NUREG/CR-6633 5 - 12

I I
5 DEVELOPMENT OF TECHNICAL BASIS

the operator and alarm system have their own specific, signal detection parameter values for sensitivity (d') and 

response criterion. The response criterion refers to the amount of evidence that is needed before an operator will 

conclude that a signaled event is actually present; this is sometimes referred to as response bias since it describes 

an operator's degree of conservatism. Sensitivity refers to the resolution of the system and determines the ease 

with which signals (represented as a statistical distribution) can be distinguished from signals and noise (also 

represented as a distribution). 

SDT research has many implications for understanding how operators process information during a disturbance. 

First, the response criterion is affected by expectancy, i.e., the expected probability that an event will occur and the 

payoff structure (rewards and penalties for making correct and incorrect detections, respectively). While alarms 

can occur frequently, significant off-normal events in NPPs typically have a low probability of occurring. Therefore, operators have low expectancy about their actual occurrence, which creates a conflict between the cost 

to productivity for falsely taking an action that shuts down the reactor versus the cost for failing to take a warranted 

action. In a real-world system, disturbances have a low probability, and operators therefore rely on redundant and 

supplemental information to confirm the alarmed condition. Upon verification of several confirmatory indicators, 

the operator can accept the alarm information as indicating an actual off-normal condition (compared with a 

spurious condition). 

There are two types of anomalies: (1) deviations from desired system function, referred to as abnormal findings, and (2) deviations from the operator's situation model, referred to as unexpected findings. The different kinds of 

anomalies lead to different follow-up reasoning and monitoring behavior: 

* Abnormal findings lead to information processing about how to cope with the disturbance (response planning) and to monitoring behavior to see if responses to coping occurred as expected, and whether they are having the 

desired effect. 

* Unexpected findings or process behavior lead to situation assessment activity and knowledge-driven 

monitoring to explain the finding. 

Failures in monitoring can include failing to observe parameters, misunderstanding their significance, or failing to 

obtain necessary reports from plant areas. Failures in detection can include failing to recognize an abnormality 

despite appropriate monitoring. An error in monitoring or detection can lead to the operator's failure to respond to 

the event, or at least, failure to respond within the required time. 

Two aspects of plant design are significant to monitoring: HSI technology and plant sophistication. HSI 

technology has a significant impact on monitoring. In conventional CRs, displays and controls are predominantly 

spatially-dedicated. They have fixed locations that cannot be changed in form or function. In such situations, 

operators develop scanning patterns to support monitoring. However, even with spatial dedication, monitoring and 

search can be difficult because information can be physically hidden from view (Barber, 1996), e.g., when displays 

are obscured by tags (as happened during the TMI event) or located away from the main CR. One of the major values of the detailed design reviews of CRs in the 1980s was making the search process easier (Van Cott, 1997). 

Improvements that facilitated interface management and reduced the chance for error were evident in the 

modifications based on NUREG-0700. These included better organization of indicators and controls, improved labeling, and improved demarcation of the relationships between indicators and controls through the use of board 

mimics. However, there was little actual manipulation of the interface - what Vicente, Mumaw, and Roth (1997) referred to as "HSI degrees of freedom." The CR information and controls were fixed and spatially dedicated in 

the way the HSI designer felt was most appropriate.

5- 13 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

By contrast to conventional HSIs, advanced HSIs are flexible because they can be configured and function in 

various operating modes. These interfaces have several degrees of freedom. For example, based on observation of 

operators monitoring a relatively advanced NPP, Vicente et al. (1997) found that operators could decide "...where 

to put a given display, what the time scale on a trend graph should be, what the range on a trend graph should be, 

what form to present a variable in (e.g., trend, bar, bar-trend, digital), what variables should be graphed together, 

and so on. Not only can the operator make these decisions, but they can make them over and over again in the 

sense that these presentation parameters can be readily changed in a moment." However, operators tended to 

ignore the flexibility offered by the system. Instead, they selected a set of displays that they felt gave them a good 

overview of the plant, and left those on the VDUs, which was done, to some extent, at the expense being able to 

access more detailed views of the plant that may have been available. This was attributed, in part, to the effort 

involved in paging through displays, a task that operators felt interfered with their ability to understand the overall 

status of the plant. 

With regard to monitoring, Vicente et al. (1997) commented: 

Computer-based HSIs can make it difficult to find information and act on the system in comparison with conventional 

plants...In a computer-based control room, there is a significant increase in the amount of knowledge required to 

operate the interface. Part of these demands arise from the fact that information is presented serially. Thus, 

operators need to know how to bring up the information they want to display on CRTs. The other part of the increase 

in knowledge demands arises from the fact that the computer-based displays are much more flexible than the hard

wired instruments. The same information can be displayed at different time scales, at different ranges, in different 

locations, in different forms, and in different groupings. Consequently, the operator has to have much more 

knowledge about the interface (not the unit) to resolve the degrees of freedom offered by the flexible design of the 

system...On the one hand, one might argue that the flexibility provided by the computer-based medium should result 

in a performance improvement because it allows operators to view information in a form that is tailored to different 

types of contexts. This should reduce the need for the work-arounds that operators engage in traditional control 

rooms to facilitate monitoring. On the other hand, one could just as well argue that this flexibility comes at the price of an increase in the time and effort that operators spend manipulating the interface rather than monitoring the unit. 

The problems associated with information retrieval and control manipulation will be compounded by the fact that a 

small set of control actions, such as positioning a cursor, will be required for all CR activities. Vicente et al. 

(1997) make a similar point for monitoring. Whereas in a conventional CR, monitoring was done primarily with 

the eyes, in a computer-based CR, it is done with the hands, which means that considerable manipulation of the 

computer-input device is needed to page through displays. 

Monitoring is also affected by the technological sophistication of the plant design. In fact, the classical view of 

monitoring is that it is a boring and tedious task that leads to vigilance difficulties associated with lack of cognitive 

stimulation. While this may be true in some systems, recent research in nuclear plants has shown that routine 

monitoring can be difficult (Mumaw et al., 1996; Vicente et al., 1997). Some factors contributing to the difficulty 

were identified by Mumaw et al. (1996) following a study of monitoring performance in two NPPs. The factors 

included the following: 

* System complexity - There are thousands of components which interact in different situations, thus it can be 

difficult to understand the implications of each. 

* System reliability - Even though the number of components is large and they are highly reliable, at any one 

moment there are many that are out of service or not working properly. Operators must factor these 

considerations into their situation assessment. 

NUREG/CR-6633 5- 14

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Display design - Given the large number of parameter displays, it is often difficult to detect when an abnormal 

situation has occurred. There are several reasons for this. First, interpretation of displayed parameter values 

is memory intensive. Displays do not give clear referent values and few aids are provided to support recall of 

recent value. In addition, few emergent features are presented to enable operators to rapidly identify higher

level information from the displays provided. Complicating the use of the displays are the same reliability 

issues as for other components, i.e., at any time, there may be displays that do not work properly. Failed 

displays also can be difficult to detect. 

Automation design - The operation of and feedback from automated systems is not well represented in the 

display system. For example, the actual status of components being automatically controlled is not displayed. 

(Murphy and Mitchel[ 1986] identified a number of effects on operator's cognition of automation and their 

implications for display design). 

These factors introduce uncertainties that make monitoring difficult. In fact, Mumaw et al. concluded that 

monitoring during normal NPP operations are "better cast as a problem-solving activity than a vigilance task" 

(1996, p. 30). It is a complex situation assessment activity dependent on imperfect, sometimes nonfunctional, 

displays depicting thousands of parameters. 

Vicente, Mumaw, and Roth (1997) examined the generalizability of these factors to a more advanced CR. While 

they found the overall reliability and trustworthiness of the instrumentation to be improved, the same basic 

problems existed. In addition, they noted a new problem associated with a CRT-based information system, the 

keyhole effect, which adds difficulty in monitoring. 

5.2.3 Response Planning 

Response planning refers to deciding upon a course of action to address an event. Response planning can be as 

simple as selecting an alarm response procedure or Emergency Operating Procedure (EOP), or it may involve more 

thoroughly developing a plan in circumstances where existing procedures have proved incomplete or ineffective. 

In general, response planning involves the operators using their situation model of the current plant state to 

identify goals and the transformations required to achieve them. The goal may be varied, such as identifying the 

proper procedure, assessing the status of back-up systems, or diagnosing a problem (Rasmussen, 1981). To achieve 

the goals, operators generate alternative response plans, evaluate them, and select the most appropriate one that is 

relevant to the current situation model. 

While this is the basic sequence of cognitive activities associated with response planning, one or more of these 

steps may be skipped or modified based on the operator's assessment of a particular situation. When procedures 

are available and judged appropriate to the current situation, the need to generate a response plan in real-time may 

be eliminated. However, even when written procedures are available, some aspects of response planning will be 

done. For example, operators still need to (1) identify appropriate goals based on their own situation assessment, 

(2) select the appropriate procedure, (3) evaluate whether the procedure-defined actions are sufficient to achieve 

those goals, and (4) adapt the procedure to the situation, if necessary. 

The decision making involved in situation assessment and response planning, especially in ambiguous situations, 

such as when available procedures do not suffice, can be a large cognitive burden and draw heavily upon working 

memory, long-term memory, and attentional resources. In such situations, information is consciously manipulated in working memory, and the ability to do so is a direct function of the attentional resources available. Working 

memory has very limited capacity, and without sustained attentional resources (or transfer of the information to

5-15 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

long-term memory), information decays rapidly. Information can be lost due to (1) insufficient attentional 

resources to keep it active, (2) overload of the working memory capacity, and (3) interference from other 

information in working memory. To increase the capacity of working memory, operators use memory heuristics, 

such as chunking, that enable them to organize various bits of information into higher-level, meaningful units. A 

heuristic, as used in this report, is a shortcut for information processing developed through experience and trial

and-error rather than systematic, formal analysis. Once this is accomplished, the higher-level units, not the 

individual elements, are stored in working memory. 

Roth, Mumaw, and Lewis (1994) demonstrated the need for operators to maintain a supervisory role even when 

responses are largely dictated by EOPs. They investigated how operators handle cognitively demanding 

emergencies. Their objective was to examine the role of situation assessment and response planning on guiding 

crew's performance in situations where EOPs were being used. NPP operators from two different utilities 

performed interfacing system loss of coolant accident (ISLOCA) and loss of heat sink scenarios on training 

simulators, where complexities made it difficult to simply follow the appropriate procedure. The results showed 

the importance of high-level cognitive functions during the use of EOP. The operators developed an understanding 

of the plant state and confirmed their situation assessment. They also attempted to understand plant performance 

that was not expected based on their current situation model. These cognitive activities enabled them to evaluate 

the appropriateness of the EOP for the high-level goal dictated by the situation assessment. Roth et al. (1994) 

noted the importance of the crew's interaction and communication to these high-level cognitive functions. This 

was partly because of the need to obtain information from many HSIs in different locations. In addition, 

communication helped operators overcome the fact that EOPs do not address all the important information about 

the current state of the plant. Roth et al. noted that these cognitive activities made it possible for crews to evaluate 

the ability of a procedure to achieve its high-level goal in the context of the current plant condition. When a 

specific procedure failed to meet the high-level goal, operators would alter its path to better address the situation. 

Thus, Roth et al. (1994) demonstrated the importance of understanding the basis of the procedure and the higher

level goals it is intended to achieve. The need to formulate modifications to procedure pathways also means that 

operators may not simply proceed linearly through a procedure. They may need to consider future steps, reexamine 

previous steps and consult other procedures to verify that their current activities are correct and will meet the high

level goals of the procedure. 

Roth (1994) considered the implication of the Roth et al. (1994) study for the design of operator support systems. 

First, the requirement of situation assessment and response planning independent from procedure activity suggests 

that operators must maintain awareness of abnormal plant symptoms, determine what malfunctions could produce those symptoms, and know the manual and system actions that are being undertaken and their effects. Second, 

since crews have to anticipate the consequences of their actions, operator support systems could support identifying 

consequences and side effects of planned actions. Third, operators must understand the assumptions and logic 

behind the procedures, i.e., the intent of the procedures, their overall strategies, and the transition logic between 

them. Since operators may not move linearly between procedures, the means for navigating the information 

system will be important to the usefulness of such information systems in complex emergencies. 

5.2.4 Response Implementation 

Response implementation is the actual performance of the actions identified in response planning. This can be as 

simple as a single operator selecting and operating a control, or it can involve communications and coordination 

with teams of operators in different locations of the plant, who each then select and operate appropriate equipment 

controls in a centrally coordinated manner. The actions may be discrete (e.g., flipping a switch) or they may 

involve continuous control (e.g., controlling steam generator level). 

NUREG/CR-6633 5-16

I I
5 DEVELOPMENT OF TECHNICAL BASIS

The results of actions are monitored through feedback loops. Two aspects of NPPs can make response implementation difficult: response time and indirect observation. Time and-teedback delays disrupt response implementation, because they make it difficult to determine whether control actions are having their intended 

effect. In such a situation, the operator's ability to predict future states using mental models can be more important in controlling responses than feedback. Further, since plant processes cannot be directly observed, their status is 

inferred through indications. Thus, errors in the cognitive process can disrupt performance. 

Failures in response implementation can lead to operating the wrong equipment, or incorrectly operating or 

controlling particular components. 

5.2.5 Implications For Information System Design 

From a consideration of the cognitive tasks performed by operators to support their supervisory control functions, 

the following general observations can be made: 

* Operators' information needs, especially in support of monitoring, detection, and situation assessment, are 

very broad, both in terms of range of activities they need to be aware of, and the level of abstraction they need 

to consider. 

* Information should be expressed in terms that are at the level that will directly impact decision making and 

control. Cognitive resources should not be required to mentally process data to provide the information that is 

needed. 

* It is important to correctly map the plant or process and the identification of information needs and their 

relationship to the operator's mental model. These mappings are necessary to ensure that the information 

representations trigger the appropriate mental representation of the situation. 

• HSI design can have a strong effect on the operator's cognitive activity and can make information difficult to 

obtain (through poor organization, high data density, and excessive interface management demands). 

The following information processing considerations for cognitive tasks were reflected in design principles supporting situation awareness developed by Endsley (1988, 1995): 

* Displays should provide information that is processed and integrated to meet the operators' need to 

comprehend the current situation and to project future states. 

* Information presentation should take into account the operator's goals and the information needed to make 

goal-directed decisions. That is, information should be organized in terms of what the operator is trying to 

accomplish and not simply the physical layout of the system. 

* Since situation awareness depends on activating the appropriate mental representation for the situation, the 

critical cues to activating the schema should be determined and displays should make these cues salient. 

* Cues with highly salient features can redirect attention away from the current goals. Such cues should be 

reserved for situations that require the redirection of goals and critical events.

NUREG/CR-6633 5- 17
5 DEVELOPMENT OF TECHNICAL BASIS

The display should provide global situatQp awareness; that is, it should give an overview of the status of all 

operator goals at all times as wee.miiving details about the current goal. This is because situation awareness 

suffers when operators focus on some information and fail to attend to other important information. 

Support should be given for projecting future states of the system (especially for less experienced operators). 

For example, displays of trends or rate of change support this process. 

Information overload should be avoided by filtering out extraneous information and using processing 

techniques that integrate lower-level data to provide operators with the high-level information needed for goal

directed decision making. 

To minimize divided attention, information should be grouped and embedded within objects when possible. 

The number of attention shifts should be minimized; thus, the number of objects should be minimized, as 

should the need to shift between separate displays. Ideally, all information should be available at a single 

glance. 

The HSI should support parallel processing by providing, for example, information through visual and 

auditory channels. This will help operators to do tasks requiring divided attention. 

5.3 Analysis of Information Requirements 

HFE guidance has been criticized for focusing on legibility and not meaningfulness. Bennett et al. (1997) stated 

that: 

Designs that consider only data availability often impose unnecessary burdens on the user: to collect relevant data, to 

maintain these data in memory, and to integrate these data mentally to arrive at a decision. These mental activities 

require extensive knowledge, tax limited resources (attention and short-term memory) and therefore increase the 

probability of poor decision making and errors. (p. 673) 

However, in the last few years, the NRC has improved the traditional guidance with the issuance of NUREG-07 11 

and NUREG-0700, Rev. 1. These documents contain guidance for information requirements that include an 

operating experience review, a functional requirements analysis, a functional allocation analysis, a task analysis, 

and a human reliability analysis. Section 5.1 summarizes these documents. 

Further, as noted in Section 5.2, Vicente and Rasmussen (1992) suggested that traditional approaches to 

information system design, based on preplanned evaluations of normal, abnormal, and emergency events, will fail 

to meet the operators' needs in the face of events that have not been planned for in advance. 

Proponents of ecological interface design (EID) suggested it as an alternative approach to traditional HFE design. 

The EID approach has implications for identifying operator information needs as well as for developing 

representations to display plant information. The former is addressed in this section and the latter in Section 5.4. 

The intent in this section is to determine whether the EID approach provides alternative or additional methods that 

may be beneficial in the information requirements area. 

5.3.1 The EID Approach to the Analysis of Information Requirements 

The concept of an ecological approach to interface design can be traced back to Gibson's theory of visual 

perception (1979). The central concept in Gibson's approach is that human behavior cannot be understood as an 

NUREG/CR-6633 5 - 18

I I
5 DEVELOPMENT OF TECHNICAL BASIS

entity independent from the environment in which it occurs. Thus, knowledge of the context or domain within 

which behavior occurs is essential to understanding human performance. Gibson called this the ecological 

approach to behavioral analysis (1979), and his concepts have influenced the EID model of interface design. The 

basis for using ecological perception theory as an approach to interface design was described in several papers 

(e.g., Flach, 1990; Vicente and Rasmussen, 1990, 1992). 

In the nuclear industry, the EID approach was supported by Vicente and Rasmussen (e.g., 1992). Rasmussen 

(1986) viewed human performance as representing behavior organized and controlled at various levels of 

abstraction, including perception, rules, and knowledge. Operators are characterized as flexible in their ability to 

respond to the situation. This characterization of action, being based upon various levels or depths of processing, led to the categorization of operator's performance into three classes: skill-based, rule-based, and knowledge-based 

(SRK scheme). 

Skill-based actions means that upon perception of particular signals, the operator initiates a well-learned "over-trained" response. Pilots are trained to this level so that the they can respond quickly without the need for 

more abstract and time-consuming processing. Behavior in this context is largely automatic, initiated, and 

controlled outside of the operator's awareness. 

Rule-based performance is generally more complex and consciously controlled, yet a deep level of processing is not 

required since the situation is interpreted as familiar and recognized as requiring a previously defined set of 

actions. The use of standardized procedures, on which an operator has been trained in response to a well-defined 

transient, reflects primarily rule-based behavior. In Rasmussen's model, most of the linkages, between skill-based 

performance and the "identification of a state of ambiguity" in the system, fall into the domain of rule-based 

performance. 

Knowledge-based processing represents the highest level of abstraction and deepest level of processing. It is 

invoked when neither skill-based nor rule-based responses are appropriate to the situation or adequate to define the 

actions needed to meet the operator's goal. The situation is defined as unfamiliar or ambiguous, and operators 

must develop an approach to attaining goals based upon their in-depth knowledge of the plant and systems. 

Higher levels of abstraction enable behavior to be goal directed. These levels can serve to define goals for lower 

levels. So, for example, the goal to define the present state of the system can structure the set of observations made 

at the lower level of abstraction. When a target or desired state is identified, it becomes a goal, and the tasks and 

procedures to be used to achieve that state are defined. As noted in Section 5.2, the importance of the goal-directed 

structure of information processing has been widely recognized in NPP operator modeling (e.g., Bainbridge, 1974, 

1986). The influence of higher-level goal states and expectations on lower-level cognitive processes also 

recognized in basic cognitive research as well, e.g., the effects of goal states on pattern recognition (e.g., Neisser, 

1976). 

In designing HSIs for these types of behavior, the issue is to determine the information necessary to support them 

at the various levels of abstraction. Based on his observations of diagnosis and problem-solving behavior, 

Rasmussen (1986) noted that information about a system varied along two dimensions: abstraction and 

aggregation. Abstraction reflects the conceptualization of system means-ends relationships along a hierarchy 

consisting of five levels from physical form to functional purpose. These levels were defined as follows: 

* Functionalpurpose- the purpose for which the system was designed 

* Abstract function - the causal structure of the process in terms of mass, energy, information or value flows 

5- 19 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Table 5. 1 Abstraction-Aggregation (A-A) Matrix 

Level of Level of Aggregation' 

Abstraction 

Plant System Equipment Component 

Functional Purpose 

Abstract Function 

Generalized Function 

Physical Function 

Physical Form 

'Levels of aggregation as typical for a nuclear power plant, but they are examples only. 

• Generalized function - the basic functions a system was designed to achieve 

* Physical function - the characteristics of the components and their interconnections 

* Physical form - the appearance and spatial location of those components. 

These levels span the physical components in the system and its overall functional purpose. Since SRK behavior 

requires information about the plant at various levels, the abstraction hierarchy provides a framework within which 

to describe those needs and consider display design. 

Aggregation reflects the degree of integration, from part to whole, that problem-solving performance is directed 

toward. The part-whole dimension also consists of levels ranging from the individual component to the entire 

system. These two dimensions form a cognitive matrix describing the information needs of plant personnel, called 

the abstraction-aggregation (A-A) matrix (see Table 5.1). 

The problem solver shifts between levels of abstraction and aggregation to solve problems. This enables the 

problem to be conceptualized in different ways. For example, Rasmussen (1986) mapped the strategy used by a 

computer analyst during troubleshooting into this problem-solving space (Figure 5.2). Itoh et al. (1995) applied 

this two-dimensional concept to interface design for nuclear plants (Figure 5.3). 

Thus, during diagnostic performance, the information needs of the operator vary within this problem space. 

However, the two dimensions tend to be coupled (Vicente, 1992); that is, one tends to think at lower levels of 

abstraction when dealing with components, and to think functionally when working at the plant level. Lower 

levels of abstraction support knowing what to do and higher levels support an understanding of why it is necessary. 

In contrast to the information needs reflected by the A-A matrix, Rasmussen (1986) suggested that in many 

industrial complexes, HSIs provide only one level of abstraction, reflecting a 'one sensor - one indicator' approach.

NUREG/CR-6633 5 -20

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Decompo

ssition Whole system Subsystem Functional unit Circuit-stage Component 

Abstraction I 

No communication Functional to tape or disk 

meaning; Test program 

purpose 1( 

Joint node in Proa nd Flags and conditions 

information flag crondaitoios9 Abstract paths function 2 4 

"••kInstructions 

Generalized 

function 

Interface to Solenoide circuit Transistor 

Physical DEC-ite 1 Solenoide driver short-circuit 

function Not Manipulation 

keyboard of power supply. 13 15 but - Study function 

Physical 

form Is here Looks burned 

Figure 5.2 Diagnostic Strategies Used in Troubleshooting 

(Source: Reprinted with the author's permission from Rasmussen, J., "Information Processing and 

Human-Machine Interaction," North Holland, New York, NY, 1986. All rights reserved by the author.) 

This makes tasks complex because the data available often do not match the information needed. Rasmussen noted 

that 

In the design of human-machine interfaces for supervisory control as, e.g., industrial process control consoles, the 

emphasis has traditionally been on the presentation of measured data representing the physical state of the system and 

its processes. In complex systems, this information has been supplemented by information about the underlying functional structure by graphical means such as mimic diagrams, etc. This information is intended to serve the 

controller's identification of the actual state of processes bottom-up through the hierarchy. Information representing the intentions behind the system design, in terms of the purpose of functions and equipment and of constraints upon the acceptable operation, for instance, as derived from safety considerations, has only been very sparsely represented. 

This means that the interface system gives little or no support in the top-down derivation of the proper or acceptable 

states of processes. This kind of information was supposed to be immediately available to operators from their basic 

training. However, as systems become complex and potentially risky.., this can no longer be assumed. During such 

situations, information about the purpose of interlocks, properties of equipment if used for untraditional purposes, 

etc., may be vital for ad hoc improvisations. Consequently, it becomes increasingly important to include in the 

support for decision making the information needed for top-down consideration of reasons. This means that the 

properties of the system to be controlled should be represented in terms of a consistent means-end hierarchy, 

systematically mapping the purpose/function/equipment relationships. (pp. 130-131)

5 -21 NUREG/CR-6633
FUNCTIONAL 

PURPOSE

SELECTRICITY IGENERATION

ABSTRACT 

FUNCTION

GENERALIZED 

FUNCTION

PHYSICAL 

FUNCTION

Figure 5.3 Electrical Production Abstraction Hierarchy 

(Source: Reprinted from Itoh, J., Sakuma, A. and Monta, K., "An Ecological Interface 

for Supervisory Control of BWR Nuclear Power Plants," Control Engineering Practice, 

3(2), 1995, Copyright 1995, with permission from Elsevier Science.)

NUREG/CR-6633

5 DEVELOPMENT OF TECHNICAL BASIS

5 - 22

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Rasmussen indicated that this situation places the burden on operators to find the information they need to support 

their current activities and to structure the representation of the system that best meets their information needs. 

The advantage. of defining information needs using the A-A matrix is that normal plant behavior is dictated by a 

set of known relationships between plant parameters governed by constraints dictated by the physical nature of 

plant equipment, the physical laws governing the process, and the purposes for which the system was designed. 

Faults are revealed when these constraints are violated. By analyzing information needs according to the A-A 

matrix, these constraints can be represented in the design of the information system. 

Vicente (1997) suggested that process control operators exist in a correspondence-driven domain where there is an 

external reality. This concept is in contrast to a coherence-driven domain where the human and the computer 

system represent a closed world and do not need to reflect external constraints. The information system should 

reflect the actual constraints of the process being controlled. Vicente refers to this as the "ecological compatibility 

principle" (in contrast to the proximity-compatibility principle, discussed in Section 5.4). 

According to Vicente and Rasmussen (1988, 1992), the general goal of ecological interface design is to "...design 

interfaces that do not force cognitive control to a higher level than the demands of the task require, but that also 

provide the appropriate support for all three levels" (p. 598). The following three general principles, related to 

skill-, rule-, and knowledge-based behavior, are proposed to achieve this goal: 

Skill-based behavior - "To support interaction via time-space signals, the operator should be able to act 

directly on the display, and the structure of the displayed information should be isomorphic to the part-whole 

structure of movements" (Vicente and Rasmussen, 1992, p. 598). This means that, as far as possible, the 

interface should give the information necessary to allow for the appropriate use of skill-based behavior, which 

is the least cognitively demanding. Designs that use the natural tendencies of the user support this objective 

(i.e., direct-manipulation interfaces). 

Rule-based behavior - "Provide a consistent one-to-one mapping between the work domain constraints and 

the cues or signs provided by the interface" (Vicente and Rasmussen, 1992, p. 598). That is, provide a 

mapping between the invariant properties of the process and the format of the HSI. This principle is supported 

by showing contextual information in a display, such as stating the limitations of displayed information. This 

allows operators to identify potential inadequacies of typical responses that may be inappropriate to the current 

situation, thus avoiding procedural traps (Vicente, 1991). 

Knowledge-based behavior - "Represent the work domain in the form of an abstraction hierarchy to serve as 

an externalized mental model that will support knowledge-based problem solving" (Vicente and Rasmussen, 

1992, p. 599). That is, display the relational structures of the processes as a functional hierarchy. This 

principle is supported by providing information in familiar representations, such as displaying functional 

relationships and information on parameters as P&IDs and in abstractions, such as subcooling margin. 

In the next section, we address the effectiveness of the EID approach in the analysis of information requirements. 

5.3.2 Research on the Effectiveness of EID Displays 

5.3.2.1 DURESS EID Display 

Vicente and colleagues conducted several studies to assess the value of the EID approach to display design. 

Vicente (1991) undertook the studies in part because "...no experiment has ever compared the performance of a

5 -23 NUREGICR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

multilevel interface based on the abstraction hierarchy with any other type of interface" (p. 65). One series of 

Vicente's studies used a thermal-hydraulic process simulation called DURESS (DUal REservoir System 

Simulation) and DURESS II, with an updated simulation model. 

The DURESS model has two redundant feedwater streams, which feed two reservoirs under various configurations. 

The operator tries to keep each reservoir at a specific temperature and to maintain water levels as needed based on 

flow rates. The operator accomplishes this task by controlling valves, pumps, and heaters. The simulation is 

dynamic with time lags on control variables. Faults can be introduced into the system and equipment can be 

damaged by improper operations (such as allowing the water level to get too low in a reservoir while it is heating). 

Operators controlled the simulation using one of two interfaces: a basic physical representation (P) of the process, 

or a physical plus a functional display (P+F). Figures 5.4 and 5.5 show the interfaces. The P interface was a 

mimic-type display showing two feedwater systems (A and B), their components and status such as: P (Pump on or 

off), V (valve position), HTR (heating rates) for reservoirs, D (demand), and T (temperature). In addition to the 

representation of system components and parameters, the P+F display contained functional information identified 

by analyzing the abstraction hierarchy. That is, the display included information on the status of the functions that 

the components were serving and the mass-balance display (see the right side Figure 5.6). 

The DURESS testbed provided an opportunity to experimentally study display effects in the laboratory with a 

simplified simulation that has many characteristics similar to actual process control systems. The simulation and 

the interfaces are described in detail in Vicente (1991) and in many of the other Vicente references discussed in 

this section. Collectively, these studies examine the effects of the following: underlying design philosophy on 

performance, varying the information content of displays, expertise on performance, displays on mental models, 

and displays on normal operation and fault management performance and strategies. 

Before examining the results of these studies, we will examine the basis for the two interfaces in more detail. This 

is important because the studies use P (traditional design) vs. P+F (EID design) as the principal independent 

variable. Therefore, general conclusions about the strengths and weaknesses of EID are based on the 

representativeness of these designs. The design of the interfaces is discussed in detail in Chapter IV of Vicente 

(1991). To briefly summarize, the EID process first requires representing the domain constraints. That is, the 

elements of the system are described in terms of the abstraction hierarchy. Thus, for example, the functional 

purpose (high-level objective) of a reservoir is to maintain sufficient water to satisfy output demand and to keep the 

water at a temperature criterion. The next step, building an interface that represents the abstraction hierarchy, was 

accomplished by mapping this information into visible geometric properties of the objects in the interface. This is 

how the representation is thought to lessen the operator's workload by providing an external, visible model of the 

system. This latter step is not actually a function of EID; EID provides the requirements for the information that 

needs to be represented, not how to represent it. The following elements in the P+F interface (see Figure 5.6) 

reflect the result of this design process: 

* Functional purpose - Demand values D1 and D2 and temperatures TI and T2 

* Abstract function - the rectangular graphic (to the right) representing mass balance of the reservoir 

* Generalized function - flowrates for feedwater streams, such as FA I and FVA 

* Physical function - valve settings, such as VB 

* Physical form - the spatial layout of the components and their connections 

NUREG/CR-6633 5 - 24

I I
5 DEVELOPMENT OF TECHNICAL BASIS

VA1 

10 

VA Reservoir 1 (40 C) Ti 

10 0 00 D1i 

VA2 

PA 10 -Vi 

0 

00 0--- h, .... T ...,,1 

0 10 VB1 HTR1 

10 

Reservoir 2 (20 C) T2 

VB 032 -1 20 

1 0 00 D 

1 20 Va2 V2 

PB 10 

0". 0 

Figure 5.4 Physical Interface (P) for DURESS 

(Source: Reprinted with the author's permission from Vicente, K., "Supporting Knowledge-Based Behavior Through Ecological Interface Design," (EPRL-91-01), University of Illinois, Engineering 

Psychology Research Laboratory, Urbana, IL, 1991.) 

The P+F EID interface appears to be a well thought-out example of an EID process. 

The design basis for the P interface, which is a traditional interface, is not discussed. However, it is not clear that 

the P interface is the product of a design approach that is an alternative to the EID process. Thus, one cannot 

determine why basic parameter values that appear in the P+F interface do not appear in the P interface. This latter 

point was discussed in an interesting exchange in Human Factors addressing a critique of Christoffersen, Hunter, 

and Vicente (1996) by Maddox (1996) and Vicente, Christoffersen, and Hunter (1996). The lack of information on 

flow rate either from the pump or to the reservoirs is noteworthy. Only information on valve position is given. 

Typical process displays would more likely include information on flow rate rather than valve position, and flow 

rate should be more useful for process control. If the P interface did not reflect a systems analysis or even a 

"traditional" task analysis, then what was used to determine the information requirements of the P interface? This

5 -25 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

02 a s 

A t

Figure 5.5 Physical + Functional (P+F) Interface for DURESS 

(Source: Reprinted with the author's permission from Vicente, K., "Supporting Knowledge-Based 

Behavior Through Ecological Interface Design," (EPRL-91-01), University of Illinois, Engineering 

Psychology Research Laboratory, Urbana, IL, 1991.) 

is an important question; its answer affects any conclusions on the generalizability of identified differences in 

performance between the P and the P+F displays. 

With this issue in mind, the research related to the EID DURESS interface will be discussed. Seven different 

experiments of the series are discussed here. 

The first experimental evaluation of the interfaces is reported in Vicente (1992) (and Experiment 1 of Vicente, 

Christoffersen, and Pereklita, 1995). The evaluation compared the two interfaces on fault diagnosis accuracy and a 

recall memory measure. A second independent variable was expertise (expert and novice). Experts were graduate 

students in mechanical or nuclear engineering, while the novices were graduate students who had never been 

enrolled in a science or engineering major. Subjects were first provided with familiarization and training on the 

simulator and interface. There were 12 participants in each expertise group. Participants observed the DURESS 

NUREG/CR-6633 5-26

I I

m •:lU •
5 DEVELOPMENT OF TECHNICAL BASIS

simulation for approximately 30 seconds. In some of the trials, faults, such as reservoir leak and a blocked valve, 

occurred. Following the presentation, the screen went blank, and the participants were asked to recall the value of 

system parameters and diagnose the functional state of the system. 

The results showed that the P+F interface was better than the P display in support of diagnosis. Generally, this 

result was stronger for experts than for novices. Vicente (1992) indicates that the superiority of diagnosis using the 

P+F interface supports the value of including higher-level functional information in a display, which he believes is 

critical to diagnosis. On the memory measures, experts had better recall than did novices; however, the effect of 

recall using the P+F display was only better for functional information. (The memory results were reported in 

more detail in Vicente [1992]. It was reported that memory for the subset of variables that were most important to 

the diagnosis was better for the participants using the P+F display.) Vicente also concluded that the study provided 

empirical support for displaying various levels of the abstraction hierarchy in an interface. That is, he concluded 

that the results are not due simply to the fact that the P+F display has more information than the P display, but that 

the additional information was specified by the EID approach as being goal relevant. 

In the second study, Pawlak and Vicente (1996) compared the P and P+F displays to determine their relative effects 

on (I) control strategies and (2) fault detection and diagnosis. They hypothesized that the EID display would lead 

to the development of more effective control strategies. Twelve participants were assigned either to the P or the 

P+F display (six each). They practiced for an hour a day for 27 days and performed the following tasks: start-up, 

tuning to new setpoints, shutdown, and fault management. For some of the trials, participants had to conduct 

secondary tasks along with their primary tasks. The secondary task loaded on either verbal or spatial cognitive 

resources. Measures of primary task performance were time and analysis of verbal protocols. 

The results showed that the P+F display led to more accurate diagnosis than the P display, although the time to 

compensate for the fault did not differ between the two groups (compensating for a fault does not necessarily 

require a diagnosis). Further, the strategies used by the two groups were different. Although some differences in 

strategy on the startup task were observed, no significant differences in general task performance were found. 

Analyses of the workload measures showed that the P+F display loaded more on spatial cognitive resources, while 

the P interface loaded more on verbal resources. 

The authors identified some limitations to the study. First, it is not known whether differences in diagnostic 

performance were due to differences in the content or in the form of the displays. (This is a key point relevant to 

interface design, because the added information could easily be given in a more standard format, resulting in a very 

different display.) Second, the participants were not skilled in detecting faults and it is not known whether the 

benefits of the EID interface would hold for true operational experts. 

In the third study, EID displays were compared with P&ID displays in a longitudinal examination of the 

performance of a variety of simulated process control tasks (Christoffersen et al., 1994, 1995, 1996). The DURESS 

II simulation was used and the P and P+F displays were compared. Six students participated in the study. They 

were assigned to either the P or the P+F condition in a between-subjects design and did the same operational tasks 

as described above. The participants engaged in 224 trials over 6 months. A brief crossover was performed where 

participants used the other group's displays. The primary performance measures were trial completion time and 

verbal protocols. 

The most significant finding was that the EID display led to faster fault detection, more accurate fault diagnosis, 

and faster compensation. These differences were observed for routine as well as non-routine faults (Christoffersen 

et al., 1995). Also significant is that their strategies were different than the P group. The P group had difficulty 

distinguishing between a fault and a normal situation requiring control action. They sometimes only recognized a 

fault after normal control actions failed. They typically did not diagnose the failure and handled it by trial and

5 -27 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

error. In contrast, P+F participants typically did not adopt such a strategy. Instead, they were often able to 

diagnose the problem, and therefore, respond appropriately. 

For performance on a normal (nondisturbance) trial, there was very little difference in mean performance between 

the two groups. However, the P+F group was significantly less variable than the P group. Interestingly, during the 

crossover trials, when participants in the P+F group used the P interface, their performance variability increased. 

Thus, the differences were likely due to the characteristics of the interface and not the characteristics of the 

participants alone. This finding is important because assuming acceptable mean performance, it is important from 

a human reliability perspective to reduce human variability. 

In the fourth study, the importance of the functional level of information was examined by having participants 

control the DURESS II process using a windows-based interface (Janzen and Vicente, 1997). The windows 

presented distinct information reflecting four of the five levels of the abstraction hierarchy: goals, principles, flow, 

and settings. No form level was included because the process was simulated. Only one level could be observed at a 

time. Of interest were which levels were used, for what purposes, and with what results. Six participants in three 

types of trails (normal, routine fault, and non-routine fault) operated the simulation for approximately one hour per 

workday for one month. The performance measures included system variables, operator 'actions, and verbal 

protocols. Regression analyses were conducted to predict process control performance, based on the participant's 

attention allocation strategies. 

The results showed that more accurate and faster control under normal conditions was achieved when more 

functional levels were consulted (relative frequency of visits to the principles level). In addition, while fault 

detection time was not predicted, allocating attention to more levels (relative frequency of visits to the principles 

and flows levels) was associated with more accurate diagnosis. For fault compensation, performance was improved 

by dwell time in the principles level. The authors noted that to simply complete the process control task, only the 

settings and goals levels were necessary. To complete it quickly and stably, the principles level is necessary. 

Further, to handle faults, moving between multiple levels is required; i.e., most faults are revealed by switching 

between the settings and flows levels or between the flows and principles levels. These results showed the 

importance of the different levels of the abstraction hierarchy to different operating tasks. 

In the fifth study, Vicente and Wang (1996) extended the comparison of effects of EID performance by comparing 

the following five interfaces, which included those previously described and some variations on them: 

• the P display, 

* the P display+trend plots (for input flow rates, level, and output flow rates for each reservoir), 

"* the P+F display, 

"* the P+F display+trend plots, and 

"• the P+F display+analytical redundance. 

Analytical redundance is the calculation of expected parameter values using a model of system performance. The 

expected value is then represented in the display along with the actual value. Deviations between the two show 

some disturbance or abnormality of the system (Figure 5.6). The difference between the actual and expected flow 

rate is represented by the difference between the solid and dashed lines connecting MI and MO, which means a 

leak in the reservoir. Such a display directly reveals the violations of process constraints introduced by the fault. 

NUREG/CR-6633 5-28

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Figure 5.6 EID P+F Interface with Analytical Redundancy (Source: Reprinted with permission from Vicente, K., and Wang, J., "Taking Full Advantage of Process Constraints in Advanced Interface Design," 

Proceedings of the 1996 American Nuclear Society International Topical 

Meeting on Nuclear Power Plant Instrumentation, Control and Human

Machine Interface Technologies, pp. 405-411, 1996. Copyright 1996 by the 

American Nuclear Society. All rights reserved.) 

Participants operated DURESS under startup tuning and fault management trials. Performance measures included 

task times, fault detection time and accuracy, and verbal reports. The fault detection time was better for the 

EID+analytical redundancy, but only for reservoir leaks, as expected. They were better at fault compensation as 

well. For diagnosis, the three EID groups were more accurate than the P groups. Including trend plots did not 

clearly improve fault management, and in some. cases even decreased performance. 

In the sixth study, Howie and Vicente (1996) followed the expertise developed by six novice operators of the 

DURESS II process simulation. They anticipated greater expertise would develop in a group using an EID 

interface. Half were assigned to a P interface group and half to a P+F interface group. Perhaps the most 

interesting aspect of this study is the dependent measures developed for analyzing performance: steady-state times, 

complexity of action transition graph, path length of state space, mass-energy inventories, and timelines. The 

operators completed 224 trials over 6 months, and the first 20 trials were compared with the last 20 trials. 

Performance improvements of larger magnitude were reported for the EID group, which may be because they tended to perform more poorly in the first 20 trials. By the last 20, both groups performed similarly. This pattern held for the first three measures. As an example, Table 5.2 shows the average steady-state times. Faster times 

show an ability to reach goals quickly and to maintain the system in the goal state. There may be two reasons for 

this result. First, with only three participants per group, there is no reason to believe the groups were equivalent at 

the start of the study. Perhaps by chance, the EID group performed more poorly at the task in the beginning. Second, the EID group may have been harder for the participants to use during early trials so they performed 

poorer. Over time they may have learned to use the EI) display and improved. If the latter interpretation was 

correct, it would not represent an especially positive effect of EID displays. For the mass-energy inventory, the 

P+F group performed better for one of the two reservoirs in terms of percent above 1/3 capacity.

5 -29 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Table 5. 2 Mean Steady-State Times (in seconds) 

Initial Trials Final Trials 

P+F Group 687.6 395.1 

P Group 602.2 393.3

In the seventh study, Vicente, Christoffersen, and Pereklita (1995) evaluated the hypothesis that superior fault 

management performance with ELI) interfaces occurs because these interfaces encourage knowledge-based 

behavior by enabling operators to (1) begin diagnosis at high levels of abstraction, reflecting functional 

considerations, and (2) use that information to work their way down to problematic components at the lower levels 

of abstraction. To evaluate this hypothesis, fault diagnosis strategies were examined as a function of type of 

expertise and success at fault management. It was found that the relevant expertise in managing DURESS faults 

was knowledge of the DURESS system and the P+F interface. Experience at controlling DURESS and 

thermodynamic knowledge were not well correlated with performance. For strategies of fault management, better 

performers began at higher levels of abstraction and slowly worked their way down to lower levels. In fact, 

diagnosis strategies starting at the lower levels had a negative effect on performance. In part, this is because at 

lower levels components have many interconnections. Thus, it is difficult to determine where a problem resides. 

In summary, the main result of these studies is that fault detection and diagnosis seemed to be improved with the 

EID display reflecting the abstraction hierarchy. There was also a reduction in control performance variability. 

However, several issues make it difficult to fully interpret the results. 

The first issue is generalization of the findings as representative of the strength and weaknesses of the design 

approaches they intended to represent. The uncertain technical basis for the P display make it unclear whether the 

results reflect the contribution of EID over traditional design methods, or whether they reflect the difference 

between a display that was designed with operator information needs defined and one that was not. Or, perhaps 

more likely, all three aspects of the EID displays (added information about flow rate, improved display design, and 

analytical redundancy) created improvements in performance, but how much each aspect contributed or which one 

is more important cannot be determined. 

The second issue is generalization of the findings to more complex systems. DURESS is a relatively simple system 

compared to a NPP. Thus, all the levels of the abstraction hierarchy could be represented in a single display, 

which would not be possible in an actual NPP. Consequently, the amount of information (note the difference in 

amount of information between the P and P+F interfaces and generalize to an entire plant) and its organization 

(with regard to functions, systems, and levels of the hierarchy) may be an issue. 

The third issue is generalization of the findings to more complex and routine tasks. The main benefits of EID were 

seen in knowledge-based diagnosis. In an actual plant, this cognitive activity is supported by alarm systems and 

other HSIs and procedures. The support of EID displays in the context of a richer HSI and task environment needs 

to be addressed. Further, while the effects on diagnostic performance are important, a NPP information system 

must support all of the cognitive activities discussed in Section 5.2. Since most of the activities are highly 

proceduralized, including responses to planned events and EOPs, using displays based on ElI) for these tasks 

should be evaluated. 

NUREG/CR-6633 5-30

I I
5 DEVELOPMENT OF TECHNICAL BASIS

The fourth issue is generalization of the findings to other participants, especially expert operators. Some of the 

studies were based on very small sample sizes, and therefore, the results may be influenced by individual 

differences (Howie and Vicente, 1996). Even when statistical tests are conducted to determine the significance of 

observed differences, there is no guarantee of the generalizability of results from small samples, because 

generalization is based on the assumption that randomization has resulted in equivalent groups. The confounding 

effects of individual differences are minimized by randomization of participants to groups. When accomplished in 

large numbers, the observed differences between groups can be attributed to the independent variable. When this is 

not accomplished, the results may be due to the manipulations of the independent variable (e.g., EID vs. traditional 

display) or to preexisting differences in ability between the groups (which is not accounted for in the statistical 

tests). Further, generalization to significant groups not represented in the studies are not supported. In this case, 

the effects on expert operators are unknown, and (as discussed below) they may interact differently with the 

displays due to their expertise. 

The fifth issue is identification of an appropriate mental model. Vicente concluded that "an interface based on an 

abstraction hierarchy can provide more support for knowledge-based behavior than an interface based on physical variables alone because it results in a better match to the theoretical expert's model" (1991, p. iii, emphasis 

added). However, the participants in these studies were generally not experts in operations. Thus, they probably had no competing model as expert, trained process control operators might have. Training was noticeably weak in 

these studies. Yet training is the primary source of the mental model developed by professional process control 

operators. 

Some of these issues, notably the use of EID displays by professional operators, were addressed in the research on 

the Rankine cycle display, which is discussed in the next section. 

5.3.2.2 Rankine Cycle EID Display 

Introduction 

Abstract function representations of any process plant (such as a NPP) can be built on the conservation of energy 

and mass and on thermodynamic principles. One example is the Rankine cycle display (Beltracchi, 1987, 1989, 

1995). Numerous other examples of such displays were developed (Duncan and Praetorius, 1992; Hollnagel et al., 

1984), which show important aspects of the process cycle (e.g., heat source and heat sink) integrated into one 

display. The Rankine cycle display was incorporated into the information system of the Experimental Breeder 

Reactor (EBR II) at Argonne National Laboratory (Lindsay, 1990). The Rankine cycle display, its thermodynamic 

rationale, and its relationship to EID was described by Vicente et al. (1996). 

The general functioning of a typical NPP, the thermodynamics behind the Rankine display, the ideal Rankine 

cycle, and the Rankine cycle EID display are discussed below.

5 -31 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Pump

Figure 5.7 Boiling Water Reactor (BWR) 

Discussion of NPP in Physical Terms 

Figure 5.7 shows a top-level view of a boiling water reactor (BWR). The heat source is the nuclear reactor, in 

which water is boiled directly in the reactor vessel, creating steam. In a BWR, a recirculation loop and pump 

provides forced circulation and allows higher power levels. Steam leaves the reactor and flows down the main 

steam lines to the steam turbine, which is commonly used in both fossil fuel and NPPs to convert the thermal 

energy of the steam to rotational energy. An electrical generator is attached to the turbine shaft and converts the 

rotational energy to electrical power. After the steam passes through the turbine, it is cooled and condensed into 

water in the main condenser. The condenser is usually supplied with river, lake, or ocean water for cooling. After 

cooling, the water is pumped back to the reactor to complete the cycle. As will be discussed later, this process is a 

thermodynamic Rankine cycle (named after William J. M. Rankine, a Scottish engineer). 

Figure 5.8 presents a top-level version of a pressurized water reactor (PWR). In this type of reactor, the primary 

water, which is heated in the reactor vessel, is kept under high pressure (typically about 2250 psia) by a pressurizer 

to prevent boiling and is circulated through a steam generator by reactor coolant pumps. The hot primary water (at 

about 6000F) serves as the heat source to the steam generator, where secondary water is boiled. The primary and 

secondary water do not mix. The steam produced in the steam generator then flows down the main steam lines to 

the turbine-generator, and condenser. After cooling, the water is pumped back to the steam generator to complete 

the cycle. 

Discussion of Water 

Water is used in both BWRs and PWRs as the reactor coolant and the fluid that is boiled and sent to the turbines to 

produce electric power. The water exists as a liquid, a gas (steam), and a combination of the two. Its particular 

state at various locations of the process systems is important for both production and safety. From a 

NUREG/CR-6633 5 - 32

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Reactor Coolant Pump 

Figure 5.8 Pressurized Water Reactor (PWR)

thermodynamic standpoint, the state of water can be described using temperature, pressure, and entropy, as shown 

in Figure 5.9. Temperature (T) is on the y-axis and entropy (s) is on the x-axis. Entropy is a mathematically 

defined property of a substance that increases as heat is added. It is a measure of the disorder of a system at the 

microscopic level (i.e., molecules vibrating more as temperature is raised or as water boils). Constant pressure 

lines (P1 to P3) also are shown. 

To the left of the curve in Figure 5.9, water is a subcooled liquid, with a temperature less than the boiling point for 

both unpressurized water and the primary water in a PWR. To the right of the curve, the water is superheated 

steam, with a temperature above the boiling point. Starting at the curve on the left side, water is saturated with a 

temperature at the boiling point for a given pressure. As pressure increases from P1, to P3, along the saturated 

water line, the boiling point temperature increases. For a given constant pressure, if saturated water is heated (as 

in a BWR reactor vessel or PWR steam generator), the water will begin to boil and its properties will move along 

the horizontal constant pressure lines on Figure 5.9. This shows the quality of the steam going from 0% (saturated 

water) through 10% steam, 90% steam, to 100% steam on the saturated steam line on the right of the curve. If 

more heat is added, the steam then becomes superheated. 

Ideal Rankine Cycle 

The power plant steam cycle is termed a Rankine cycle and can be displayed graphically on the temperature (T) 

versus entropy (s) plot for water. This is shown in Figure 5.10.

5 -33 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

800 

70 Saturated Saturated Steam Line 

Water LiUne , * 600 - Constant 

Pressure 

Lines "So 0 Sub-Cooled i-Superheated Steam 

Liquid Water/ Liquid-Vapor \ ?. (Vapor Phase) 

400 -Phase of Water 

tI- I 

F.-200 1 or. Quality 90• Qualiyh 

100 / . 

0 

0 0.2 0.4 0.6 0.8 LO 1.2 1.4 1.6 1.8 2. 2,2 

ENTROPY, BTU/#m °F (s) 

Figure 5.9 Thermodynamic Diagram of Water and Steam 

Figure 5.10 shows an ideal Rankine cycle with no thermodynamic losses (such as friction). The cycle physically 

consists of a boiler of some type, a turbine where work is extracted from the steam (for conversion to electricity), a 

condenser where the exhaust steam from the turbine is condensed into liquid, and a pump where the condensed 

water is pumped back to the boiler to begin the cycle again. Each of these steps is shown on the figure. Step b-c is 

the boiler where the water is heated along a constant pressure line. The water is first heated to saturation, then 

boiled (following the constant temperature horizontal line), and finally the steam is superheated, following the 

same constant pressure line. U.S. reactors do not superheat their steam, but rather send saturated steam (at about 

99.25% quality) to the turbine. The area under the line b-c represents the total heat or energy added from the 

boiler. Step c-d represents an ideal turbine where the energy is extracted from the steam at a constant value of 

entropy. A real turbine contains some losses (not all of the steam energy is converted to useful work), and hence, 

line c-d for a real turbine would slope to the right as entropy increases. Step d-a represents the condensation of the 

steam to water in the condenser across the constant temperature line, and then some subcooling of the water along 

a constant pressure line that is very close to the saturated water line. The area under line d-a represents the heat or 

energy rejected to the heat sink in the condenser, which is called unavailable heat or energy. The area between the 

lines b-c and d-a represents the energy converted to work in the turbine. By ratios of these areas, the theoretical 

efficiency of the process can be calculated. Step a-b represents the increase in pressure of the water in the 

condensate and feedwater pumps, as it is sent back to the boiler. In the real case, there would be some losses in the 

pumps and some increase in entropy, causing the line to curve slightly to the right. Processes to the left of the 

curve stay very close to the saturated liquid line in this figure, due to the relative incompressibility of water. 

Discussion of Rankine Cycle Display 

Figure 5.11 shows the display that was evaluated in the experiment described in Vicente, Moray, et al. (1996). The 

display, based on the Rankine cycle,, represents a PWR plant which uses the following: a steam generator, a two

stage turbine with a high-pressure and a low-pressure stage, steam reheated and then superheated in between the 

two turbine stages, a condensate pump and a main feedwater pump to return the condensed water to the steam 

generator, and two stages of feedwater heaters to preheat the feedwater before its return to the steam generator. 

NUREG/CR-6633 5-34

I I
5 DEVELOPMENT OF TECHNICAL BASIS

LiquidVapor 

E 

UnaviEIable+eat 

Entrop. (s) 

Figure 5.10 Rankine Cycle 

This display reflects a real (rather than ideal) Rankine cycle, with additional information and graphics 

superimposed upon it. The correspondence of the display shown in Figure 5.11 to the ideal Rankine cycle 

(Figure 5.10) will be described first and then the added information on the display will be addressed. For 

discussion purposes, the letters a through g were added. 

The process step b-c in Figure 5.11 represents the addition of heat and boiling in the steam generator similar to the 

ideal step b-c in Figure 5.10. This includes heating of the water to saturation and then boiling all the water to 

100% steam. There is no superheating of the steam in the steam generator. The lines to the left of the saturated 

water line appear different; this will be discussed below. Step c-d in Figure 5.11 represents extracting energy from 

steam in the turbine. After the first drop in temperature in the high pressure turbine, reheat is added to the steam 

to first bring it to the saturated steam line, and then to superheat it. Next, temperature drops again in the low 

pressure turbine down to point d. Step d-a represents the condensation of the steam to water in the condenser 

across a constant temperature line and then some subcooling of the water. Finally, Step a-b represents the increase 

in pressure of the water in the condensate and feedwater pumps, as it is sent back to the steam generator, as well as 

the feedwater preheating that will be discussed below. 

Figure 5.11 differs noticeably from the ideal Rankine cycle in one respect. The two lines representing extraction of 

energy from steam in the turbine (along step c-d) slant to the right rather than being a vertical constant entropy 

line, due to various losses in the process. 

A substantial amount of additional information was added to the display in Figure 5.11. Points and information on 

the display are determined by measured parameters sensed at key points in the plant, such as temperature, pressure, 

and water levels. As parameter values change, the polygon display adjusts to these points as appropriate.

5 -35 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

P-.rU 00 :1 

a

Figure 5.11 Rankine Cycle Display 

(Source: Reprinted with minor modification with permission from Vicente, K., Moray, N., Lee, J., 

Rasmussen, J., Jones, B., Brock, R., and Djemil, T., "Evaluation of a Rankine Cycle Display for 

Nuclear Power Plant Monitoring and Diagnosis," Human Factors, 38(3), 1996. Copyright 1996 by 

the Human Factors and Ergonomics Society. All rights reserved.) 

While the display is based on a standard temperature-entropy (T-s) chart, it does not directly display the entropy 

axis. The constant pressure lines are only displayed in the two-phase central portion and the superheat portion of 

the display, since they would all be too crowded together in the subcooled region. Also, the portion of the figure to 

the left of the liquid saturation line (letters a, b, e, and f) was changed to represent the degree of subcooling by 

distance to the left of the curve. Thus, points a and b in Figure 5.11 appear differently from Figure 5.10. 

Furthermore, the display in this area after the condenser represents the following: first, a pressure increase in the 

condensate pump, next an increase in temperature in a feedwater heater, then an increase in pressure in the 

feedwater pump, and a further increase in temperature in a second feedwater heater. This portion of the display is 

not discussed by Vicente et al. (1996). 

The display also includes information on the primary loop, the pressurizer, some liquid levels, and relief valves on 

the primary and secondary sides, as follows. The temperatures of the primary loop (reactor coolant system or RCS) 

and the heat transfer from the primary to secondary sides are displayed by the small trapezoidal-shaped figure with 

the line e-f. The horizontal line from point e represents the cold leg of the RCS. It is drawn at the temperature of 

the cold leg, Tc. Also, the distance from the saturation curve line to point e represents the amount of subcooling in 

the cold leg. This is an important feature because the RCS in a PWR should always remain subcooled. Similarly, 

the line from point f represents the hot leg, and is drawn at the hot leg temperature, TH. The subcooling is less for 

the hot leg than the cold leg. The distance between TH and Tc is proportional to the heat transferred from the RCS 

to the steam generator. From this trapezoid, the operator can infer that the primary system is properly subcooled

NUREG/CR-6633

I&

5-36
5 DEVELOPMENT OF TECHNICAL BASIS

and is correctly transferring heat to the secondary side. Actual values are not included here, and other displays or 

indicators would be needed to show, for example, the subcooling margin (how far away the primary is from 

saturation). As an example, a typical SPDS alarm for low subcooling margin is set at 32°F to allow a margin for 

instrument uncertainty. 

The horizontal lines within the saturation curve represent the steam generator (the line left from point c), the 

condenser (the line left from point d), and the pressurizer (the line left from point g). These lines are at the correct 

temperature and pressure for each of those components. For example, the line left from point g for the pressurizer 

is at a pressure of about 2250 psia and a temperature of about 653 *F. The water level in each of these three 

components is represented by the small square box on the horizontal line. The state of "all water" corresponds to 

the box at the left on the saturated water curve and "all steam" (no water) is with the box at the right on the 

saturated steam curve. In between, the position of the box represents a percent level. 

The P&ID-like graphic under the Rankine curves represents the circulating water system that cools the condenser. 

Also, relief valve icons were added for a pressurizer relief valve and a steam generator relief valve, and a main 

steam line valve icon was added on the horizontal line for steam generator. These changed color when open or 

closed. Various other portions and lines on the display were color coded to aid the operators. 

Thus, while the display of Figure 5.11 is based primarily on the Rankine cycle, significant additions and changes 

were made to give more information relevant to various aspects of the NPP. 

Evaluation of EID-Tvve Displays 

According to Lindsay (1990), the use of this EID type of display has certain advantages. The display takes 

advantage of human pattern recognition. However, unlike simpler configural displays, the underlying pattern is 

related to plant thermodynamics. The display also integrates important information together in one display, which 

gives a significant overview of plant's performance. A properly trained operator can use pattern recognition to 

detect plant disturbances by distortions in the pattern. 

The effects of the Rankine cycle display on diagnosing events have been investigated (Moray et al., 1993; Moray et 

al., 1994; and Vicente et al., 1996). The Rankine display was compared with single-parameter, linear-meter 

displays and a display that included both single-parameter meter display (Figure 5.12) and a pressure vs. 

temperature plot. The study used three groups of 42 participants each: undergraduate engineering students, 

graduate engineering students, and professional NPP operators. 

The procedure required the participants to monitor nine transients for two to four minutes on a desktop VDU. The 

data for the transients were obtained from a PWR training simulator. Each participant used only one type of 

display. Following each transient, the participants were given either a quantitative recall task (e.g., "what is the 

value of parameter x"), or a qualitative recall task (e.g., "did the primary coolant remain sub-cooled for the 

duration of the trial?"), and asked to indicate whether something was abnormal. If something abnormal did occur, 

they were asked to make a diagnosis. 

For accuracy of detection, a significantly greater percent correct was obtained with the Rankine display in 

comparison to the other two. In addition, the NPP operators were better at detection than the other two groups. 

The Rankine display also led to more accurate diagnoses than the others, and operators were more accurate than 

the other groups. 

The Rankine display also led to better qualitative recall, but the difference was not significant. Quantitative recall 

was better with the meter display. The authors concluded that diagnosis is more associated with qualitative

5 -37 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Figure 5.12 Simulated Analog Meter Display (Source: Reprinted, with minor modification, with permission from Vicente, K., Moray, N., Lee, J., Rasmussen, J., Jones, B., Brock, R., and Djemil, T., "Evaluation of a Rankine Cycle Display for 

Nuclear Power Plant Monitoring and Diagnosis," Human Factors, 38(3), 1996. Copyright 1996 by the 

Human Factors and Ergonomics Society. All rights reserved.) 

information rather than quantitative information. The Rankine display was better at supporting the diagnostic 

performance of experts. 

It is interesting that, despite the performance effects noted above, the operators did not like the Rankine display because they were unfamiliar with it. They also claimed that they missed the alarm systems that they normally use 

for diagnosis. In their actual plants, the alarms are the first indication of a disturbance, and their pattern typically 

provides information about the nature of a disturbance. 

The authors suggested that failures in instrumentation can affect performance and interpretation of the display. Figure 5.13 shows two displays: one representing a real LOCA and the other a failed instrument. 

While only one of thirty-five instrumentation points failed, the display was very difficult to interpret. Moray et al. 

(1994) noted that 

The calculations involved in coupling information from many sources to produce the DPIs [direct perception interfaces] are extremely vulnerable to certain classes of failures. Little or no research exists in this aspect of DPIs, and is urgently needed. The real advantages of such displays during normal operation and during many classes of

NUREG/CR-6633 5-38
5 DEVELOPMENT OF TECHNICAL BASIS

, III 2P

(w Rl 

(a) Rankine cycle display for loss of coolant accident

f / I / / 

•" / / I i

I . r t I

*4 9 

(b) Rankine cycle display for failed sensor transient 

Figure 5.13 Rankine Cycle Display of Abnormalities 

(Source: Reprinted with permission from Moray, N., Lee, J., Vicente, K., Jones, B., and Rasmussen, J., "A 

Direct Perception Interface for Nuclear Power Plants," Proceedings of the Human Factors and Ergonomics 

Society 38th Annual Meeting, 1994. Copyright 1994 by the Human Factors and Ergonomics .Society. All 

rights reserved.) 

transients may be more than offset if they collapse under other classes of abnormalities. An understanding of the 

failure modes of DPIs (as distinct from the failure modes of the plant itself) is as necessary as an understanding of 

their design for normal conditions. Existence proofs of interface designs are no substitute for full empirical 

evaluations, and yet very few advanced DPIs have been exhaustively evaluated over a wide range of abnormal 

conditions. It is clear from our results that DPIs and/or ecological interfaces can fail in catastrophic ways, and it is 

probable that such failures may be particularly dangerous in large richly coupled systems. (p. 485)

NUREG/CR-6633

I I

5-39
5 DEVELOPMENT OF TECHNICAL BASIS

To briefly summarize the literature for the Rankine cycle display, it clearly reflects the principles of EID and has 

been used in designs of new power plants (see also the development of Toshiba HSI, below) and by professional 

operators (see the EBR II application). There is currently only one empirical evaluation of the display, which 

despite some negative operator comments, was associated with better performance of detection and diagnosis than 

other types. 

Despite the success of the experiment in beginning to address a real research need, there may be several limitations 

to the study. The first issue is generalization of the findings as representative of the strengths and weaknesses of 

the design approaches they intended to represent. There are two problems with the study. First, if the research is 

to provide a valid comparison of an EID approach and an alternative, the alternative approach to identifying 

information requirements was not clearly defined, and the alternative displays were not well designed. Second, 

there is a confound between information presented and display representation or format. That is, any difference 

between the alternative experimental conditions may be due to the information requirements analysis (EID vs. 

other) or to the different format (graphical display vs. single meter displays). Contrasting the Rankine-cycle 

display with a set of 35 individual analog meter displays (shown in Figure 5.12) is a biased comparison - even 

when augmented with a pressure-temperature plot. 

The second issue is identification of information requirements. What cells of the abstraction hierarchy does the 

display represent? What information needs is this display meeting? What analysis, ecological or otherwise, led to 

the selection of the Rankine cycle as an appropriate mental model for operators to conduct their tasks? Are 

operators sufficiently familiar with thinking of the plant on these terms to effectively use the display? Another way 

to approach the question of information requirements is noted below in item (1) under General Issues to Consider 

Relevant to EIDs. 

The third issue is generalization of the findings to other tasks, both complex and routine. The study was not a fully 

interactive dynamic simulation and operators did not have the full HSI available to them. Thus, potential 

interactive effects with the Rankine display in the context of other plant HSIs could not be evaluated; nor could its 

effects on other operational tasks. 

A fourth issue relates to proper interpretation the experiments' results. It was noted that the operators commented 

that they did not like the Rankine EID display and would have preferred to have some of their normal HSI 

available, namely the alarm system. Perhaps they would have preferred to have their entire display system rather 

than the EID display, and would have done better with it. In fact, given the low absolute performance values (78% 

for detection and 45% for diagnosis) recorded by the experienced operators, they may have worked better than this 

with their typical HSI suite rather than with the Rankine display. 

General Issues to Consider Relevant to EIDs 

This experiment, and particularly the Rankine display, show a few important points to consider that are relevant to 

the design and implementation of EID displays. Where appropriate, the more generic aspects of these issues were 

included in Section 5.6, General Summary and Issues. 

(1) In developing a higher-level display, one needs a clear definition of what higher-level functions and goals are 

to be addressed. As an example, are you addressing the goal of safety or electric production, or both? For the 

goal of safety, the NRC established five critical safety functions needed to achieve overall safety in NUREG

1342 (NRC, 1988), namely, reactivity control, reactor core cooling and heat removal, RCS integrity, 

radioactivity control, and containment conditions. Tables 2 and 3 of the NUREG give sample parameters to be 

monitored to address these functions for both BWRs and PWRs.

NUREG/CR-6633 5 -40
5 DEVELOPMENT OF TECHNICAL BASIS

The experiment did not clearly define which goals or functions the EID was to address. Examining the five 

critical safety functions, one sees that three (reactivity control, radioactivity control, and containment 

conditions) are not addressed at all in the display. RCS integrity and reactor core cooling and heat removal 

are addressed somewhat, but all of the recommended parameters are not included. For example, reactor vessel 

level, core exit thermocouples, and residual heat removal (RHR) flow are not included. Also, the 

discrimination of some of the provided important parameters (reactor pressure, subcooling margin, and 

pressurizer level) is not fine enough to detect relatively small changes, and there does not appear to be any 

trending capability. 

Thus, while the Rankine display gives some higher level functional information, it does not appear to provide 

a sufficient amount, and not nearly as much as a typical NPP SPDS implementation. Perhaps a useful 

experiment would compare a typical SPDS with an improved EID display based on the Rankine cycle. The 

improvements would be to incorporate the five SPDS functions and the related important parameters. This 

certainly appears to be a feasible experiment. 

(2) For any higher-level display, where the information is sophisticated and a large amount of it is consolidated 

into one or a few graphics, training becomes very important. One needs to understand all of the aspects of the 

display and how it reacts to various operational transients, accidents, and instrument failures. Further, many 

of the items in the display are not labeled (e.g., TH and Tc) and would be burdensome to estimate unless 

operators were highly familiar with the display. 

(3) The effect of failures of instrumentation on EID displays was recognized as a potentially significant problem 

that needs to be addressed. There are several subissues associated with this problem. 

Can operators detect a failure of instrumentation? 

• Can failures of instruments result in representations that are interpreted by operators as real process 

failures; and, perhaps more importantly, can real process failures be misinterpreted as failures of 

instruments? 

• If operators detect a failure, should use of the display be suspended? 

• Since the display integrates many parameters into a single display, what is the effect on operations of its 

loss, and how effectively can operators switch to backup displays? 

(4) The integration of a new and significantly different EID into the remainder of the standard HSI of a CR is an 

important consideration that needs attention. 

(5) The issue of the operator's acceptance of a new and different type of display (such as an EID) also is 

important, as shown by the operator's comments during the experiment. One interesting idea was introducing 

the displays initially in training and then perhaps gradually introducing them into the plant. 

5.3.2.3 Toshiba Advanced BWR HSI 

The EID approach to interface design is guiding Toshiba's development of an intelligent interface for an advanced 

BWR information system. A series of papers (discussed below) described the development of the information 

system, giving examples of the displays, and discussing some validation experiments. This application of EID is 

interesting because it represents an effort to develop an entire NPP information system based on EID principles. 

However, while the importance of the evaluation of EID displays was identified, little information on the

5 -41 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Table 5. 3 Operator Cognitive Activity 

Level of Level of Aggregation 

Abstraction Plant System Equipment Component

Functional Purpose X X 

Abstract Function X X X 

Generalized Function X X

Physical Function I I X X 

effectiveness of the interface is available. 

The basis for the design is a cognitive task analysis reflecting the abstraction hierarchy (Itoh et al., 1990). As part 

of this effort, operators engaged in situation assessment, diagnosis, and response planning for simulated plant 

disturbances involving multifault scenarios. The scenarios were designed to elicit knowledge-based behavior. 

Operators were asked to verbalize their thought processes. The data were analyzed and sorted into the cells in the 

A-A matrix. Table 5.3 shows the envelope of cognitive activity. The correlation between the higher levels of 

abstraction and higher levels of aggregation is noteworthy (e.g., operators did not reason at the plant level in terms 

of physical form, nor did they reason about components at the functional level). 

The authors concluded that information reflecting these cells is needed, at a minimum, in the interface. At each 

level of abstraction, the following types of information were preliminarily identified as necessary: 

• Functional purpose - the goal of each function should be expressed in flow of mass, energy, and information. 

No information on equipment or components is necessary. 

* Abstract function - information should be presented on the flow and distribution of mass and energy (the 

relationship between origins and destinations of substance or energy flows). 

* Generalized function - information should be presented on the relationships between objects, input-output 

relationships, and information on the operating sequence. 

* Physical function - information should be centered around the properties of objects and can be categorized in 

terms of underlying physical processes. 

* Physical form - information should be provided on component status and appearance, and how it is accessed. 

For specific purposes, operators may need information on component settings and parts drawings. 

Examples of displays supporting some of these levels were presented elsewhere (Sakuma et al., 1990; Monta et al., 

1991). Monta et al. (1991) also discuss an intelligent "advisor" that can generate strategies and procedures after 

faults. 

NUREG/CR-6633 5-42

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Figure 5.14 EID Production Goal and Display 

(Source: Reprinted with permission from Monta, K., Takizawa, Y., Hottori, Y., Hayashi, T., Sato, N., Itoh, J., Sakumna, A. and Yoshikawa, E. (1991), "An Intelligent Man-Machine System for BWR Nuclear Power 

Plants," Proceedings of the AI91 Frontiers in Innovative Computing for the Nuclear Industry. LaGrange Park, IL: American Nuclear Society, pp. 383-392. Copyright 1991 by the Valley Litho, Idaho Falls, Idaho. 

All rights reserved.) 

Figure 5.14 is an example of a production goal display from Monta et al. (1991). The uppermost section of the 

display represents the flow of electrical production. Its center shows the process for converting the energy and 

includes a display based on the Rankine cycle. The lower part of the display depicts the mass balance for the 

reactor vessel and the condenser, showing inlet and outlet flows and water levels. 

This display was intended to be an externalized mental model of the process. Thus, workload should be reduced 

because operators do not have to recall these relationships from memory and mentally compute information related 

to higher-level goals. 

The display was tested using a full-scope, advanced boiling water reactor (ABWR) simulator (Monta et al., 1992). 

Six experienced crews of utility operators were presented with simulated plant disturbances. Subjective evaluations 

of the interface were obtained following each trial. The operators' evaluations were generally positive; however,

5 -43 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

Figure 5.15 Toshiba EID Display 

(Source: Reprinted from Itoh, J., Sakuma, A. and Monta, K., "An Ecological Interface for 

Supervisory Control of BWR Nuclear Power Plants," Control Engineering Practice, 3(2), 1995, 

Copyright 1995, with permission from Elsevier Science.) 

more detailed results were not given. Itoh et al. (1995) gave additional information on the evaluation of the 

Toshiba EID display (Figure 5.15). This EID was subjected to an empirical evaluation (Takizawa et al., 1994). 

Itoh et al. (1995) presented a typical comment from operators participating in the empirical evaluation of the 

display. "EID display would suit the novice operator training to build up his/her mental model. However, it is too 

crowded with information to easily understand the situation" (p. 238). This is not a very comforting result. 

Operators may have experienced difficulty understanding the plant situations shown in the displays. An 

independent examination of the display shows that many aspects are not self-evident and would probably require 

substantial training before operators could use it in operational situations. This also illustrates the importance of 

evaluating the relationship between the representation in the display and the operator's mental model of the 

domain, as well as its relationship to training. 

Itoh et al. (1995) also discuss the difficulties of evaluating this type of interface. The authors stated that 

"evaluation of the man-machine interface, especially for new designs such as EID, is one of the most important 

issues, since it should guarantee that the design objectives have been completely achieved before interface usage. 

Its importance as well as its difficulty increases for the present case, because the main objective is to support the 

operator's knowledge-based behavior to cope with unanticipated events" (p. 237). Methods are needed to evaluate 

the technical accuracy as well as the human factors and performance aspects of the design's use.

NUREG/CR-6633 5-44
5 DEVELOPMENT OF TECHNICAL BASIS

In summary, Toshiba has put substantial resources into developing and testing an EID-based display system for the 

ABWR and believes that it will provide benefits over traditional displays. Since the ABWR has now been 

operating for some time, operating experience on the EID should soon become available. This would provide 

valuable information on its use in real-life situations. 

5.3.2.4 Other EID Studies 

This section discusses nine additional studies of EID-type displays that addressed a variety of different 

applications. 

Edlund and Lewis (1994) compared five types of displays for controlling a simulation of a simplified steam plant. 

Eighteen participants were given the task to maintain boiler level and turbine speed in response to variations in 

load. When a fault occurred, they were to stop the simulation and make a diagnosis. The five displays (see Figure 

5.16) used in the test were 

• Dials display - Traditional gages were used to present process parameters. No information on constraints or 

relationships between parameters was given (see Fig. 5.16a). 

Fluid-tanks display - This display represents boiler and turbine levels. Feedwater, fuel, and make-up controls 

are shown as valves, which control flow in the system. Pipes and valves were animated to depict flow 

characteristics. Unlike the mimic display, the effects of parameters were not displayed as functions of 

commanded pumping rates (e.g., feedwater pump rate). The fluid-tank display is closest to the EID-type 

displays discussed above (see Fig. 5.16b). 

• Seesaw display - The process is displayed as a seesaw with balances representing the fuel, feedwater, and 

makeup pump rates. The seesaw display is interesting because it incorrectly represents the dynamics of the 

process (see Fig. 5.16c). 

Mimic display - The same process parameters were presented as in the dials display, but the mimic provided a 

context revealing the relationships between them and their constraints (such as direction of flow). Its 

disadvantage was that the operator's task is to control states rather than individual components. This 

information is not given and the operator must mentally determine how to integrate it to manage the states 

(see Fig. 5.16d). 

• Object display - The process parameters are integrated into a single-object display (see Fig. 5.16e). 

The general results were that the best performance was obtained with the fluid-tank, dial, and mimic displays. The 

integrated object display fared more poorly. The authors hypothesized that this was because it presents state 

information with no context or process information. The seesaw display led to the worst performance, which 

emphasizes the importance of accurately mapping the features of the display to the dynamics they are intended to 

represent. 

In another study, an EID display was compared with a mimic display for a chip refiner in a pulp factory (Brehmer 

et al., 1995). Fifteen process operators used the displays in a process simulation. The operators were divided into 

three groups: P&ID, EID, and EID with special training. The EID display was associated with better performance, 

and interestingly, no special training-was found to be necessary.

5 -45 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

4: :4.E 

E3

(a) Dials display

I¶ n1

(b) Fluid-Tanks display 

-n _

SAn 

Wc Seesaw display

W -- I"-----"a 

(d) Mi~mic display

RZ~ 2~ 

(e) Object display

Figure 5.16 Edlund and Lewis Displays ' 

(Source: Reprinted with permission from Edlund, C. and Lewis, M., "Comparing Ecologically 

Constrained and Conventional Displays in Control of a Simple Steam Plant," Proceedings of the 

Human Factors and Ergonomics Society 38th Annual Meeting, 1994. Copyright 1994 by the Human 

Factors and Ergonomics Society. All rights reserved.)

NUREG/CR-6633 5 -46
5 DEVELOPMENT OF TECHNICAL BASIS

Similarly, Meshkati et al. (1994) evaluated the effectiveness of an ecological interface for handling process control 

disturbances. They found that the interface led to significantly more accurate diagnosis of events and faster 

response than a computer emulation of the current traditional console. 

Reising and Sanderson (1996) conducted a detailed analysis of a pasteurization plant using the abstraction 

hierarchy. They found the following important factors for EID: 

* Limitations in sensor technology may limit the designers' ability to represent, in displays, the information 

found in the analysis of the abstraction hierarchy. 

• The displays of higher order variables and their relationships are based on computations using lower-level 

information, such as volume and temperature. The information displayed will be limited by the quality of the 

lower-level information. 

* Sensor failures or errors may affect the form and content of the display. 

One important contribution of a thorough analysis of abstraction hierarchy is that it can reveal not only the 

operators' information needs, but what sensors are needed and where in the plant they should be placed. In fact, 

the authors state that EID is "in a position to bring about a profoundly user-centered approach to system design in 

which the ultimate information needs of human controllers drive the engineering agenda of sensors and 

instrumentation in a feed-forward manner" (p. 296). 

Based on their experience in applying the process, Reising and Sanderson (1996) concluded that building an 

abstraction hierarchy is an iterative process that required expertise in thinking in terms of abstraction hierarchies 

and in conducting the analyses. The knowledge of domain experts also is necessary. In addition, keeping track of 

the links in the hierarchy was very difficult and tools to support such analyses are needed. Lack of such support 

limits their use. 

Shively (1995) examined the use of ecological approaches to 911 dispatcher interfaces. Ten experienced 

dispatchers completed eight trials, half with a traditional text display and half with a graphic EID display. Their 

task was to assign the most appropriate resource to each incident. Time and ratings of situation awareness, 

workload, and preference were collected. Response times did not differ; however, the operators were very 

experienced with the text display and had little training on the EID display. There were trends toward improved 

situation awareness and reduced workload with the EID display. The EID display was preferred as well. 

In the medical domain, an EID display for hemodynamic monitoring was compared with a conventional display 

(Effken et al., 1992). The display provides clinical patient information, including current state and goal state. 

Data to determine the relevant information were obtained from expert clinicians. From this information, an 

integrated object display was developed. The display represents pressures by interconnected balloons for cardiac, 

venous, and arterial compartments, and cardiac flow as a bellows. This display reveals the functional relationship 

of the hemodynamic system. Traditionally, this information is monitored using strip chart displays of individual 

parameters. 

Thirteen participants at three experience levels used the displays to handle simulated clinical events using a 

within-subjects design. Performance measures included time to initiate treatment, time to achieve target range, 

percent of time in target range, and the number of drugs, drug changes, and combined drug and dose changes. 

Compared to the strip chart display, the EID display performed better, showing the following: higher percent of 

problems corrected, greater percent time in target range, fewer drugs, drug changes, and combined drug and dose 

changes. There was an interaction with time to initiate treatment, such that the more experienced participants took

5 -47 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

longer than novices. Ten of the thirteen participants preferred the EID display as well. Overall, the results showed 

that the EID display simplified the clinical problem by revealing how pressures and flows are related. 

Xu et al. (1996) developed an EID interface for a hypertext application that provided ergonomic principles for 

workstation design. The interface was compared with a traditional interface using a hierarchical structure. An 

experiment was conducted comparing the performance of the two displays supporting search tasks. The EID 

interface was found to be superior to the traditional interface. The report did not give additional details on the 

study design or the results. 

The application of EID principles to other industrial domains was discussed in the literature as well, e.g., discrete 

manufacturing systems (Moray, 1995) and network scheduling (Edlund et al., 1995). However, no specific 

evaluations are given. 

Lee (1995) discusses the proliferation of electronic HSIs aboard ships and concludes that they may offer many 

opportunities to detract from the safety and efficiency of operations. He noted that 

Left unchecked, the ongoing changes in maritime technology have the potential to overwhelm mariners with 

information and distance them from critical tasks. To avoid this risk, careful attention must be paid to interface 

design. EID promises to be a valuable approach to integrate potentially disparate information in a way that enables 

mariners to better understand the state of the ship. 

Lee found several areas where EID principles can be applied to improve performance, including remote engine 

room monitoring, navigation and collision avoidance, and hull stress management. 

In general, these nine studies showed positive results for using EID-type displays, including better operator 

performance and a subjective operator preference for the EIDs. 

5.3.3 Conclusions 

Flach (1990) stated that "...there appears to be a convergence of opinions that an ecological approach is most 

appropriate for the problems of human-machine systems." Others suggested that an ecological approach is 

important to developing interface designs that support reduction in errors (Reason, 1990). The BNL assessment of 

the state of EIDs is summarized below. While we concur that the EID approach is promising, there are several 

technical and research issues to be addressed. 

Four aspects of the EID approach were addressed to varying extents in the many studies reviewed. They are the 

following: 

* specification of information requirements 

* organization of information among the levels of an abstraction hierarchy 

* development of innovative displays to show functionality 

* addition of analytical redundancy 

Each appears to have some performance-enhancing features.

NUREG/CR-6633 5 -48
5 DEVELOPMENT OF TECHNICAL BASIS

For the specification of information requirements, the EID approach may add a design method that goes beyond 

the standard systems engineering methods. In the systems engineering approach, as discussed in NUREG-071 1, 

-information requirements are derived from function and task decomposition and analysis. The use of the A-A 

matrix provides a framework for identifying information needs that goes beyond the systems engineering approach, 

by providing a structure that specifies levels of functional decomposition. Bennett et al. (1997) consider 

"abstraction hierarchy analysis and task analysis (traditional or cognitive) as complementary processes that are 

necessary for the development of effective displays" (p. 684). 

For organizing information among the levels of an abstraction hierarchy, interfaces based on EID and the 

abstraction hierarchy and EID principles provide better support for knowledge-based behavior than interfaces 

based on physical variables alone. By using EID goals, the relevant system requirements, constraints, and 

parameters can be evaluated and represented. Thus, the displays can provide an appropriate "mental model" of the 

system, which is external to the often "buggy" models of the operators. 

The development of innovative displays to show functionality may not be considered purely a feature of E1Ds, but 

once the levels of functionality that should be displayed are specified, the designers should develop appropriate 

displays. Such displays will probably not be found in the design engineer's tool box of past successful applications, 

and thus, will likely need to be developed for the particular application in question. Some of the variable results 

shown in the experiments may be due to either very good or perhaps not so good displays used to represent the 

functionality as required. This aspect of EIDs may be more of an art than a science. 

The addition of analytical redundancy is another feature of EIDs that is not always applied, but which appears to be 

valuable. This is a particularly effective method of verifying that higher-level functions and goals are, in fact, 

being met, and if they are not, determining why and what to do about it. 

To the extent that they have been tested, EIDs have shown beneficial effects. These effects may be more for fault 

diagnosis and disturbance handling rather than normal operations, but these aspects are important to safety. 

Several issues remain to be addressed for EID. They are organized below into ten general headings. Research 

issues are discussed in greater detail in Section 7. 

(I) Lack of a Specific Process 

Reising and Sanderson (1996) suggest that "an important practical problem is that at present, there are very few 

published examples of AHs [abstraction hierarchies] that are sufficiently detailed to really derive the process of 

design" (p. 293). Developing a well-defined process for conducting an analysis using the abstraction hierarchy is 

important to its broader application to the design process. 

(2) Task and Temporal Considerations 

The A-A matrix and the EID approach address the plant at a functional level rather than on a task or temporal 

basis. The proper role of task-based information in display design and how it is integrated into the EID approach 

needs to be addressed (e.g., see Heslinga and Herbert, 1995). Carroll and Olson (1988) state that "Optimizing a 

design with respect to task-oriented analysis will not necessarily include any of the design considerations that 

would be indicated by optimizing the presentation of a good mental model." (p. 57). Figure 5.15 shows an 

example of a task-based EID display. used for the startup of a reactor (Itoh, Sakuma, and Monta, 1995). 

Much of the operators' goal-directed activity is centered around temporal constraints, which is one reason 

operators like trend displays. The operator's tasks unfold along a temporal continuum with some tasks being

5 -49 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

performed in parallel and some sequentially. This temporal dimension to operations may be lacking in the A-A 

analysis, except insofar as time is reflected in higher-level information, such as rates. 

Additionally, the EID approach seems to be built around functional decomposition of means-ends relationships. 

How well such an approach will support common tasks or the handling of disturbances that are highly proceduralized needs to be determined. While it is important to be able to deal with unanticipated and unplanned 

events, still most of the operator's activities are well planned and the information system needs to support these 

activities as well. 

(3) Information Volume and Density 

Due to the extensive analytical process used for EIDs, the process, including the A-A matrix, may identify more 

information than can practically be displayed. Also, too many pages may be required to satisfy EID information 

requirements (see also discussion of density in Section 3.5). Potentially negative impacts of designing displays using the abstraction hierarchy are the potential proliferation in the number of pages and the increase in the 

density of information on each page. 

(4) Integration of EIID Displays into the Remainder of the HSI 

Integrating a new and significantly different ElI) display into the remainder of the standard HSI of a CR is an 

important consideration to address. 

(5) Critical Testing and Evaluation of EID Concepts 

There has not been much thorough research that carefully assesses the various aspects of EID, e.g., information 

requirements, effect of organization of information along functional lines, display representation, and use of 

analytical redundancy. Studies tend to confound these characteristics or give weak assessments of the contribution 

of the various aspects of EID concepts. 

Successfully handling unplanned or unanticipated events with EIDs, under actual operational conditions in 

complex systems, has not been demonstrated. 

Further research also is necessary to more clearly identify which cells of the A-A matrix are important to 

operations. 

(6) Evaluation of Operating Experience 

For EID displays that were installed in operating facilities, there needs to be a thorough assessment of the 

operating experience and how it applies to the EID. Actual application of this approach to NPP HSI design 

includes the Toshiba ABWR CR (discussed earlier), a NPP feedwater system (Dinadis and Vicente, 1996), and the 

EBRII (Lindsey, 1990). 

(7) Internal vs. External Mental Models 

Carroll and Olson (1988) note, "If the interface suggests or reflects an appropriate model, then the user could 

conceivably learn it with less guidance and perform with fewer errors. The question is: What should the model 

be?" (p. 56). Compatibility of information representation and operator mental models is characteristic of "user

centered approaches to design and design review." For example, the high-level design review principle of User 

Model Compatibility from NUREG-0700, Rev. 1 states as follows:

NUREG/CR-6633 5 -50
5 DEVELOPMENT OF TECHNICAL BASIS

All aspects of the system should be consistent with the users' mental models (understanding and expectations 

about how the system behaves as developed through training, use of procedures, and experience). All aspects of 

the system also should be consistent with established conventions (i.e., expressed in customary, commonplace, 

useful and functional terms, rather than abstract, unusual or arbitrary forms, or in forms requiring 

interpretation). (pp. A2-3) 

Vicente (1990, 1997) and others (Flach and Dominguez, 1995) criticized the "user-centered perspectives" and 

disagreed with the notion that the plant representations provided by displays must be compatible with the 

operator's mental model of the plant. Vicente argues that is because the models are incomplete and sometimes 

incorrect (as was discussed in Section 5.1). Thus, Vicente asks "what good is it having the computer be compatible 

with the agent's model if that model does not correspond to reality?" (1990, p. 496). 

An example of the failures of user-centered design is the use of operators to support interface design. Vicente 

(1997) gives an example of a nuclear plant HSI that was designed with the aid of an "exceptional operator." When 

the design was shown to other operators, it was not accepted. Vicente concluded that "no two operators seemed to 

have the same mental model of the plant" and that "the control room in question had to be redesigned to reflect 

better the way in which the plant actually worked" (p. 3). Instead Vicente argues that process control operators 

exist in a correspondence-driven domain where there is an external reality in which the operator must exist. This 

is in contrast to a coherence-driven domain, where the human and the computer system represent a closed world 

and need not reflect external constraints. Rather than reflect the operator's model, the display should reflect the 

actual constraints of the process being controlled. Vicente refers to this as the "ecological compatibility principle." 

We feel the criticism of the user-centered perspective takes the principles out of context. HSI design is part of a 

systems engineering process that attempts to capture the functions, relationships, and constraints as part of 

functional requirements analysis and task analysis. Further, the systems approach derives training requirements 

that are meant to provide operators with a concept of operations that includes an understanding of the system. 

Thus, for example, the NUREG-0700 principle identified above is part of a greater process described inNUREG

0711. This context was recognized by Pejtersen and Rasmussen (10997) as well, who identify one of the questions 

to be addressed in the evaluation of the complex system as "Does the presentation match the user's cognitive 

processes and mental models?" This is not the only question, and Pejtersen and Rasmussen suggest that questions 

on the content of the information in representations are better addressed by analytical evaluation methods, while 

questions addressing form are better addressed using empirical evaluation methods. In fact, other authors view 

EID as a movement toward user-centered design. Reising and Sanderson (1996) stated that EID is "in a position to 

bring about a profoundly user-centered approach to system design in which the ultimate information needs of 

human controllers drive the engineering agenda of sensors and instrumentation in a feed forward manner" (p. 

296). 

The point of user-centered principles (in their full design context) is that once the systems analyses are completed 

and operators are trained, the HSI should be consistent with human cognitive and physical constraints and should 

be compatible with the training and experience used to develop operators' expertise with the system. 

Thus, we do not disagree with Vicente's ecological compatibility principle. The HSI must reflect the reality of the 

system being operated. Display design should reflect the systems engineering process. If it does, the process 

should not lead to displays based on faulty mental models of individuals. For the NPP CR design discussed above, 

the failure is not in the user-centered aspects of the design, but in the design process at large. Operators' input 

should be sought after the systems analyses to help develop representations that link display forms and functions to 

the way the plant works. The design process should have allowed for display requirements based on systems 

analyses to have been developed and for prototyping and testing alternatives of design early in the process (see

5 -51 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

NUREG-07 11). HSIs should not be developed based on the input of a single operator without the checks and 

balances offered by the full design process. 

In fact, the representation of a system as complex as a NPP will ultimately be based on many sources of human 

expertise, including operators, engineers, and physicists. All of these experts are likely to have some gaps in their 

mental models. 

An extreme expression of the ecological compatibility principle was reflected in a paper by Lindsey and Staffon 

(1988; from Hansen, 1995). They developed a modified Rankine cycle display and concluded that operators no 

longer needed to construct a mental model of the process because the display accurately depicts the plant state. 

While displays should certainly provide accurate representations of the systems they represent, the relationship between them also must be consistent with operator models of the system. Consider the implications of violating a 

principle, such as the User-Model Compatibility. There are ample data to show that when displays are not 

consistent with operator models and expectations are not well integrated into task environments, they will not be 

used, or even worse, they will be misunderstood. 

Operators interpret information using their mental models, which provide meaning and coherence to information. 

Ignoring this information processing reality is as serious as ignoring the constraints of the system. Both need to be 

equally addressed designing an effective HSI. Our point is consistent with the notion of the cognitive system triad 

of Woods and Roth (1988), which includes the world to be acted on, the agents who will act, and the representation 

through which the agent acts. All three are essential parts of the overall system. 

(8) Training and Qualification Implications 

Christoffersen et al. (1995) concluded that "to experience the benefits of EID, it seems likely that operators need to 

be trained to think ftmctionally rather than procedurally. It would seem that this would require a fundamental shift 

in NPP operation philosophy...it may be that operators have to possess certain types of cognitive characteristics that 

may not be considered in the traditional selection process in the nuclear industry" (p. 143). Concern also was 

expressed that EID interfaces may inhibit long-term learning and retention (Wickens, 1992). 

Also, for any such display, where the information is sophisticated and a large amount is consolidated into one or a 

few figures, the requirements for training become very important. One needs to understand all of the aspects of the 

display and how it reacts to various operational transients, accidents, and failure of instruments. Finally, the long

term effects of EID-type displays on operators' performance and strategies are unknown. 

(9) Operator Acceptance 

The issue of operator acceptance of a new and different type of display (such as an EID display) also is important, 

as shown by the operators' comments during experiments. One interesting idea that was noted during the Rankine 

cycle experiments is introducing the displays initially in training, and then perhaps gradually introducing them 

into the plant. 

(10) Effect of Instrumentation Failures on Interpreting Displays 

The effects of failures of instrumentation on EID displays is a potentially significant problem that needs to be 

addressed. There are several subissues associated with this problem: 

Can a failure of instrumentation be detected by operators?

NUREG/CR-6633 5 -52
5 DEVELOPMENT OF TECHNICAL BASIS

* Can failures result in representations that are interpreted by operators as real process failures, and perhaps 

more importantly, can real process failures be misinterpreted as instrument failures? 

* If operators detect a failure, should they suspend use of the display? 

* Since the display integrates many parameters into a single display, what is the effect of its loss on operations, 

and how effectively can operators switch to backup displays? 

Many designers have implemented advanced features that address this concern to some extent. For example, the 

concepts of redundancy and diversity in instrumentation, coupled with automatic parameter validation and 

appropriate notification of operators when instruments fail their validation, have been shown to be powerful. 

5.4 Research on Information Representation 

As discussed in the previous section, advances in the analysis of information requirements have led to potentially 

enhanced methods for identifying levels of information about a plant that vary along A-A dimensions. However, it 

is not adequate to identify the information needed and simply make it available in the CR (Woods and Elias, 1988; 

Woods and Roth, 1988). Data, in and of themselves, have no inherent meaning. Meaning is acquired when data 

are put in context. Thus, "data availability" is distinguished from "information extraction" Further, this context 

should be reflected in the representation of the information. An intelligent display system would "create and 

manipulate the representation through which the domain problem solver sees the world" (Woods and Elias, 1988, 

p. 1352). Representation addresses the manner in which the information needed by the operator is displayed. 

Conventional CRs present individual parameters on analog displays as separate pieces of information. The 

displays are usually grouped logically. However, integration and interpretation of data is mainly accomplished by 

the operators based on training and experience. When computer-based display technology was first introduced into 

CRs, the displays tended to follow well-known information presentation formats showing single parameter values, 

such as digital displays, multiple parameter values, such as bar charts, and simple relationships between variables, 

such as trend graphs vs. time. The first attempts at making displays ecological linked individual parameter 

displays with mimic lines to reveal their structure with respect to the plant and process they represented. While 

this approach was a common modification of a NPP CR following the TMI event, the tradition has continued in the 

first generation of digital CRs, which make extensive use of displays to show plant components and systems at low 

levels of abstraction; i.e., individual components may be represented by icons. Such displays can be based on a 

functional layout of the plant, using the format of simplified piping and instrumentation diagrams (P&IDs). 

Comparisons between some of the above simple representational formats revealed differences in performance 

which appear to be task specific. For example, Hollands and Spence (1992) found that change was judged more 

quickly and accurately with line and bar graphs than with tiered bar graphs and pie charts, while the opposite 

results were obtained for proportion judgements. Research examining the effects of basic display representations 

on specific tasks is still the subject of ongoing research (e.g., Gillan et al., 1998; Hink et al., 1996; Hollands and 

Dyre, 1997; Hollands and Spence, 1997; Meyer, 1997; Meyer, Gopher, and Levy, 1997; Meyer, Shinar, and Leiser, 

1997; Shamo et al., 1996). 

The guidance on design review contained in NUREG-0700 addresses these basic representational formats fairly 

well. Representational formats based on graphical features that are tied to perceptual processes, such as polar 

graphic displays, and formats using graphic forms that reveal higher-level plant information, such as Rankine 

cycle displays are not addressed in sufficient detail. Thus, the main focus of this section is on representation 

formats that attempt to reveal plant information by making better use of human perceptual and cognitive processes.

5 -53 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

The term representational aiding has been used to refer to using the capability of computer-generated displays to 

create novel representations of a system that can be manipulated to support operator's decision making (Woods and 

Roth, 1988). 

Sections 5.4.1 and 5.4.2, respectively, discuss the theory and research directed to representational aiding. Section 

5.4.3 gives the conclusions. 

5.4.1 Theory of Representational Aiding 

Direct Perception and Representational Mappini. 

As noted earlier, NPPs generate a great quantity of data that, at times, can overload the operator. To lower the 

workload associated with extracting meaningful information from data, efforts were undertaken to support the task 

of integrating data into more meaningful units of information. Within the context of representational aiding, the 

goal of display design is "...to map the domain semantics (low-level data, high-level constraints, and relevant 

performance goals) into the appearance and dynamic behavior of a graphic display so that this information is 

readily available (easily extracted or decoded by the user)" (Bennett et al., 1993, p. 73). 

The success of representational aids is a function of the characteristics of the system, the tasks to be performed, and 

human information processing (Bennett, 1992). Woods and Roth (1988) suggested that displays can be 

conceptualized as involving a cognitive system triad that includes the world to be acted on, the agent who will act 

on the world, and the representation through which the agent acts on the world. All three are essential parts of the 

overall system. The representation involves the following two types of mapping (Bennett et al., 1997): 

Correspondence map between the properties and characteristics of the system to be represented and the 

features in the representation (how well the display communicates meaningful information about the plant that 

is needed by operators) 

* Coherence map between the features in the representation and the physical and cognitive characteristics of the 

operator (how comprehensible the representation is to the operator) 

The systems engineering approach (see Section 5.1) and abstraction hierarchy (see Section 5.3) provide approaches that address half of the correspondence issue - the identification of system functions and constraints that need to be 

represented. 

In the previous section, EIDs were discussed in terms of Gibson's ecological psychology and its focus on the work 

domain or system to be represented. Two aspects of Gibson's approach to perception are important to representing information about the system: the concepts of direct perception and affordances (Flach, 1989; Flach and Hancock, 

1992). Direct perception means that the information presented in the display has immediate meaning within the 

context of the operator's current information needs, including task requirements and system status. There is little 

need for the operator to analyze and interpret direct perception displays (see Flach, 1988, for a discussion of the 

use of the term "direct" in HSI design). Affordances are the features of an interface that suggest an interpretation 

or a response by the crew. They are a product of the interaction between the crew and the interface within the 

context of the ongoing tasks of the crew. This is an extension of what Gibson meant by direct perception of the 

physical features of the perceptual environment. Direct perception is made possible by a one-to-one mapping of 

affordances and domain constraints.

NUREG/CR-6633 5-54

I
5 DEVELOPMENT OF TECHNICAL BASIS

Thus, the degree to which a display supports operator's performance is a function of both the quality of the 

mapping between the emergent display features perceived by the operator and the relationships and constraints of 

the system. Bennett et al. (1997) stated that 

Problem solving can be critically influenced by the nature of visual representations. Building effective 

representations requires designers to go beyond the simple psychophysical questions of data availability to the more 

complex questions of information availability, where information refers to the specification of domain constraints and 

boundary conditions. This specification depends both on the mapping from display to human (i.e., coherence) and on 

the mapping from display to domain (i.e., correspondence). (p. 682) 

Efforts to create direct links between graphical forms and operator actions are discussed below (these links are the 

affordances). Two basic approaches are discussed: the information relationships approach and the proximity

compatibility approach. The primary focus of the information relationships approach is on the mapping between 

display characteristics and characteristics of the system being represented, while the primary focus of the 

proximity-compatibility approach is the mapping between display characteristics and the characteristics of the 

tasks being performed. 

Information Relationships: Representing Separable, Integral, and Configural Relationships 

One attempt to accomplish the objective of direct perception is based on the mapping of low-level perceptual 

processes to the relationships between plant information. Bennett and Flach (1992) defined three potential 

relationships between data that can be represented in displays: separable, integral, and configural. Separable 

relationships are those that can be showed by individual data points. They are characterized by a lack of 

interaction among the data dimensions; i.e., each dimension retains its unique perceptual identity. Integral 

relationships are defined by a strong interaction among data dimensions such that the unique identities of 

individual dimensions are lost. When such a relationship exists, a change in one dimension necessarily produces a 

change in a second dimension. Bennett and Flach used the example of color perception, which is a function of the 

dimensions of hue and brightness. In color perception, the perception of each individual dimension is lost in the 

integration. Configural relationships refer to an intermediate level of interaction between dimensions, where each 

maintains its unique identity, but new emergent properties are created as a result of the interaction between the 

dimensions. 

These three types of relationships are associated with three generic types of displays. An important goal in display 

design is to provide the most appropriate type based on how the operator must use information about the domain. 

Separable displays are representations where each process parameter is presented individually and no relationships 

between the parameters are provided by the representation itself. The key aspect of separable displays is not that 

individual parameters are presented, but that no interaction or relationship between them is perceived. There may 

be, in fact, a relationship between the parameters, but the separable displays do not show it. The studies on the 

Rankine cycle display, discussed in Section 5.3, used a separable display - a set of individual meters - as a basis of 

comparison (see Figure 5.12). 

Integral displays show the integration of information in such a way that the individual parameters used to generate 

the display are not represented in it. A display that provides information on system status by the appearance of an 

icon is an example. The icon may change appearance based on the computation of lower-level parameters, but 

these parameters are not presented. When decisions are based on the integrated information, the concept of 

integral display suggests that information processing is supported by integrating information into a single object 

(Kahneman and Triesman, 1984). Integral displays are thought to support direct perception by capitalizing on the

5 -55 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

abilities of human pattern recognition, and thereby reduce demand on working memory associated with 

simultaneous consideration of many individual variables. 

Configural displays provide a combination of the other two types. Information about the basic parameters is 

available; however, information on their relationships becomes evident as features emerge from the representation. 

An emergent feature is a "high-level, global perceptual feature that is produced by the interactions among 

individual parts or graphical elements of a display (e.g., lines, contours, and shapes)" (Bennett et al., 1993, p. 73). 

Configural displays often use simple graphic forms, such as a polygon. Information that could be presented by 

separate display formats is integrated into a single format in which each of the separate pieces of information is 

represented, for example, by the distance of a polygon's vertex from its center. In addition, the geometric shape of 

the polygon provides a high-level summary (the emergent feature) (Buttigieg et al., 1988; Sanderson et al., 1989). 

The SPDS display described in Section 4 is an example of a configural display (Woods et al., 1981). By glancing 

at the shape of the polygon, the operator can readily see whether the plant is in a safe or unsafe mode. However, 

information about the individual parameter values also is still available. Such displays are thought to enhance 

parallel processing (lowering cognitive workload), enable operators to better understand the relationships between 

display elements, and ultimately lead to a more rapid and accurate awareness of the situation (Bennett and Flach, 

1992; Flach and Bennett, 1992). The extraction of information for high-level task requirements is supported by the 

emergent features produced by a configural display. 

Another example of a configural display is the Rankine cycle graphic discussed in Section 5.3 (see Figure 5.11). 

The form of the graphic indicates overall performance, and the operator can obtain specific parameter information 

if needed. However, in this Rankine display, the parameters would have to be estimated from the figure, and the 

scales are large. This almost makes it into an integral display (without parameter values), since the operators 

would really need to refer to separate indicators to obtain useful specific information on pressure, temperature, or 

levels. 

Emergent features can be formed by purely separable displays; i.e., where no specific graphic is incorporated into 

the display (Wickens and Carswell, 1995). A systematic pattern formed by a series of bars on a bar chart or the 

parallel appearance of a set of analog meters may form an emergent feature (Figure 5.17). Operators may, for 

example, be able to glance at a row of meters and be able to determine that all systems are properly functioning 

because the needles are all pointed in the same direction. Similarly, the blackboard (or darkboard) concept in 

alarm system design is based on an emergent feature (O'Hara et al., 1994). The darkboard concept means that 

when all alarmed parameters are within normal operating range, no lighted alarms can be seen (on a conventional 

board) or no messages appear on alarm VDUs. When the alarm system is designed in this way, the operators can 

glance at the alarm display and determine the general acceptability of overall plant performance. The emergent 

feature is effective if it conveys high-level information without requiring operators to analyze the individual 

parameters. 

The degree to which the individual and emergent features are perceptually salient to the observer is important in 

allocating attention to the elements of the display. The design goal is to map meaningful aspects of the operator's 

task information requirements to the visual characteristics of the display (the display grammar, as discussed in 

Section 4). Significant aspects of domain structure and behavior must be mapped to the characteristics of the 

representation so that the domain's semantics are directly visible to the operator (Woods and Roth, 1988). Issues 

that need to be addressed include selecting, scaling, and assigning lower-order data to characteristics of the 

representation. The challenge is to set up a mapping so that the task-meaningful semantics are directly visible to 

the observer and that both normal and disturbance conditions are clearly perceived.

NUREG/CR-6633 5 -56
5 DEVELOPMENT OF TECHNICAL BASIS

Pzr Water Temp 

550.0 .'4I 

525.0 

500.0 

675.0 

OF

Pzr Level 

70.0 

50.0 

30.0 

10.0 

% 0.0

(a) Emergent features showing normal operating conditions

Pzr Water Temp 

7oo 

525.0 

350.0 

175.0 

OF 0.0

Pzr Press 

3000.0 2250.0 

750.0 

P311p

(b) Same normal operating conditions without emergent features 

Figure 5.17 Separable Display With and Without Emergent Features 

The value of each type of display - separable, integral, and configural - may depend on the task requirements 

(Bennett and Flach, 1992). That is, if operators must conduct their tasks based merely on individual parameters, 

then separable displays may be preferred. If they depend on the integration of individual parameters, then integral 

displays may be preferred. If performance is based on a combination of the two, then configural displays may be 

the best solution for representation. 

Bennett et al. (1997, pp. 291-292) identified the following design principles based on information relationships: 

(1) Each relevant process variable should be represented by a distinct element within the display. If precise 

information about this variable, is needed, then a reference scale or supplemental digital information should be 

provided. 

(2) The display elements should be organized so that the emergent properties (symmetries, closure, parallelism) 

that arise from their interaction correspond to higher-order constraints within the process. Thus, when process 

constraints are broken (i.e., a fault occurs), the corresponding geometric constraints also are broken (the 

display symmetry is broken). 

(3) The symmetries within the display should be nested (from global to local) in a way that reflects the 

hierarchical structure of the process. High-order process constraints (e.g., at the level of functional purpose or

7NUREG/CR-6633

Pzr Pressure 

2275.01 

2250.0 

2225.0 

2200.0 

2175.0 

Psia

Pzr Level 

1.00 

0.75 

0.50 

0.25 

0.00 •

5 -57
5 DEVELOPMENT OF TECHNICAL BASIS .

abstract function) should be reflected in global display symmetries; lower-order process constraints (e.g., 

functional organization) should be reflected in local display symmetries. 

Proximity-Compatibility Principle 

The proximity-compatibility principle (PCP) was developed to guide the mapping of operator task characteristics 

and display design (Carswell and Wickens, 1987; Wickens and Carswell, 1995). Task proximity and display proximity need to be jointly considered when designing displays to support task performance. The task and display 

characteristics are described in more detail below. 

Task proximity can be described by several dimensions, including the following: 

* Temporal proximity - the degree to which two tasks must be performed in close temporal proximity to achieve 

a goal 

* Processing proximity - the overlap between the information processing that is needed to process different 

information 

• Statistical proximity - the degree of covariation between information (e.g., when two parameters covary or 

when a change in one is reflected in a change in the other, the relationship has high statistical proximity) 

• Functional proximity - the similarity of objects as represented in the operator's mental model (e.g., all 

parameters describing the performance of a single component have high functional proximity) 

Three general categories of tasks are defined: integrative, nonintegrative processing of similar tasks, and 

nonintegrative processing of dissimilar tasks. Integrative processing tasks require operators to engage in 

(1) computational integration, i.e., combining or integrating data to make a decision or take an action; or (2) 

boolean integration, i.e., determining the satisfaction of boolean conditions to make a decision or take an action. 

In both cases, operators are using many information sources to arrive at a few decisions or to take a few actions (a 

many tofew relationship). 

In contrast to integrated tasks, nonintegrative processing of dissimilar tasks is characteristic of task independence 

or low task proximity. There is little similarity or interaction between the information or processing mechanisms 

required by two tasks. Nonintegrative processing of similar tasks is characteristic of tasks with moderate 

proximity. 

Display proximity addresses how perceptually similar two displays are that convey information on the same task. 

There are several dimensions that define perceptual proximity, including spatial proximity, chromatic proximity, physical dimensions used to code information (such as length), perceptual coding (such as digital vs. analog), and 

geometric (integrated or separate displays) (Bennett et al., 1997). Display proximity is enhanced by (1) arranging displays close together, (2) using shared information channels, (3) adding lines to connect or enclose information 

channels, (4) giving related information sources the same codes, such as color and orientation, (5) integrating 

information so it appears as a single object (object integration), and (6) configuring homogenous features into a 

new pattern (configuration). 

The PCP states that there should be a high correlation between task proximity and display proximity. High proximity tasks, such as those involved in process fault detection, must be performed in a highly integrated 

manner. These types of tasks are conducted better using high-proximity displays, e.g., by presenting data 

relationships as a single object (an integral object display), because many plant variables must be considered

NUREG/CR-6633 5 -58
5 DEVELOPMENT OF TECHNICAL BASIS

simultaneously. High proximity is accomplished by high integration of data. Low-proximity tasks are those in 

which the operator responds by focusing on individual system elements. These tasks are better supported by low

proximity displays, such as a series of bar charts. Essentially, the PCP states that low-proximity tasks will be better 

performed with low-proximity displays and vice versa. 

The basic phenomena that account for high-proximity display support of high-proximity tasks are reduced 

information access cost (IAC), object integration, and emergent features (Wickens and Carswell, 1995; Wickens, 

1986). Reduced IAC is accomplished by (1) allowing all display axes to be within foveal vision, thus minimizing 

search and scanning problems; (2) reducing the memory load associated with retaining part of the needed 

information while another display is accessed to get additional needed information; and (3) reducing the mental 

effort required to integrate the information. The more separate the information, the more IAC is involved. 

Emergent features and object integration enable perception to be used instead of mental computation. Emergent 

features, however, need to be mapped to task-related variables. 

These same phenomena can make the performance of low-proximity tasks more difficult because high-proximity 

displays will create clutter on the screen and lead to confusion. Close spatial proximity can make it difficult to find 

relevant information and discriminate it from other information. When a relevant object is within one visual 

degree of an irrelevant object, the presence of one has a negative effect on the processing of the other (and will 

increase IAC). Conversely, decreasing proximity of the display makes it easier to focus attention on a single object 

or parameter, and thereby supports the performance of low-proximity task. Thus, the PCP places emphasis on the 

mapping of display characteristics to task demands. Compatibility between the two will support performance while 

incompatibility will detract from performance. 

5.4.2 Research on the Effectiveness of Representational Aiding 

In this section, we consider the research addressing these approaches to representational aiding. Where possible, published reviews of the literature were used. These reviews were supplemented with individual research reports. 

For discussion of the studies addressing representational aiding, the term focused tasks refers to tasks that are 

independent from each other in terms of requiring separate information, and the term integrated tasks is used to 

refer to tasks that require integrating information. The discussion below centers around two main questions: (1) what are the overall effects of integral, configural, and separable displays on performance? and (2) what are the 

specific design features that explain these effects? 

Bennett and Flach (1992) conducted a literature review of studies examining the relative effects of configural graphical displays and separable displays on performance of integrated and focused tasks. They grouped the 

studies by test methodology, as follows: signal detection, multiple-cue judgement, retrospective memory probe, 

and system control. 

In the signal detection methodology, a participant observes a display and detects an abnormality. No actual system control is performed by the participant. The results of studies using this methodology generally showed that the 

configural displays led to better detection performance than separable displays, especially in integrated tasks 

requiring higher-level information. For tasks focused on lower-level information, the results were either 

insignificant or favored the separable displays. 

Studies using a multiple cue judgement methodology require the participants to observe a display and make a 

judgement about the system's status; The results were generally consistent in support of configural displays in 

integrated tasks. Bennett and Flach found only one study using focused tasks where performance was better with 

the separable display.

5 -59 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

In studies using the retrospective memory probe methodology, participants were asked to recall information about a 

system based upon display conditions. Again, participants performed better with configural displays for integrated 

tasks, but the display differences for focused tasks were insignificant. 

Finally, in the system control method, participants monitor and control a dynamic system. Performance measures 

reflecting performance goals are obtained. Bennett and Flach found little research using this method (see 

discussion of Bennett et al., 1993, below). In one area, the results from multiaxis tracking tasks supported the use 

of configural displays. 

In summary, the review showed that better performance was found with configural displays when the task was 

integrated, and therefore, required performance based on abstract integration of data. For focused tasks, where 

performance required responses based on specific aspects of the data, a "mixed pattern of results" was found 

(Bennett and Flach, 1992; Bennett et al., 1997); i.e., results were either insignificant or favored separable displays. 

Barnes and Suantak (1996) evaluated the effects of configural representations of complex tactical situations in 

comparison to conventional graphic and alphanumeric formats on the speed and accuracy of tactical decisions. 

Seventeen army officers made situational decisions based on the displays, and responses were faster and more 

accurate with the configural displays. 

Bennett et al. (1993) used the system control method to compare the performance of 20 students on a simulated 

manual feedwater control task, using either separable or configural displays in two experiments. In experiment 1, 

the task was to adjust the flow rate to maintain steam generator level. Task performance was the primary 

dependent variable. No significant differences between the displays was observed. In experiment 2, using the 

same participants and control tasks, a memory task was included to examine recall for focused information (e.g., 

rate of steam flow) or integrated information (e.g., the difference between steam and feedwater flow). To ask the 

memory question, the simulation was stopped and computer screens went blank. After the memory question was 

answered, the simulation was resumed. The results showed that there was a significant interaction of display type 

and question type. The configural display significantly improved performance on the integrated information, but 

not on the focused information. Also, while the configural display improved memory for important high-level 

information, differences in actual task performance were not significant. This result may parallel those of Vicente 

and colleagues (discussed in Section 5.3), where the configural information in the DURESS EID display did not 

greatly affect the control task performance. 

Coury and Boulette (1992) compared several graphical formats and found that a task's time constraint was an 

important factor affecting the display-task relationships. They found that "different representations of the same 

system data can have a profound effect on performance" (p. 707). They further stated that 

One clear conclusion from this research is that the effects of time stress complicate the selection of display formats 

for state identification tasks. Although designers of operator interfaces may be tempted to select the polygon display 

because of its apparent superiority under the most severe time constraints, such a choice would ignore a number of 

important trade-offs in performance. Consider changes in processing strategy. Although overall performance with 

the polygon display was good, the digital display was superior in many situations. (p. 723) 

Hurts (1996) investigated memory for configural information. Twelve undergraduate students responded to 

questions about interpretation of graphed information using either separable graphs or configural graphs. 

Dependent measures were time and accuracy. The results revealed that while configural graphs were superior to 

separable graph displays when the graphs were present, the effect was reversed when retrieval from memory was 

required to respond to questions on lower-level information. That is, performance on the memory test was superior 

NUREG/CR-6633 5 -60

I I
5 DEVELOPMENT OF TECHNICAL BASIS

with separable graphs. They also found that the effect was 16ssened when there was greater data-to-graph 

compatibility. 

While integral and configural displays appear to support some aspects of integrated task performance, some 

questions remain. What accounts for the effects, i.e., is it that the information is integrated into objects displays, such as polygons, or is it the emergent features? Further, how might performance of focused tasks be better 

supported in higher-level displays? These issues are discussed below. 

Flach and colleagues addressed the 'object vs. emergent feature' aspect of displays using a fault detection task 

(Buttigieg et al., 1988; Sanderson et al., 1989). Fault detection is a high-proximity or integrated task because 

many variables must be simultaneously considered. In such a situation, the PCP predicts that a high-proximity display will lead to superior performance. An integral display where all variables are integrated into a single object is a high-proximity display. The emergent features approach would predict that it is not the object per se 

that will lead to superior performance, but the salience of emergent features even if the individual displays are 

separate. 

To compare these predictions, participants engaged in a fault detection task for a simple process consisting of two 

inputs and one output. Four displays were used. One was an integral display where the inputs and outputs were 

integrated into a triangle display. The other three displays were separate bar charts, which were arranged in 

different ways to vary the salience of the emergent feature created by the three bar charts. Reaction time and false alarms were the dependent variables. Contrary to the proximity-compatibility principle, performance was better on 

separate bar chart displays than with an integrated object display. The bar chart associated with the best 

performance had the most salient emergent feature (linearity of the bar chart arrangement). 

Thus, it may not be the integration of information into an object that supports performance, but rather the emergent features. The failure of integral object displays to support focused tasks was observed as well. Montgomery and 

Bivona (1997) compared bar charts, two types of deviation bar charts, and an integrated object display for detecting differences in the temporal variability of information sources. Fourteen participants observed the displays and 

were asked to decide which of two sources was least variable. Accuracy was the dependent measure. The 

magnitude of the difference was varied. The deviation bar chart resulted in the best performance; the integral and 

standard bar chart displayed were the worst. The authors concluded that the deviation displays were more 

compatible with the task, which required the source variability to be considered separately. 

Thus, focused tasks are not supported by integral object displays and may not be supported well by configural displays. Bennett et al. (1996) addressed methods for enhancing the extraction of lower-level information from 

configural displays. Configural displays with and without several enhancements were compared: (1) color coding, layering, and separation; (2) tickmarks and gridlines, (3) bar graphs and extenders, and (4) digital values. These 

enhancements were systematically included and excluded. Eight students participated in the experiments and performed a simulated process control task. The experimental sessions were interrupted at random intervals and 

participants were asked either focused memory probes (addressing lower-level information) or integration memory probes (addressing higher-level information). Latency and accuracy were the dependent variables. The results 

showed that the inclusion of digital values was associated with the greatest increase in performance. Another 

positive enhancement was the use of tickmarks on the display axes and gridlines that extended the scale 

information into the display area. Thus, configural displays can be designed to better support focused tasks. 

Hansen (1995) also examined the enhancement of configural displays. The evaluation included the degree of configurality (configural or separable), presence of digital information (presented or not), and presence of time 

history information (presented or not). Configural and separable displays were compared with and without digital and time information. The experimental task was to monitor eight variables to determine an "event" defined by a

5 -61 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

decrease in all eight. Measures of reaction time, event hit rate, and false alarm rates were obtained. The displays 

were compared in two experiments. In the first, the rate of change of the event was either slow or rapid. In 

experiment 2, the signal-to-noise ratio was varied between noisy and not. Each experiment had 11 participants, 

graduate students, in a within-subjects design. Overall, the results showed that configural displays were better than 

separable displays for event hit rate, but led to higher rates of false alarms. The addition of time information was 

found most useful in noisy events. In an analysis of the combined results of the two studies, providing digital 

information led to faster response time, but there was no effect on hit or false alarm rates. The digital information 

provided a means of responding when the graphic display began to indicate the event. At times, digital 

information interfered with the emergent features of the integral displays. Hansen (1995) concluded that there was 

much support for the observation that 

...graphical representations can be effective for communicating information for controlling dynamic systems. 

However, there is less agreement as to the dimensions that distinguish effective graphical representations from 

ineffective ones. Experimental comparisons of display formats varying in degree of configurality, inclusion of 

supplemental digital information, and inclusion of explicit time dimensions were found to have significant effects on 

the detection of temporal events and the ability to detect trends. For example, it was observed in some conditions the 

inclusion of digital information in an integral graphical display can detract from its emergent features. (p. 539) 

Several studies focused on polygon displays. Woods et al. (1981, 1982) compared a safety panel composed of a 

configural polar graphic display (see Section 4) and a plant mimic display to one composed of trend graphs of 

safety parameters. Eight crews of professional operators experienced transients on a NPP simulator during their 

normal training sessions. No quantitative comparisons of the displays were performed. In a qualitative analysis, 

the trend display did not support operator decision making because of implementation problems, e.g., the update 

rate was too slow. The polar graphic display gave a useful overview of the plant's safety status, but the authors felt 

that lack of familiarity limited its use. The mimic display was the most effective and frequently used display. 

Other studies found an effect of polygons on performance. Performance on a fault detection task was found to be 

better with the polygon display (Hughes and MacRae, 1994). Greaney and MacRae (1996) evaluated the effect of 

polygon displays on fault diagnosis. As the number of vertices increased, the reaction time for diagnosis increased. 

Reaction time also increased as a function of polygon regularity, that is, the more irregular the display, the longer 

it took to diagnose a fault. Thus, such displays may be of more value for indicating higher-level information than 

for the display of individual parameters that must be assessed separately. 

Green and Logie (1996) examined the use of polygon displays to indicate the physiological status of a patient in 

intensive care. Polygons of eight to ten sides were developed. They found that information on status could quickly 

be derived from the displays and that performance was better for simpler polygons. They also observed that 

physicians were more attentive to information reflected by the top of the polygon; that is, patient information 

represented by the upper part of the polygon was more salient. In another experiment, polygons were developed to 

indicate specific medical conditions that were typically observed in intensive care units. Physicians were readily 

able to use these displays for diagnosing the nature and severity of the conditions. 

Pionek et al. (1997) investigated the effects of polygon characteristics on reaction time. The characteristics were 

color (color vs. no color), size (large vs. small, 30% of large), and number of vertices (6 or 12). Fourteen 

engineering students viewed the polygon displays to identify the presence of an abnormal state using reaction time 

as a measure. The results showed that color and size were significant, as was the interaction of color and number 

of vertices. 

Coury et al. (1989) found that, for fault diagnosis, a polygon display was worse than a bar chart display, but was 

superior to a digital display. Casey (1986) also found a polygon display worse than an integrated bar chart display. 

NUREG/CR-6633 5 -62

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Coury and Pietras (1989) compared the following three display formats: graphic, alphanumeric, and multiple formats. Performance in a simulated process control task was worse with graphic displays and best with the multi

format display. Thus, the overall results on the effectiveness of polygon displays are mixed. 

5.4.3 Conclusions 

In general, we agree with the conclusion of Bennett and Flach (1992) that configural displays support integrated task performance, and that a mixed pattern of results is found when focused tasks are performed (sometimes 

separable displays are better for focused tasks). Further, it appears that configural displays can be enhanced to 

better support focused task performance, such as by the inclusion of digital information. However, including 

digital information can detract from the emergent features, so it must be done cautiously. 

The aggregate of the studies related to information representation appears to support the PCP that performance is 

best supported when both the types of displays and their details closely match operators' tasks. Operators of NPPs 

have a wide variety of tasks, from high-level fault detection and diagnosis to lower-level component manipulation. Thus, one can conclude that operators should have a variety of different levels of displays that span the A-A 

hierarchy. (A careful review of NPP instrumentation and displays shows that to some extent this already exists). The operators should be able to select from this suite of displays or be able to reconfigure the level of displays, based on the task at hand. When this level of diversity of displays and flexibility in selection is provided, 

performance issues naturally arise (see Issue [6] below). 

In general, while the research reported in this section was more rigorous from a standpoint of experimental methodology than the EID studies, these studies also were limited in generalizability, because of the following 

factors: 

Domain representation - The domains represented by the displays did not address the complexity of real 

systems. While there were exceptions (e.g., Woods et al., 1982), in many of the studies the domains 

represented were very simple. Since the emergent features are meant to map to constraints of the system, increasing the complexity of the system can be expected to make the mapping considerably more complex. 

Tasks - In part due to simplifications of the domain, the tasks also were simple. Even the fault detection in 

several studies was simplified and based on only a few variables. Further, the studies did not address a wide 

range of tasks. 

Participants - More often than not, the participants were students and not domain experts. Further, they did 

not receive the training that, for example, Vicente's participants did. Thus, issues associated with the 

contribution of mental models to interpreting the displays did not reflect the complexity of interpretation that 

would arise in NPP operations. 

The limits on generalizability give rise to several possible research issues, as noted below. Resolving them is 

necessary for developing further guidance. 

(1) Lack of Specific HFE Guidance 

Bennett and Flach (1992) concluded that "Although there is general consensus that this type of display has the 

potential to improve human-machine system performance, there is less agreement on the principles or heuristics 

that should be used as design guidelines" (p. 514). The research on advanced graphic forms is at an early stage,

5 -63 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

and while we can formulate a few general guidelines, the research is not yet substantial or definitive enough to 

provide a solid technical basis on which to develop a complete set of guidelines. 

(2) System Complexity and Emergent Features 

As the number of vertices in a polygon display increased, the displays became complex and performance was 

reduced. Further, emergent features can be affected by unpredictable i"nteractions between component parts and 

produce unintended effects that may be misunderstood (Hansen, 1995). This illustrates the issue of increasing the 

complexity of the underlying domain to which integral and configural displays map. For complex dynamic 

systems, such as NPPs, not much is known about the dynamic aspects of the emergent features that may be used to 

represent them. In addition, as the graphic representation increases in complexity, so does the display grammar. 

(3) Perceptual Resolution 

Configural and integral displays require a perceptual process to take place, such as recognizing a change in an 

emergent feature. However, the degree to which a geometric form needs to change, before it is perceived as a 

distortion, is not well understood. 

(4) Elements of Configural Display 

The research discussed above suggested that configural displays could be enhanced, especially in support of 

focused tasks, for example, by including digital information. Research is needed to better understand the effects of 

display elements on performance and the effects of their interactions with other types of displays. 

(5) Evaluation of Display 

More comprehensive methods of evaluation are needed; this was also noted in Section 5.3 on the evaluation of EID 

displays. For example, the lack of definitive review guidance will need to be compensated for with dynamic 

evaluations. The criteria for the evaluations will have to be addressed. Hansen (1995) suggested a signal detection 

approach, i.e., an examination of hits, false alarms, and misses, because there is some evidence that subjects may 

generally be less conservative using configural displays under some situations, so that the hits as well as the false 

alarms increase. 

(6) Operator Use of a Large Span and Variety of Displays 

If and when the operators of a NPP have a large variety of different levels of displays from which to choose, how 

does the, designer decide how many displays are enough? Have the operators been given too much information, 

either in a single display or in the entire suite of displays that are available? Will the operators be able to select the 

appropriate display for the task or tasks at hand? Will operators tend to choose a few "favorite" displays even 

though they may not be the most appropriate for the tasks? If operators do switch displays based on varied tasks, 

will they pick the proper display? What sort of training should be developed to address these concerns? 

5.5 Organization of Displays 

The investigation of display representations discussed in Section 5.4 addressed predominantly simple systems, 

where task performance was accomplished with a single display. Even the studies directed at complex systems 

such as NPPs (discussed in Section 5.3) looked at performance with a single display page. However, real-world 

complex systems cannot be represented with a single graphic display (Bennett et al., 1997), and this is certainly 

NUREG/CR-6633 5-64

II
5 DEVELOPMENT OF TECHNICAL BASIS

true of NPPs, where display pages number in the hundreds and thousands (O'Hara et al., 1996). Thus, in complete information systems, representations are combined into display pages and pages into networks. 

There are two characteristics of computer-based HSIs that impact the organization of information: information 

volume and available display area. Computer-based HSIs equipped with digital I&C systems typically provide much more real-time information than conventional CRs. Gaddy et al. (1991) observed "a common occurrence 

that often accompanies the introduction of computer-based technologies is an order-of-magnitude increase in the 

quantity of information at the user's fingertips" (p. 256). 

An interesting paradox is that while the volume of information increases considerably, it is available through the 

very limited viewing area provided by workstation VDUs. This characteristic has been referred to as the "keyhole effect" (Woods et al., 1990), since at any given time most of the information is hidden from view; i.e., the operator has only a glimpse of the current plant information through the display devices. Therefore, operators must know 

what information and controls are available in the virtual information space, where they are, and how to navigate and retrieve them. If insufficient viewing area is available for operators to accomplish their tasks, they may have 

to frequently repeat navigation tasks. A problem related to the keyhole effect is that access to controls and display tends to be serial, e.g., only a few controls can be accessed at one time, which is in contrast to the parallel presentation of controls and displays in conventional CRs. 

Thus, to present plant information in a limited viewing area, multiple levels of display pages are needed, and each 

must be accessed from a large network of pages before it is viewed. This way of presenting information may interfere with the ability of operators to quickly scan indications and assess plant conditions. Not all desired 

information may be presented at one time via the HSI. Because information in computer-based CRs must be 

retrieved from display networks, operators must know where to look to obtain status information. That operators 

may only monitor what they can observe and may ignore information not immediately presented has been 

identified as a human performance concern associated with computer-based systems (Fujita, 1992; Elm and 

Woods, 1985; Stubler et al., 1991). 

Therefore, decisions on the design of display pages and their organization into networks are important considerations. This section will address these two aspects of display organization. 

5.5.1 Organization of Display Pages 

The limited display area in VDU-based systems means that when information is needed by an operator, it may be 

contained on more than one display page, which may require the operator to make rapid transitions between pages, try to remember values, or write values on paper. Compared to the sweeping wall-to-wall panels of older 

conventional CRs, CRs using advanced information systems have a more restricted view, even when multiple display devices are provided. Thus, the design of individual pages is critical to performance. The design review 

issues related to display pages are (1) the information to be combined into a single display page, and (2) the 

interaction of different display formats within a display page. However, very little research was identified that 

addressed these issues. 

Information Requirements for Single Pages 

In Section 5.2, the cognitive tasks supported by the information system were discussed. Those tasks included 

situation assessment, monitoring and detection of process disturbances, response planning, and response implementation. The information requirements for each of these activities are different and a task analysis, perhaps augmented by an analysis of the abstraction hierarchy, will help define the operator's information needs

5 -65 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

for performing these cognitive activities. After developing the overall information needs, the specific organization 

and coordination of the information into distinct pages must be addressed. 

Design considerations for display pages are highly related to and dependent on other factors, notably the following: 

* Design of information (how much information is packed into individual display pages) 

* Amount of display area that is available 

• Interface management functions (how operators move within the organization to retrieve information) 

These characteristics also interact with each other. Since the amount of information is large and the display area 

limited, more information-dense displays lead to fewer pages and less need to manage the interface. The tradeoff 

is that very dense displays can reduce the salience of important information and make it difficult to locate needed 

information within the display page. 

Recent research showed a preference for a high density of information in displays. Thomas et al. (1998) evaluated 

the design of a wall panel information system for an advanced CR. A mimic-type display was compared with a 

display with considerably more information at various levels of abstraction. They found the crew's situation 

awareness increased and workload decreased with the more dense design. In addition, operators preferred the new, 

more dense design. In this study, however, operators did not have control over the amount of information on the 

display. 

In another study (Roth and O'Hara, 1998), operators had control over the amount of information they could put on 

the display. They tended to cram their VDUs with information. For example, they would place multiple trend 

plots on a single VDU. It was not unusual for operators to divide a VDU screen into two windows and place four 

trend plots in each. Since each trend plot could have up to 4 parameters trended, operators could have up to 32 

parameters trended per VDU. They were observed putting up multiple windows per VDU, occasionally 

overlapping and covering up one display with another. Covering up information by overlapping or covering one 

display with another was not found to be a problem in this study. 

An important reason for preferring information-dense displays was that operators rarely modified or pulled up new 

displays once a scenario was started. Rather, they tended to select a set of trend plots to display at the beginning of 

a scenario and kept that arrangement. This finding is consistent with the finding that operators are generally 

reluctant to engage in interface management tasks in high workload situations (O'Hara et al., 1997). 

Another aspect of information requirements for single pages is how complex information that extends beyond the 

physical limits of single pages, such as a detailed P&ID or procedure flow chart, should be divided to form 

individual pages. Again, tradeoffs must be considered with approaches for interface management. 

Representational Format Interactions 

While operators appear to prefer information-dense displays (to reduce interface management tasks), very little 

research has been conducted on the effects of mixing representation formats. Payne and Lang (1995) examined the 

effects of mixed vs. pure format displays on performance of focused vs. separable tasks. In pure format display, all 

indicators within a display are the same, either words (such as "fifty-five"), digital (such as "55"), or analog 

(graphic representation). In mixed format displays, a variety of formats are presented. They hypothesized that 

performance on focused tasks would be better supported with mixed formats. Undergraduate students visually 

monitored a dynamic simulated system. Their task was to indicate whether an indicator exceeded the safe range. 

NUREG/CR-6633 5 -66

I I
5 DEVELOPMENT OF TECHNICAL BASIS

Accuracy and latency were the dependent variables. Responses were faster with the analog displays than with 

verbal or digital displays. There was no support for the contention that mixed format displays would better support 

focused tasks. In fact, more errors were committed in the mixed condition. Thus, mixing formats may have 

undesired effects on performance. However, the results of this study are suggestive only. The displays, 

participants, and tasks were not representative of a NPP context. 

5.5.2 Organization of Networks 

As previously noted, NPPs can have hundreds or even thousands of display pages. Therefore, their organization 

into a network is essential for rapidly and efficiently retrieving information. A variety of schemes for organizing 

information are possible: e.g., by physical equipment layout, systems, functions, and operator tasks. While there 

has not been a great deal of research on the organization of information in a display network, numerous 

organizational schemes have been criticized. Endsley (1995) criticizes physical organization, and Heslinga and 

Herbert (1993, 1995) criticize a plant systems organization. 

The most common organization scheme for display networks in NPPs is by plant functions and systems. Such 

organizations were effective for the layout of conventional plants, but a recent survey of power plants suggests it 

may not be effective for organizing display networks. Heslinga and Herbert (1993, 1995) surveyed designers, 

managers, and operators of plants of various types from eight European countries on their experiences with the 

introduction of advanced HSI technology, either as part of new plant design or plant modernization. They found 

that the displays differed between a system-based hierarchal organization (e.g., an organization around plant 

systems with displays arranged hierarchically from overview to detailed levels) and a task-oriented organization 

(e.g., an organization around operator tasks). In comparison to task-oriented displays, operators reported having 

problems finding information in system-based organizations. This preference was especially true for non-routine 

operations. The authors suggested that operators' information needs were not centered along the system hierarchy, 

but along task requirements. Since the system-based organization did not conveniently provide the information 

needed for task performance, operators had to search through multiple displays. Heslinga and Herbert (1993) 

concluded that displays "dedicated to a certain task instead of the currently used system-based displays, could be a 

valuable alternative" (p. 33). For non-routine situations, there was insufficient time to search for information and 

controls within a hierarchy of displays, leading to the operators' reluctance to search through functionally-oriented 

sets of display pages. However, it may be more difficult to establish task-based displays for non-routine situations, 

due to their unpredictable nature and potentially large number. While Heslinga and Herbert's observations are 

important, we found no research that specifically addressed and manipulated the organization of display pages. 

Woods and Roth (1988) proposed using multiple organizational schemes. They discussed fixed vs. adaptive 

collections of information. Given that there is too much data to see all at once, the question is how can the 

representational design be manipulated to help the operator find the right set at the right time? While needed data 

may be available in the system, the data needs to be grouped based on some criterion, such as where the device is 

located, physical system boundaries, time, or function. 

When the designer preselects fixed collections of data, the effect on performance depends on the amount of 

searching that the operator must do across the predefmed groupings, which in turn, depends on (1) the degree to 

which the designer's choice of criteria anticipates the questions to be asked by the operator, (2) the degree to which 

links between data sets and domain questions vary, and (3) the characteristics of the representation in which the 

search occurs (e.g., parallel vs. serial display). 

Since the questions asked may vary, adaptive collections of data may be best. In adaptive collections, the grouping 

of data can change as a function of the state of the plant or of the operators' intent. Suppressing alarms based on

5 -67 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

plant mode is an example of grouping changes based on the plant's state. Operators, however, may make changes 

to individual display pages, create new ones, or even globally change the grouping of pages. Such global changes could be predefimed (e.g., normal operations, startup, normal shutdown, post-scram) and then invoked at the 

operators' discretion. 

5.5.3 Conclusions 

While the issues of organization of pages and networks are important and interact with many other design considerations, little data were found on the organization of large, complex information systems. Some 

information seemed to contradict current HFE guidance and nuclear practice. That is, very dense displays were 

found to lead to better performance and to be preferred by operators. The common industry practice of network 

organization by function and system was found to inadequately support operators' task performance. 

Both of these findings may reflect the effects of workload associated with interface management, i.e., that operators seek to minimize the demands of interface management. Less dense displays increase the number of display pages 

and the amount of interface management needed to retrieve information. Similarly, organizing display pages by plant systems increased the interface management workload associated with performing tasks. A further 

complicating factor may be the flexibility of the HSI, which may inhibit the development of an accurate model of 

the display structure. If operators can view the display network in many different ways, it may be more difficult to 

understand its organization, and as a result, navigation plans may be less predictable. Thus, the organization of 

display pages and of networks remain human performance issues and further research to address them is 

warranted. 

5.6 General Summary and Issues 

The purpose of Section 5 was to establish a technical basis upon which to develop review guidance for information 

systems and to identify human performance issues where the technical basis is insufficient for developing guidance. To accomplish this objective, we examined research in the following four areas: (1) generic, operator 

cognitive tasks that an information system must support, (2) analysis of information requirements, (3) representation of information, and (4) organization of information. Section 5.6.1 summarizes the results of this 

research, and Section 5.6.2 gives the remaining research issues. 

5.6.1 Information System Design and Operator Performance 

Cognitive Tasks 

The plant information systems provide the basis for operators to perform their role in the plant. The general tasks 

that the information systems must support include situation assessment, monitoring and detection, response 

planning, and response implementation. The following general observations were made: 

• Operators' information needs, especially in support of monitoring, detection, and situation assessment, are 

very broad, both in terms of the range of activities that operators need to be aware of, and the level of 

abstraction they need to consider. 

• Information should be expressed in terms that are at the level that will directly impact decision making and 

control. Cognitive resources should not be required to mentally process data to obtain information that is 

needed.

NUREG/CR-6633 5 -68
5 DEVELOPMENT OF TECHNICAL BASIS

• It is important to correctly map the plant or process and the identification of information needs and their 

representation to the operator's mental model. These mappings are necessary to ensure that the 

representations of information trigger the appropriate mental representation of the situation. 

* HSI design can strongly affect the operator's cognitive activity and can make information difficult to obtain 

(through poor organization, high data density, and excessive interface management demands). 

The information processing basis for the operators' cognitive tasks should be considered in the guidance for 

reviewing information system designs. The way in which these needs are reflected in the information system's 

design is through analysis of information requirements, representation, and organization. 

Analysis of Information Requirements 

In NPP design, a systems engineering approach was recommended to identify information requirements. In such 

an approach, plant functions are decomposed based on performance requirements and allocated to human and 

machine resources. Human functional requirements are further analyzed in task analyses to better define the 

information requirements of task performance. 

Such traditional approaches to information system design were criticized because the information requirements 

they define could fail to meet the operator's needs in the face of unplanned events. EID was identified as a 

potential alternative approach. There are four aspects of the EID approach (Section 5.3): 

* Specification of information requirements 

* Organization of information among the levels of an abstraction hierarchy 

• Development of innovative displays to show functionality 

* Addition of analytical redundancy 

Each may have some performance-enhancing features. 

In specifying information requirements, using the A-A matrix may provide a framework for identifying 

information needs by giving a structure that specifies levels of functional decomposition. Further, it appears that 

interfaces based on EID provide better support for knowledge-based behavior than those based on physical 

variables alone. By using EID goals, the relevant system requirements, constraints, and parameters can be 

evaluated and represented. 

Developing innovative displays to show functionality may not be considered purely a feature of EID, but once the 

levels of functionality that should be displayed are specified, the designers should develop appropriate displays. 

Such displays will probably not be found in the design engineer's tool box of past successful applications, and thus 

will likely need to be developed for the particular application in question. 

The addition of analytical redundancy is another feature of EIDs that is not always applied, but which appears to be 

quite valuable. This is a particularly.effective method of verifying that higher level functions and goals are, in fact 

being met, and if they are not being met, determining why and what to do about it.

5 -69 -NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

While several methodological issues were raised related to the studies of EID effects, EID designs have shown 

some beneficial effects. These effects may be stronger for fault diagnosis and disturbance handling than for normal 

operations, and these aspects may be important to safety. Several issues remain to be addressed; they are 

summarized in Section 5.6.2. 

Representation of Information 

Once information requirements are identified, they must be represented in the information system. That is, display formats are developed to communicate the information to the operators. The success of representational aids was 

discussed as being a function of the characteristics of the plant, the operators, and the representation through which the operators view and act on the plant. All three are essential parts of the overall system. The 

representation involves the following two types of mapping: 

* Correspondence map between the properties and characteristics of the system to be represented and the 

features in the representation (how well the display communicates meaningful information about the plant that 

is needed by operators) 

* Coherence map between the features in the representation and the physical and cognitive characteristics of the 

operator (how comprehensible the representation is to the operator) 

Three generic types of displays were identified: separable, integral, and configural displays. Separable displays 

are representations where each process parameter is presented individually and no relationships between the 

parameters are provided by the representation itself. Integral displays are displays which show the integration of 

information in such a way that the individual parameters used to generate the display are not represented in it. 

Configural displays provide a combination of the other two types. Information about the basic parameters is 

available; however, information on their relationships becomes evident as features emerge from the representation. An emergent feature is a "high-level, global perceptual feature" that is produced by the interactions among individual parts or graphical elements of a display. 

Research studies tended to compare these display types based on the types of tasks participants performed. Focused 

tasks are independent from each other in terms of requiring separate information. Integrated tasks required the 

integration of information. 

In general, the research found that configural displays support integrated task performance, and that a mixed 

pattern of results is found when focused tasks are performed (sometimes separable displays are better for focused 

tasks). The studies found that performance is best supported when the representation of information matches 

operator's tasks. Operators of NPPs have a wide variety of tasks, from high-level fault detection and diagnosis to 

lower-level component manipulation. Thus, one can conclude that operators should have available to them a 

variety of different levels of displays that span the A-A hierarchy. 

However, most of the studies of information representation also were found to be limited in generalizability due not 

to methodological issues, but to the use of simple systems, tasks, and inexperienced participants. Thus, the 

extension of the research findings to complex systems such as NPPs must be made with caution. 

Organization of Information 

While organization of display pages and networks is important, little data on organization of large, complex information systems were found. The study of these issues is complicated due to the interaction of the

NUREG/CR-6633 5 -70
5 DEVELOPMENT OF TECHNICAL BASIS

information's organization and other design considerations, such as display area available and interface 

management functions. 

Some of the research seemed to contradict current HFE guidance and nuclear practice. That is, very dense displays 

were found to lead to better performance and were preferred by operators. Additionally, the common industry 

practice of organization by functions and systems was found to be difficult for operators to use in some studies. 

Both of these findings may reflect the effects of workload associated with interface management, i.e., operators 

seek to minimize the demands of interface management. Less dense displays increase the number of display pages 

and the amount of interface management needed to retrieve information. Similarly, organizing display pages for 

plant systems increased the workload of interface management associated with performing tasks. A further 

complicating factor may be flexibility of the HSI, which may inhibit the development of an accurate model of the 

display's structure. If operators can view the display network in many different ways, it may be more difficult to 

understand its organization, and as a result, navigation plans may be less predictable. Thus, the organization of 

display pages and of networks are human performance issues, and further research is needed to address them. 

5.6.2 Issues 

This section summarizes the human performance issues associated with information systems that were identified 

based on the literature review. The issues, as described below, are not mutually exclusive. They overlap, and some 

are more general than others; thus, some may be considered secondary to others. As the issues all pertain to the 

interactions within an integrated human-machine system, interdependencies are unavoidable. The issues are 

organized into the following three sections below: 

* Technical Basis Issues 

* Design Review Issues 

* Operator-Related Issues 

From a researcher's standpoint, these issues represent topics for which research is necessary before additional 

guidance can be developed. From a regulatory review perspective, many of them can be addressed on a case-by

case basis as part of the design process review discussed in Part 2 of this document. 

5.6.2.1 Technical Basis Issues 

There were five issues identified in the Technical Basis area. 

(1) Lack of a Well-Defined EID Process 

While the EID approach may be a promising advance for the system engineering process, developing a well

defined process for conducting an analysis using the abstraction hierarchy is important to its broader application to 

the design process.

5-71 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

(2) Lack of Specific Representation Guidance 

The research on advanced graphic forms is still at an early stage, and while a few general guidelines can be 

established, the research is not yet substantial or definitive enough to provide a solid technical basis on which to 

develop a complete set of guidelines. 

(3) Evaluation of Operating Experience 

For those EID displays which were installed in operating facilities, the operating experience and its application to 

EID should be thoroughly assessed. Such experience is important to formulating criteria for design review to 

address EID aspects of the HSI. 

(4) Critical Testing and Evaluation of EID Concepts 

There has not been much thorough research that carefully assesses the various aspects of EID, e.g., information 

requirements, effect of organization of information along functional lines, display representation, and use of 

analytical redundancy. Studies tend to confound these characteristics or provide weak assessments of the 

contribution of the various aspects of EID concepts. 

The successful handling of unplanned or unanticipated events with EID displays under actual operational 

conditions in complex systems has not been demonstrated. 

Further research also is necessary to more clearly determine which cells of the A-A matrix are important to 

operations. 

(5) Evaluation of Displavs 

More comprehensive methods for evaluating displays are needed. For example, the lack of definitive review 

guidance will need to be compensated for with dynamic evaluations. The criteria for the evaluations will have to 

be addressed. For example, signal detection approaches (i.e., an examination of hits, false alarms, and misses) 

were recommended. This is due to some evidence that subjects may generally be less conservative using configural 

displays under certain situations, so that the hits as well as the false alarms increase (Hansen, 1995). 

5.6.2.2 Design Review Issues 

There were ten issues identified in the area of Design Review. 

(1) Task and Temnoral Considerations 

The A-A matrix and the EID approach address the plant at a functional level rather than on a task or temporal 

basis. However, the importance of presenting information consistent with task requirements has been a 

fundamental basis of the systems approach, and deviating from it is a problem in many new plant information 

systems (where information is organized around plant systems rather than operator tasks). 

Much of the operators' goal-directed activity is centered around temporal constraints, which is one reason 

operators like trend displays. The operators' tasks unfold along a temporal continuum with some tasks being 

performed in parallel and some sequentially. This temporal dimension to operations may be lacking from the A-A 

analysis, except where time is reflected in higher-level information, such as rates.

NUREG/CR-6633 5 -72
5 DEVELOPMENT OF TECHNICAL BASIS

Additionally, the EID approach seems to be built around functional decomposition of means-ends relationships. 

How well will this relate to common tasks and disturbances, that are highly proceduralized, such as in EOPs? 

While it is important to be able to address unanticipated and unplanned events, still most of the operator's 

activities are well planned and the information system needs to support these activities as well.' 

The proper role of task-based information in the design of displays and how it is integrated into the EID approach 

needs to be addressed. 

(2) Volume of Information 

Due to the extensive analytical process used in EmI, there is the potential that the process, including the A-A 

matrix, may lead to identifying more information than can be displayed practically. Also, too many display pages 

may be required to satisfy information requirements identified by EID. 

(3) Density of Display Information 

The increase in information may be linked to an increase in the density of information on individual displays. 

While this may be done to minimize the need for interface management tasks, such as navigating to retrieve 

additional display pages, the density may be associated with a lack of salience of important information' Another 

issue with dense displays is that, for any given operator task, the amount of irrelevant information.increases, and 

from a human performance perspective, performance decreases as the amount of irrelevant data increases (Mitchell 

and Miller, 1983). 

(4) Operator Use of a Large Span and Variety of Displays 

If and when the operators of a NPP have a large variety of different levels of displays from which to choose, how 

does the designer decide how many displays are enough? Have the operators been given too much information, 

either in a single display or in the entire suite of displays that is available? Will the operators be able to select the 

appropriate display for the task or tasks at hand? Will operators tend to choose a few "favorite" displays even 

though they may not be the most appropriate for the tasks? If operators do switch displays based on varied tasks, 

will they pick the proper one? What sort of training should be developed to address these concerns? 

(5) System Complexity and Emergent Features 

As the number of vertices in a polygon display increased, the displays became complex and performance was 

reduced. Further, emergent features can be affected by unpredictable interactions between component parts and 

produce unintended effects that may be misunderstood (Hansen, 1995). This illustrates the issue of increasing the 

complexity of the underlying domain to which integral and configural displays map. For complex dynamic 

systems, such as NPPs, not much is known about the dynamic aspects of the emergent features that may be used to 

represent them. In addition, as the graphic representation increases in complexity, so does the display's grammar. 

(6) Perceptual Resolution 

Configural and integral displays require a perceptual process to take place, such as recognizing a change in an 

emergent feature. However, the degree to which a geometric form needs to change before it is perceived as a 

distortion is not well understood.

5-73 NUREG/CR-6633
5 DEVELOPMENT OF TECHNICAL BASIS

(7) Elements of Configural Display 

The research suggested that configural displays could be enhanced, especially in support of focused tasks, for 

example, by including digital information. Research is needed to better understand the effects of display elements 

on performance and the effects of their interactions with other types of displays. 

(8) Effect of Instrumentation Failures 

The effects of instrumentation failures on EID displays is a potentially significant problem that needs to be 

addressed. There are several subissues associated with this problem. 

• Can operators detect a failure of instrumentation? 

Can failures of instruments result in representations that are interpreted by operators as real process failures; 

and, perhaps more importantly, can real process failures be misinterpreted as failures of instruments? 

If operators detect a failure, should they suspend use of the display? 

* Since the display integrates many parameters into a single display, what is the effect of its loss on operations 

and how effectively can operators switch to backup displays? 

Many designers have implemented advanced features that address this concern. For example, the concepts of 

redundancy and diversity in instrumentation, coupled with automatic parameter validation and appropriate 

notification of operators when instruments fail their validation, were demonstrated successfully. 

(9) Organization of Information 

The issues related to organization of display pages and networks remain as important research topics. 

(10) Integration of EID Displays into Remainder of HSI 

The integration of a new and significantly different EID display into the remainder of the standard HSI of a CR is 

an important consideration that needs to be addressed. 

5.6.2.3 Operator-Related Issues 

There were three Operator-Related issues. 

(1) Implications for Training and Qualification 

Christoffersen et al. (1995) concluded that "...to experience the benefits of EID, it seems likely that operators need 

to be trained to think functionally rather than procedurally. It would seem that this would require a fundamental 

shift in NPP operation philosophy...it may be that operators have to possess certain types of cognitive 

characteristics that may not be considered in the traditional selection process in the nuclear industry" (p. 143). 

Concern was also expressed that EID interfaces may inhibit long-term learning and retention (Wickens, 1992). 

Also, for any display where the information is sophisticated and a large amount is consolidated into one or a few 

figures, the requirements for training become very important. One needs to understand all of the aspects of the

NUREG/CR-6633 5-74
5 DEVELOPMENT OF TECHNICAL BASIS

display and how it reacts to various operational transients, accidents, and instrument failures. The long-term 

effects of EID-type displays on operators' performance and strategies are unknown. 

The training aspects of Design Review Issue (4), Operator Use of a Large Span and Variety of Displays, should 

also be addressed. 

(2) Acceptance by Operators 

The issue of operators' acceptance of a new and different type of display is also important, as shown by their 

comments during experiments. One suggestion from the Rankine cycle experiments was to introduce the displays 

initially in training and then perhaps gradually into the plant. 

(3) Internal vs. External Mental Models 

Section 5.3 discussed the issue of the appropriate model(s) to apply as a basis for designing the display. Providing 

a model that accurately characterizes the process and its supporting systems, yet that reflects operators' training, 

experience, and cognitive capabilities, is an important factor. Designing displays that characterize the system in 

ways that may not reflect the cognitive requirements of plant operators to perform situation assessment, monitoring 

and detection, response planning and response implementation, may result in negative performance. Achieving 

this balance will require additional research.

5 -75 NUREG/CR-6633
6 DEVELOPMENT OF GUIDANCE 

NUREG-0711 provides the high-level design process criteria used by the NRC for reviewing the overall HFE 

programmatic goals and objectives. NUREG-0711 does not address the detailed review of final HSI designs as 

implemented, such as displays, control, or procedures; rather, it references NUREG-0700 for detailed guidance. In 

Part I of NUREG-0700, Rev. 1, the design process is covered. The general framework of NUREG-0711 was used 

to structure the design review process developed in NUREG-0700. However, Part 1 also considers aspects of the 

design process of the HSI review in general terms; i.e., it does not identify the specific considerations that may be 

important in reviewing an individual technology, such as information systems. The only detailed HSI technology

specific guidelines are in Part 2 of NUREG-0700. They relateto the detailed form and functional characteristics of 

HSIs, such as displays and controls, and not the unique design process considerations. 

For information systems, both types of guidance are necessary to perform a design review. That is, while there is a 

sufficient technical basis to develop detailed guidance for design implementation for some characteristics of 

information systems, as is typical in NUREG-0700, several limitations in the technical basis were identified (see 

Section 5.6.2.1). Thus, there are many issues remaining for which typical NUREG-0700 guidance could not be 

developed. Until it can, these issues can be addressed for specific information systems. To support the latter, 

considerations were identified for reviewing the information system design process. Thus, guidance was developed 

to address both the design process and human factors engineering design. Each is described below. 

6.1 Guidance for Reviewing the Information System Design Process 

Guidelines were developed to address important design process considerations identified in the literature, and to 

provide a place where human performance issues could be identified and assessed, case-by-case, during a design 

review. The format of the guidelines corresponds to the NRC's general design process guidance NUREG-071 1. 

They were organized into the following sections: 

* Operating Experience Review 

* Function and Task Analysis 

* Human-System Interface Design 

* Training Program Development 

These guidelines are contained in Section 9. 

6.2 Guidelines for HFE Design Review 

A draft set of guidelines was developed from the findings and source materials discussed in Section 5. The 

methodology was conservative in the sense that guidelines were developed only for those aspects of display design 

that, in our interpretation, are supported by the literature. Many of the research studies evaluated were either weak 

in experimental methodology or limited in generalizability to complex systems. This situation constrains the 

development of new HFE guidelines. 

The guidelines adopted the standard format in NUREG-0700, Rev. 1. An example is presented below:

6-1 NUREG/CR-6633
6 DEVELOPMENT OF GUIDANCE

10.1-1 Correspondence Mapping 

There should be an explicit mapping between the characteristics and functions of the system to be represented 

and the features of the display representation, i.e., changes in the appearance of the display form should have a 

one-to-one relation with the plant states it represents. These changes should result from explicit rules relating 

the physical form of the display and its meaning with respect to the plant state represented. 

ADDITIONAL INFORMATION: Correspondence mapping addresses how well the display communicates 

meaningful information about the plant to operators. The physical form and functions of the display should be 

explicitly tied to its meaning with respect to the plant's functions and states. The meaning of the display must 

consider the instrumentation and the data processing that drives the display format. If a single display can 

lead to more than one interpretation, the display is ambiguous and can be more easily misunderstood. 

Changes in the graphic display should be unambiguously related to the plant's state. The same graphic 

change should not be associated with more than one interpretation. 

Discussion: Woods and Roth (1988) conceptualized displays as involving a cognitive-system triad consisting 

of the world to be acted on, the agents who will act on the world, and the representation through which the 

agents act on the world. The success of a representation is a function of two types of mapping (Bennett et al., 

1997): correspondence and coherence. (Coherence is addressed in Guideline 10.1.3.) The importance of 

providing consistent one-to-one mapping between the work domain's constraints and the features of the 

display was identified as an important principle derived from research on representing complex systems 

(Bennett, Nagy, and Flach, 1997; Vicente and Rasmussen, 1992). 

Each of the guidelines is composed of the following components: 

Guideline Number - Within each section, individual guidelines are numbered consecutively. Each guideline has a 

number which reflects its section and subsection location, followed by a dash, and then its unique number. 

Guideline Title - Each guideline has a brief, unique, descriptive title. 

Review Criterion - Each guideline contains a statement of an HSI characteristic so that the reviewer may judge the 

HSI's acceptability. The criterion is not a requirement, and discrepant characteristics may be judged acceptable 

based on the procedures in the review process. 

Additional Information - For many guidelines, there is additional information which may address clarifications, 

examples, exceptions, and details on measurements, figures, or tables. This information is intended to support the 

reviewer's interpretation or application of the guideline. 

Discussion - This section summarizes the technical basis on which the guideline was developed. It may identify 

the primary source documents, the technical literature such as journal articles, or the general principle from which 

the guideline was derived. This section will be removed when the guidance is integrated into NUREG-0700. 

In place of the Discussion section will be a Source field. 

Source - The source field identifies the NUREG or NUREG/CR.(or other document) containing the technical basis 

and development methodology for the guideline. As is the standard practice, the source field will cite this 

document as it will appear in its final form. 

The guidelines were organized into the following sections: 

Guidelines for General Display 

NUREG/CR-6633 6-2

I I
6 DEVELOPMENT OF GUIDANCE

* Display Format 

- Integral Formats 

- Configural Display Formats 

The guidance was submitted for peer review and was revised accordingly. 

The new review guidelines for information system design will be integrated with the information system guidance 

in NUREG-0700, Rev. 1.

6-3 NUREG/CR-6633
7 GENERAL SUMMARY 

The objective of this study was to develop HFE review guidance for information systems based on a technically 

valid methodology for guidance development. To support this objective, the following tasks were performed: 

* Development of a framework for describing key design characteristics of information systems 

Development of a technical basis using human performance research and analyses pertinent to information 

systems and related areas 

Development of HFE review guidelines for information systems in a format consistent with NUREG-0700, 

Rev. 1, and other NRC review guidance 

* Identification of remaining issues regarding information systems for which research was insufficient to support 

the development of NRC review guidance 

The status of each will be briefly addressed below. 

Characterization Framework for Information Systems 

The first step in guidance development was to identify the areas for which guidance was needed. Existing 

information systems were reviewed to identify the dimensions and characteristics along which such systems can be 

defined. Characterization was important because it provided a structure within which the reviewer could request 

information about a system, and with which to structure the guidance. 

The characterization of information systems included the following major components: information requirements, 

representation system, management functions, and display devices. Representation systems were further 

hierarchically broken down into display elements (basic building blocks of displays, such as axes, alphanumeric 

elements, and icons), display format (such as P&ID and trend graphs), display pages (a defined set of information 

that is intended to be displayed as a single unit), and display networks (the organization of display pages). This 

report mainly addressed information requirements and representation. 

Technical Basis Development 

The development of detailed review guidelines began with the collection of technical information on which to base 

the guidance. Research in the following areas was examined: (1) generic cognitive tasks that an information 

system must support, (2) analyses of information requirements, (3) representation of information, and 

(4) organization of information. 

The plant's information systems are the basis for operators to perform their roles. The general tasks that the 

information system must support include situation assessment, monitoring and detection, response planning, and 

response implementation. 

The information processing requirements of the operators' cognitive tasks have to be reflected in the design review 

guidance for the information system. The way in which these needs are reflected in designing information systems 

is in analyses of information requirements, the representation of information, and the organization of information. 

In NPP design, a systems engineering approach has been recommended for identifying information requirements. 

Plant functions are broken down based on performance requirements and allocated to humans and machines. 

Human functional requirements are further analyzed to better define the information required to perform the tasks. 

7- 1 NUREG/CR-6633
7 GENERAL SUMMARY

EID was identified as a potential additional approach. Its four aspects, discussed to varying extents in the studies 

reviewed in Section 5.3, are the following: 

* Specification of information requirements 

* Organization of information among the levels of an abstraction hierarchy 

* Development of innovative displays to show functionality 

* Addition of analytical redundancy 

Each of them may have some performance-enhancing features. However, the associated research had 

methodological weaknesses that limited the conclusions that could be drawn. 

Once information requirements are identified, they must be represented in the information system; that is, display 

formats are developed to communicate the information to the operators. The success of representational aids was 

discussed as being a function of the characteristics of the plant, the operators, and the representation through 

which the operators view and act on the plant. All three are essential parts of the overall system. 

A variety of approaches to representation were discussed, including separable displays (individual parameters 

displayed in unrelated form), integral displays (individual parameters integrated into object display where the 

individual ones are not visible), and configural displays (individual parameters are displayed, but emergent 

perceptual features are created by the arrangement and organization of parameters). 

In general, some research support was found for the representation principles; however, most of the studies could 

not be generalized due to the use of simple systems and tasks, and inexperienced participants. Thus, extending the 

research findings to complex systems such as NPPs must be done with caution. 

While the organization of the display page and network are important, there were few data on organizing large, 

complex information systems. Such studies are complicated due to the interactions between informational 

organization and other design considerations, such as the display area available and interface management 

functions. Further, some research seemed to contradict current HFE guidance and nuclear practice; that is, very 

dense displays were found to lead to better performance and were preferred by operators, and the industry's 

common practice of function-system network organization was found to be difficult for operators to use when 

performing certain tasks. 

HFE Review Guidelines 

Once the technical information was assembled, a draft set of guidelines was developed. The guidelines were 

organized and specified in a standard format and are given in Part 2 of this document. In general, guidelines were 

only developed for those aspects of display design that, in our interpretation, are supported by the literature. As 

discussed in Section 5, many research studies were either weak in experimental methodology or limited in 

generalizability to complex systems. This situation constrains the development of new HFE guidelines. 

Guidance was developed to support (1) Design Process Review, and (2) HFE Design Review, both of which are 

needed to evaluate an information system design. While there was a sufficient technical basis to develop detailed 

design-implementation guidance, as is typical in NUREG-0700, several limitations in the technical basis were 

identified, and so, many issues remain for which typical NUREG-0700 guidance could not be developed. However, 

NUREG/CR-6633 7-2

I I
until it can be developed, these issues should be addressed for specific information systems case-by-case. To 

support reviewing such issues, design process review guidance was developed. 

The new guidance will be integrated into the existing guidance on information systems in NUREG-0700, Rev. 1. 

Information Systems Issues 

Where there was insufficient information to provide a technical basis upon which to develop valid design review 

guidance, an issue was defined. There were several human performance issues associated with information 

systems (in Section 5.6.3). The issues were organized into three topic areas: 

* Technical Basis Issues 

- Lack of a Well-Defined EID Process 

- Lack of Specific Representation Guidance 

- Evaluation of Operating Experience 

- Critical Testing and Evaluation of EID Concepts 

- Evaluation of Displays 

* Design Review Issues 

- Task and Temporal Considerations 

- Volume of Information 

- Density of Display Information 

- Operator Use of a Large Span and Variety of Displays 

- System Complexity and Emergent Features 

- Perceptual Resolution 

- Elements of Configural Display 

- Effect of Instrumentation Failures 

- Organization of Information 

- Integration of EID Displays into the Remainder of the HSI 

* Operator-Related Issues 

- Implications for Training and Qualification 

- Acceptance by Operators 

- Internal vs. External Mental Models

7-3 NUREG/CR-6633
8 REFERENCES 

Bailey, R. (1982). Human-performance engineering: A guide for system designers. New Jersey: Prentice-Hall, 

Inc. 

Bainbridge, L. (1986). What should a good model of the NPP operator contain. In Proceedings of the 

International TopicalMeeting on Advances in Human Factors in Nuclear Power Systems. LaGrange Park, IL: 

American Nuclear Society. 

Bainbridge, L. (1974). Analysis of verbal protocol from a process control task. The human operator in process 

control. London, England: Taylor and Francis. 

Barber, C. (1996). Human factors of human-computer interaction techniques in the control room. In N. Stanton 

(Ed.) Human factors in nuclear safety. London, England: Taylor and Francis. 

Barnes, M. and Suantak, L. (1996). Configural display of tactical information for instantaneous decision making. 

In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. Santa Monica, CA: Human 

Factors Society. 

Beltracchi, L. (1996). Display navigation design issues. In Nuclear Plant Instrumentation, Control, and Human

Machine Interface Technologies. LaGrange Park, IL: American Nuclear Society. 

Beltracchi, L. (1995). An OLE interface concept for Rankine cycle based heat engines. In Proceedings of the 

Topical Meeting on Computer-Based Human Support Systems: Technology, Methods and Future. LaGrange 

Park, IL: American Nuclear Society, Human Factors Division. 

Beltracchi, L. (1989). Energy, mass, model-based displays, and memory recall. IEEE Transactions on Nuclear 

Science, 36, 1367-1382. 

Beltracchi, L. (1987). A direct manipulation interface for water-based Rankine cycle heat engines. IEEE 

Transactions on Systems, Man and Cybernetics, 17, 478-487. 

Bennett, K. (1992). Representation aiding: Complementary decision support for a complex, dynamic control task. 

Control Systems, 19-24. 

Bennett, K. and Flach, J. (1992). Graphical displays: Implications for divided attention, focused attention, and 

problem solving. Human Factors, 34, 513-533. 

Bennett, K., Nagy, A., and Flach, J. (1997). Visual displays. In G. Salvendy (Ed.), Handbook of human factors 

and ergonomics (Second Edition). New York, NY: John Wiley and Sons. 

Bennett, K., Nittoli, B., and Walters, B. (1996). Design techniques to offset potential costs in configural displays. 

In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Bennett, K., Toms, M., and Woods, D. (1993). Emergent features and graphical elements: Designing more 

effective configural displays. Human Factors, 35, 71-97. 

Brehmer, A., Hill, J., and Brehmer, B. (1995). Using process operators to compare a traditional interface with i 

three dimensional interface for better process control. In Proceedings of 5th European Conference on Cognitive

8-1 NUREG/CR-6633
8 REFERENCES

Science Approaches to Process Control, VTT Symposium 158. Espoo, Finland: VTT Automation Finnish 

Psychological Society. 

Buttigieg, M., Sanderson, P., and Flach, J. (1988). Object vs. separate displays for process failure detection: The 

emergent features approach. In Proceedings of the Human Factors Society 32nd Annual Meeting. Santa Monica, 

CA: Human Factors Society. 

Carroll, J. and Olson, J. (1988). Mental models in human-computer interaction. In M. Helander (Ed.), Handbook 

of human-computer interaction. New York, NY: Elsevier Science Publishers. 

Carswell, C. and Wickens, C. (1987). Information integration and the object display - an interaction of task 

demands and display superiority. Ergonomics, 30, 511-527. 

Casey, E. (1986). Visual display representation of multidimensional systems: The effect of information 

correlation and display integrality. In Proceedings of the Human Factors Society 30th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Christoffersen, K., Hunter, C., and Vicente, K. (1996). A longitudinal study of the effects of ecological interface 

design on skill acquisition. Human Factors, 38, 523-541. 

Christoffersen,.K., Hunter, C., and Vicente, K. (1995). Ecological interface design and fault management 

performance: Long-term effects. In Proceedings of the Human Factors and Ergonomics Society 39th Annual 

Meeting. Santa Monica, CA: Human Factors and Ergonomics Society. 

Christoffersen, K., Hunter, C., and Vicente, K. (1994). Research on factors influencing human cognitive behavior 

(CEL 94-05). Toronto, Canada: University of Toronto (Cognitive Engineering Laboratory). 

Coury, B. and Boulette, M. (1992). Time stress and the processing of visual displays. Human Factors, 34, 

707-725. 

Coury, B., Boulette, M., and Smith, R. (1989). Effect of uncertainty and diagnosticity on classification of 

multidimensional data with integral and separable displays of system status. Human Factors, 31, 551-569. 

Coury, B. and Pietras, C. (1989). Alphanumeric and graphic displays for dynamic process monitoring and control. 

Ergonomics, 32, 1373-1389. 

DeGreene, K. (1970). Systems psychology. New York, NY: McGraw-Hill Book Company. 

Dien, Y. and Montmayeul, R. (1995). Taking Account of Human Factors in Control-Room Design. In 

Proceedings of the Topical Meeting on Computer-Based Human Support Systems: Technology, Methods and 

Future. LaGrange Park, IL: American Nuclear Society. 

Dinadis, N. and Vicente, K. (1996). Ecological interface design for a power plant feedwater subsystem. IEEE 

Transactions on Nuclear Science, 43, 266-277. 

Drury, C., Paramore, B., Van Cott, H., Grey, S., and Corlett, E. (1987). Task analysis. In G. Salvbndy (Ed.), 

Handbook of human factors. New York, NY: Wiley-Interscience.

NUREG/CR-6633 8-2
8 REFERENCES

Duncan, K. and Praetorius, N. (1992). Flow displays representing complex plant for diagnosis and process control. 

Reliability Engineering and System Safety, 36,239-244. 

Eckenrode, R. (1991). Technical note: Safety parameter display systems - 10 years later. Nuclear Safety, 32, 65

67. 

Edlund, C., Hume, S., and Lewis, M. (1995). Ecological design for a network scheduling interface. In 

.Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Edlund, C. and Lewis, M. (1994). Comparing ecologically constrained and conventional displays in control of a 

simple steam plant. In Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Effken, J., Kim, N., Kadar, E., and Shaw, R. (1992). Enhancing computer displays to support coordination of 

hemodynamic monitoring and treatment. In Proceedings of the Human Factors Society 36th Annual Meeting. 

Santa Monica, CA: Human Factors Society. 

Elm, W. and Woods, D. (1985). Getting lost: A case study in interface design. In Proceedings of the Human 

Factors Society 29th Annual Meeting. Santa Monica, CA: Human Factors Society. 

Endsley, M. (1988). Design and evaluation for situation awareness enhancement. In Proceedings.of the Human 

Factors Society 32nd Annual Meeting. Santa Monica, CA: Human Factors Society. 

Endsley, M. (1995). Towarda theory of situation awareness in dynamic systems. Human Factors, 37, 32-64. 

Ephrath, A. and Young, L. (1981). Monitoring vs. man-in-the-loop detection of aircraft control failures. In J. 

Rasmussen and W. Rouse (Eds.), Human detection and diagnosis of system failures. New York, NY: Plenum 

Press. 

EPRI (1993). Advanced light water reactor utility requirements document, Volume 1X1, passive plant (Revision 6). 

Palo Alto, CA: Electric Power Research Institute. 

EPRI (1992). Advanced light water reactor utility requirements document, Volume 11, evolutionary plant 

(Revision 4). Palo Alto, CA: Electric Power Research Institute. 

Flach, J. (1990). The ecology of human-machine systems 1: Introduction. Ecological Psychology, 2, 191-205. 

Flach, J. (1989). An ecological alternative to egg-sucking. Human Factors Society Bulletin, 32 (9), 4-6. 

Flach, J. (1988). Direct manipulation, direct engagement and direct perception: What's directing what? In 

Proceedings of the Human Factors Society 32nd Annual Meeting. Santa Monica, CA: Human Factors Society. 

Flach, J. and Bennett, K. (1992). Graphical interfaces to complex systems: Separating the wheat from the chaff. 

In Proceedings of the Human Factors Society 36th Annual Meeting. Santa Monica, CA: Human Factors Society. 

Flach, J. and Dominguez, C. (1995). User-centered design: Integrating the user, instrument, and goal. 

Ergonomics in Design, 4, 19-24.

8-3 NUREG/CR-6633
8 REFERENCES

Flach, J. and Hancock, P. (1992). An ecological approach to human-machine systems. In Proceedings of the 

Human Factors Society 36th Annual Meeting. Santa Monica, CA: Human Factors Society. 

Fracker, M. (1988). A theory of situation assessment: Implications for measuring situation awareness. In 

Proceedings of the Human Factors Society 32nd Annual Meeting. Santa Monica, CA: Human Factors Society. 

Fujita, Y. (1992). Time for tailoring human-machine interface technology to humans. 1992 IEEE Conference on 

Human Factors and Power Plants. New York, NY: Institute of Electrical and Electronics Engineering. 

Gaba, D. and Howard, S. (1995). Situation awareness in anesthesiology. Human Factors, 37, 20-31. 

Gaddy, C., Taylor, J., and Fray, R. (1991). Human factors considerations in digital control room upgrades. In 

Proceedings of the American Power Conference. Chicago, IL: Illinois Institute of Technology. 

Gagne, R. and Melton, A. (1988). Psychologicalprinciples in system development. New York, NY: Holt, 

Rinehart and Winston. 

Gibson, J. (1979). The ecological approach to visualperception. Boston, MA: Houghton Mifflin, Co. 

Gillan, D., Wickens, C., Hollands, J., and Carswell, C. (1998). Guidelines for presenting quantitative data in 

HFES publications. Human Factors, 40, 28-41. 

Greaney, J. and MacRae, A. (1996). Diagnosis of fault location using polygon displays. Ergonomics, 39, 400-411. 

Green, C. and Logie, R. (1996). Aberdeen polygons - computer displays of physiological profiles for intensive 

care. Ergonomics, 39, 412-428. 

Green, D. and Swets, J. (1988). Signal detection theory andpsychophysics. New York, NY: J. Wiley and Sons. 

Hansen, J. (1995). An experimental investigation of configural, digital, and temporal information on process 

display. Human Factors, 37, 539-552. 

Heslinga, G. and Herbert, M. (1995). Experiences with advanced systems for human-machine interaction. In 

Proceedings of the 6th IFAC/IFIP/IFORS/JEA Symposium on Analysis, Design and Evaluation of Man-Machine 

Systems. Cambridge, MA: MIT. 

Heslinga, G. and Herbert, M. (1993). Experience with the introduction of new systems for human-machine 

interaction from a human factors perspective (CORECH-HMI Summary Report). Netherlands: KEMA. 

Hink, J., Wogalter, M., and Eustace, J. (1996). Display of quantitative information: Are grables better than plain graphs or tables? In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Hollands, J. and Dyre, B. (1997). Bias in proportion judgments with pie charts: The cyclical power model. In 

Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

NUREG/CR-6633 8-4

I I
8 REFERENCES

Hollands, J. and Spence, I. (1997). Integral and separable dimensions in graph reading. In Proceedings of the 

Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, CA: Human Factors and 

Ergonomics Society. 

Hollands, J. and Spence, 1. (1992). Judgments of change and proportion in graphical perception. Human Factors, 

34,313-334. 

Hollnagel, E., Hunt, G., Praetorius, N., and Yoshimura, S. (1984). Reports from the pilot experiment on multilevel 

flow modeling displays using the GNP simulator (HWR 114). Halden, Norway: OECD Halden Reactor Project. 

Howie, D. and Vicente, K. (1997). Using self-explanation to exploit ecological interface design. In Proceedings of 

the Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, CA: Human Factors and 

Ergonomics Society. 

Howie, D. and Vicente, K. (1996). Measuring the impact of ecological interface design on operator skill 

acquisition. In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. Santa Monica, 

CA: Human Factors and Ergonomics Society. 

Hughes, T. and MacRae, A. (1994). Holistic peripheral processing of a polygon display. Human Factors, 36, 645

651. 

Hurts, K. (1996). The effect of configural graphs on concurrent and retrospective performance. In Proceedings of 

the Human Factors and Ergonomics Society 40th Annual Meeting. Santa Monica, CA: Human Factors and 

Ergonomics Society. 

IAEA (1994). Development and implementation of computerized operator support systems in nuclear installations 

(Technical Reports Series No. 372). Vienna: International Atomic Energy Agency. 

IEC (1989). Design for control rooms of nuclear power plants (IEC-964). Geneva, Switzerland: International 

Electrotechnical Commission (Bureau Central de la Commission Electrotechnique Internationale). 

IEEE (1996). Guide for the application of human factors engineering in the design of computer-based monitoring 

and control displays for nuclear generating stations (IEEE Std. P1289, Draft 13). Piscataway, NJ: Institute of 

Electrical and Electronics Engineers. 

IEEE (1988). IEEE guide to the application of human factors engineering to systems, equipment, and facilities of 

nuclear power generating stations (IEEE Std. 1023-1988). New York, NY: Institute of Electrical and Electronics 

Engineers. 

Itoh, J., Sakuma, A., and Monta, K. (1995). An ecological interface for supervisory control of BWR nuclear power 

plants. Control Engineering Practice, 3(2), 231-239. 

Itoh, J., Yoshimura, S., Ohtsuka, T., and Masuda, F. (1990). Cognitive task analysis of nuclear power plant 

operators for man-machine interface design. In Proceedings ofAdvances in Human Factors Research on 

Man/Computer Interactions: Nuclear and Beyond. Nashville, TN: American Nuclear Society. 

Janzen, M. and Vicente, K. (1997). Attention allocation within the abstraction hierarchy. In Proceedings of the 

Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, CA: Human Factors and 

Ergonomics Society.

8-5 NUREG/CR-6633
8 REFERENCES

Kahneman, D. and Triesman, A. (1984). Changing views of attention and automaticity. In R. Parasuraman and 

R. Davies (Eds.), Varieties of attention. New York, NY: Academic Press. 

Kockler, F., Withers, T., Podiack, J., and Gierman, M. (1990). Systems engineering management guide 

(Department of Defense AD/A223 168). Fort Belvoir, VA: Defense Systems Management College. 

Lee, J. (1995). Ecological interface design: Applications in the maritime industry. In.Proceedings of the 

International Symposium on Human Factors on Board: Man-Machine Interface on Safety of Navigation. Bremen, 

Germany. 

Lindsay, R. (1990). A display to support knowledge based behavior. In Proceedings of the Topical Meeting on 

Advances in Human Factors Research on Man/Computer Interactions: Nuclear and Beyond. LaGrange, IL: 

American Nuclear Society. 

Lindsey, R. and Staffon, J. (1988). A model based display system for the Experimental Breeder Reactor-II. ANS 

Transactions, 57, 264-265. 

Liner, R. and DeBor, J. (1987). Progress review of six safety parameter display systems (NUREG/CR-4797). 

Washington, DC: U.S. Nuclear Regulatory Commission. 

Lorenzo, D. and Brune, R. (1985). Computer-based controls can reduce operator errors. In 7th International 

System Safety Conference. San Jose, CA. 

Maddox, M. (1996). Critique of"A longitudinal study of the effects of ecological interface design on skill 

acquisition." Human Factors, 38, 542-545. 

Malin, J., Schreckenghost, D., Woods, D., Potter, S., Johannesen, L., Holloway, M., and Forbus, K. (1991). 

Making intelligent systems team players: Case studies and design issues. Volume I: Human-computer interaction 

design. (NASA Technical Memorandum 104738). Houston, TX: National Aeronautics and Space 

Administration. 

Meshkati, M., Buller, B., and Azadeh, M. (1994). Simulation and experimental studies of operator's decision 

styles and crew composition while using an ecological and traditional user interface for the control room of a 

nuclear power plant. In Proceedings of the Twenty-Second Water Reactor Safety Information Meeting. 

Washington, DC: U.S. Nuclear Regulatory Commission. 

Meyer, J. (1997). A new look at an old study on information display: Washburne (1927) reconsidered. Human 

Factors, 39, 333-340. 

Meyer, J., Gopher, D., and Levy, J. (1997). Discrimination between functions in tables and graphs. In 

Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Meyer, J., Shinar, D., and Leiser, D. (1997). Multiple factors that determine performance with tables and graphs. 

Human Factors, 39, 26-286. 

Mitchell, C. and Miller, R. (1983). Design strategies for computer-based information displays in real time control 

systems. Human Factors, 25, 353-369. 

NUREG/CR-6633 8-6

I I
8 REFERENCES

Monta, K., Itoh, J., and Makino, M. (1992). An advanced man-machine system for BWR nuclear power plants. In 

Proceedings of the 1992 IEEE Fifth Conference on Human Factors and Power Plants. Monterey, CA. 

Monta, K., Takizawa, Y., Hottori, Y., Hayashi, T., Sato, N., Itoh, J., Sakuma, A., and Yoshikawa, E. (1991). An 

intelligent man-machine system for BWR nuclear power plants. In Proceedings of the A191 Frontiers in 

Innovative Computing for the Nuclear Industry. LaGrange Park, IL: American Nuclear Society. 

Montgomery, D. and Bivona, L. (1997). The effects of graphical display formats on observer's accuracy in 

detecting differences in source variability. In Proceedings of the Human Factors and Ergonomics Society 41st 

Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society. 

Moray, N. (1995). Information needs in the control of continuous and discrete processes. Fifth European 

Conference on Cognitive Science Approaches to Process Control. Finland: Valtion Teknillinen Tukimuskeskus 

(VTT). 

Moray, N. (1986). Monitoring behavior and supervisory control. In K. Boff, L. Kaufman, and J. Thomas (Eds.), 

Handbook of Human Perception and Performance. New York: Wiley. 

Moray, N., Lootsteen, P., and Pajak, J. (1986). Acquisition of process control skills. IEEE Transactions on 

Systems, Man, and Cybernetics, 16, 497-504. 

Moray, N., Jones, B., Rasmussen, J., Lee, J, Vicente, K., Brock, R., and Djemil, T. (1993). Development of a 

performance indicator for the effectiveness of human-machine interfaces for nuclear power plants (NUREG/CR

5977). Washington, DC: U.S. Nuclear Regulatory Commission. 

Moray, N., Lee, J., Vicente, K., Jones, B., and Rasmussen, J. (1994). A direct perception interface for nuclear 

power plants. In Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Mumaw, R., Roth, E., Vicente, K., and Bums, C. (1996). A model of operator cognition performance during 

monitoring in normal operations. Pittsburgh, PA: Westinghouse Science and Technology Center (prepared for the 

Atomic Energy Control Board). 

Murphy, E. and Mitchell, C. (1986). Cognitive attributes: Implications for display design in supervisory control 

systems. International Journal of Man-Machine Studies, 25, 411-438. 

NAS (1995). Digital instrumentation and control systems in nuclear power plants: Safety and reliability issues. 

Washington, DC: National Academy Press. 

Neisser, U. (1976). Cognition and reality. San Francisco, CA: W.H. Freeman & Co. 

Neisser, U. (1967). Cognitive psychology. New York, NY: Appleton-Century Crofts. 

O'Hara, J. (1994). Advanced human-system interface design review guideline: Volume 1: General evaluation 

model, technical development, and guideline description (NUREG/CR-5908). Washington, DC: U.S. Nuclear 

Regulatory Commission. 

O'Hara, J., Brown, W., Higgins, J., and Stubler, W. (1994). Human factors engineering guidelines for the review 

ofadvanced alarm systems (NUREG/CR-6105). Washington, DC: U.S. Nuclear Regulatory Commission. 

8-7 NUREG/CR-6633
8 REFERENCES

O'Hara, J., Brown, W., Stubler, W., Wachtel, J., and Persensky, J. (1996). Human-system interface design review 

guideline (NUREG-0700, Rev. 1). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Brown, W., and Nasta, K. (1996). Development of the human-system interface design review guideline 

(BNL Technical Report L1317-2-12/96). Upton, NY: Brookhaven National Laboratory. 

O'Hara, J., Higgins, J., Stubler, W., Goodman, C., Eckenrode, R., Bongarra, J., and Galletti, G. (1994). Human 

factors engineering program review model (NUREG-071 1). Washington, DC: U.S. Nuclear Regulatory 

Commission. 

O'Hara, J., Stubler, W., Brown, W., and Higgins, J. (1997). Integrated system validation: Methodology and 

review criteria (NUREG/CR-6393). Washington, DC: U.S. Nuclear Regulatory Commission. 

O'Hara, J., Stubler, W., and Higgins, J. (1996). Hybrid human-system interfaces: Human factors considerations 

(BNL Technical Report J6012-T1-4/96). Upton, NY: Brookhaven National Laboratory. 

O'Hara, J., Stubler, W., and Nasta, K. (1997). Human-system interface management: Effects on operator 

performance and issue identification (BNL Technical Report W6546-1-1-7/97). Upton, NY: Brookhaven 

National Laboratory. 

Patrick, J. (1987). Information at the human-machine interface. In J. Rasmussen, K. Duncan, and Leplat, J. 

(Eds.), New technology and human error. New York, NY: John Wiley and Sons. 

Pawlak, W. and Vicente, K. (1996). Inducing effective operator control through ecological interface design. 

International Journal of Human-Computer Studies, 44, 653-688. 

Payne, D. and Lang, V. (1995). Mixed versus pure display format in integration and nonintegration visual display 

monitoring tasks. Human Factors, 37, 507-527. 

Pejtersen, A. and Rasmussen, J. (1997). Effectiveness testing of complex systems. In G. Salvendy (Ed.), 

Handbook of human factors and ergonomics (Second Edition). New York, NY: John Wiley and Sons. 

Pionek, J., Heidenreich, M., Shanahan, M., and Schoepke, D. (1997). Factors affecting user response to polygon 

displays. In Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting. Santa Monica, 

CA: Human Factors and Ergonomics Society. 

Rasmussen, J. (1986). Information processing and human-machine interaction. New York, NY: North Holland. 

Rasmussen, J. (1983). Skills, rules, knowledge: Signal, signs, and symbols and other distinctions in human 

performance models. IEEE Transactions on Systems, Man, and Cybernetics, 13, 257-267. 

Rasmussen, J. (1981). Models of mental strategies in process control. In J. Rasmussen and W. Rouse (Eds.), 

Human detection and diagnosis of system failure. New York, NY: Plenum Press. 

Reason, J. (1990). Human error. New York, NY: Cambridge University Press. 

Reising, D. and Sanderson, P. (1996). Work domain analysis of a pasteurization plant: Using abstraction 

hierarchies to analyze sensor needs. In Proceedings of the Human Factors and Ergonomics Society 40th Annual 

Meeting. Santa Monica, CA: Human Factors and Ergonomics Society. 

NUREG/CR-6633 8-8

I I
8 REFERENCES

Reynolds, A. and Flagg, P. (1977). Cognitive psychology. Cambridge, MA: Winthrop. 

Roth, E. (1994). Operator performance in cognitively complex simulated emergencies: Implications for computer

based support aids. In Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Roth, E., Bennett, K., and Woods, D. (1987). Human interaction with an "intelligent" machine. International 

Journal of Man-Machine Studies, 27,479-525. 

Roth, E. and Mumaw, R. (1995). Using cognitive task analysis to define human interface requirements for first-of

a-kind systems. In Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Roth, E., Mumaw, R., and Lewis, P. (1994). An empirical investigation of operator performance in cognitively 

demanding simulated emergencies (NUREG/CR-6208). Washington, DC: U.S. Nuclear Regulatory Commission. 

Roth, E. and O'Hara, J. (1998). Integrating digital and conventional HSIs: Lessons learnedfrom a control room 

modernization program (BNL Report J6012-3-4-5/98). Upton, NY: Brookhaven National Laboratory. 

Sakuma, A., Sato, N., Mizukami, M., Yoshikawa, E., and Ikeda, J. (1990). An intelligent man-machine interface 

for supporting operator knowledge-based behavior. In Proceedings of the ANS Topical Meeting on Advances in 

Human Factors Research on Man/Computer Interactions: Nuclear and Beyond. LaGrange Park, IL: American 

Nuclear Society. 

Sanderson, P., Flach, J., Buttigieg, M., and Casey, E. (1989). Object displays do not always support better 

integrated task performance. Human Factors, 31, 183-198. 

Shamo, M., Meyer, J, and Gopher, D. (1996). Predicting values and trends from tables and graphs. In 

Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. Santa Monica, CA: Human 

Factors Society. 

Sheridan, T. (1976). Toward a general model of supervisory control. In T. Sheridan and G. Johannsen (Eds.), 

Monitoring behavior and supervisory control. New York: Plenum Press. 

Shively, R. (1995). Emergency (911) dispatcher decision making: Ecological display development. In 

Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Santa Monica, CA: Human 

Factors and Ergonomics Society. 

Sorkin, R., Kantowitz, B., and Kantowitz, S. (1988). Likelihood alarm displays. Human Factors, 30, 445-459. 

Sorkin, R. and Woods, D. (1985). Systems with human monitors: A signal detection analysis. Human Computer 

Interaction, 1, 49-75. 

Stubler, W., Higgins, J., and O'Hara, J. (1996). Evaluation of the potential safety significance of hybrid human

system interface topics (BNL Technical Report J6012-T2-6/96). Upton, NY: Brookhaven National Laboratory. 

Stubler, W. and O'Hara, J. (1996). Group-view displays: Functional characteristics and review criteria (BNL 

Technical Report E2090-T4-4-12/94). Upton, NY: Brookhaven National Laboratory.

8-9 NUREG/CR-6633
8 REFERENCES

Stubler, W., O'Hara, J., and Kramer, J. (2000). Soft controls: Technical basis and human factors review guidance 

(NUREG/CR-6635). Washington, DC: U.S. Nuclear Regulatory Commission. 

Stubler, W., Roth, E., and Mumaw, R. (1991). Evaluation issues for computer-based control rooms. In 

Proceedings of the Human Factors Society 35th Annual Meeting. Santa Monica, CA: Human Factors Society. 

Takizawa, Y. et al. (1994). An intelligent man-machine system for future nuclear power plants. Nuclear 

Technology, 107, 72-82. 

Thomas, V., Sugibayashi, N., Lin, L., and Roth, E. (1998). Enhanced operator support through integrated 

resources and large wall panel display application within a compact main control room. In Proceedings of the 

11th Pacific Basin Nuclear Conference. 

Thurman, D. (1997). The Interactive Monitoring and Control (IMAC) design methodology: Application and 

empirical results. In Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting. Santa 

Monica, CA: Human Factors and Ergonomics Society. 

Van Cott, H. (1997). Were the control room design reviews worth it? In 1997 IEEE Sixth Conference on Human 

Factors and Power Plants: Global Perspectives of Human Factors in Power Generation. New Jersey. 

Van Cott, H. and Kinkade, R. (1972). Human engineering guide to equipment design. Washington, DC: U.S. 

Government Printing Office. 

Vicente, K. (1997). Should an interface always match the operator's mental model? CSERIAC Gateway, 8 (1), 

1-4. 

Vicente, K. (1996). Evaluation of a Rankine cycle display for nuclear power plant monitoring and diagnosis. 

Human Factors, 38, 506-521. 

Vicente, K. (1992). Multilevel interfaces for power plant control rooms I: An integrative review. Nuclear Safety, 

33, 381-397. 

Vicente, K. (1991). Supporting knowledge-based behavior through ecological interface design (EPRL-9 1-01). 

Urbana, IL: University of Illinois, Engineering Psychology Research Laboratory. 

Vicente, K. (1990). Coherence- and correspondence-driven work domains: Implications for systems design. 

Behavior and Information Technology, 9, 493-502. 

Vicente, K., Christoffersen, K., and Hunter, C. (1996). Response to Maddox critique. Human Factors, 38, 546

549. 

Vicente, K., Christoffersen, K., and Pereklita, A. (1995). Supporting operator problem solving through ecological 

interface design. IEEE Transactions on Systems, Man, and Cybernetics, 25(4), 529-545. 

Vicente, K., Moray, N., Lee, J., Rasmussen, J., Jones, B., Brock, R., and Djemil, T. (1996). Evaluation of a 

Rankine cycle display for nuclear power plant monitoring and diagnosis. Human Factors, 38, 506-52 1. 

Vicente, K., Mumaw, R., and Roth, E. (1997). Cognitive functioning ofcontrol room operators -final phase 

(Report prepared for the Atomic Energy Control Board). Toronto: University of Toronto. 

NUREG/CR-6633 8- 10

I
8 REFERENCES

Vicente, K. and Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions 

on Systems, Man, and Cybernetics, 2, 589-606. 

Vicente, K. and Rasmussen, J. (1990). The ecology of human-machine systems II: Mediating 'direct perception' 

in complex work domains. Ecological Psychology, 2(3), 207-249. 

Vicente, K. and Rasmussen, J. (1988). On applying the skills, rules, knowledge framework to interface design. In 

Proceedings of the Human Factors Society 32nd Annual Meeting. Santa Monica, CA: Human Factors Society. 

Vicente, K. and Wang, J. (1996). Taking full advantage of process constraints in advanced interface design. In 

Proceedings of the 1996 American Nuclear Society International Topical Meeting on Nuclear Power Plant Instrumentation, Control and Human-Machine Interface Technologies. La Grange Park, IL: American Nuclear 

Society. 

Welch, D. and O'Hara, J. (1996). Human factors in the process of developing nuclear power plant systems. In 

N. Stanton (Ed.), Human factors in nuclear safety. London: Taylor and Francis. 

Wickens, C. (1992). Computational models of human performance (Report ARL 92-4). Champaign, IL: 

University of Illinois. 

Wickens, C. (1986). The object display: Principles and experimentalfindings (Report CPL-86-6). Champaign, 

IL: University of Illinois. 

Wickens, C. (1984). Engineering psychology and human performance. Columbus, OH: Merrill Publishing 

Company. 

Wickens, C. and Carswell, C. (1995). The proximity compatibility principle: Its psychological foundation and 

relevance to display design. Human Factors, 37, 473-494. 

Wickens, C. and Kessel, C. (1981), The detection of dynamic system failures. In J. Rasmussen and W. Rouse 

(Eds.), Human detection and diagnosis of system failures. New York, NY: Plenum Press. 

Woods, D. and Elias, G. (1988). Significance messages: An integrated display concept. In Proceedings of the 

Human Factors Society 32ndAnnual Meeting. Santa Monica, CA: Human Factors Society. 

Woods, D., Johannesen, L., Cook, R., and Sarter, N. (1994). Behind human error: Cognitive systems, computers, 

and hindsight. CSERIAC, 94-01, SOAR. 

Woods, D., Potter, S., Johannesen, L., and Holloway, M. (1991). Human interaction with intelligent systems: 

Volume 1-trends, problems, new direction (CSEL Report 1991-001). Columbus, OH: Cognitive Systems 

Engineering Laboratory. 

Woods, D. and Roth, E. (1988). Cognitive systems engineering. In M. Helander (Ed.), Handbook of human

computer interaction. New York, NY: North-Holland. 

Woods, D., Roth, E., Stubler, W., and Mumaw, R. (1990). Navigating through large display networks in dynamic 

control applications. In Proceedings of the Human Factors Society 25th Annual Meeting. Santa Monica, CA: 

Human Factors Society.

8-11 NUREG/CR-6633
8 REFERENCES

Woods, D., Wise, J., and Hanes, L. (1982). Evaluation of safety parameter display concepts (NP-2339, Vol. 1). 

Palo Alto, CA: Electric Power Research Institute. 

Woods, D., Wise, J., and Hanes, L. (1981). An evaluation of nuclear power plant safety parameter display systems. 

In Proceedings of the Human Factors Society 25th Annual Meeting. Santa Monica, CA: Human Factors Society. 

Woodson, W. (1981). Human factors design handbook. New York, NY: McGraw-Hill Book Company. 

Xu, W., Dainoff, M., and Mark, L. (1996). Externalizing a work domain structure using an abstraction hierarchy: 

An alternative approach for hypertext interface designs. In Proceedings of the Human Factors and Ergonomics 

Society 40th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.

NUREG/CR-6633 8- 12
PART 2 

Review Guidelines for Information System Design

NUREG/CR-6633
9 REVIEW GUIDANCE FOR THE INFORMATION SYSTEM 

DESIGN PROCESS 

Guidance for reviewing the design process for information systems was developed to address important design 

process considerations identified in the literature and to provide a means whereby human performance issues may be assessed during a design review (see Section 6.1). The review guidelines were formatted to correspond to the 

NRC's general design process guidance in NUREG-07 11. The guidelines are organized into the following 

sections: 

* Operating Experience Review 

• Function Analysis 

* Task Analysis 

* Human-System Interface Design 

* Training Program Development. 

Guidelines may specify that some aspect of an information system needs to be "evaluated." General approaches to 

evaluation methods and criteria development are defined in NUREG-0700. Since the guidance in this section will 

eventually be incorporated into that document, the methods by which such evaluation should be performed and by 

which criteria should be identified are not repeated below. 

9.1 Operating Experience Review (OER) 

(1) Available operating experience with advanced information systems should be reviewed to take advantage of 

lessons learned in the operational use of the systems, as well as to ensure that problematic aspects of their 

design implementation are addressed. 

Discussion: There has not been much operating experience with advanced graphic displays in NPPs. An 

analysis of the available experience will help ensure the design supports the intended aspects of operational 

performance. This is consistent with the NUREG-07 11 criterion to review the operating experience associated 

with approaches for implementing HSI designs. 

9.2 Function and Task Analysis 

(1) The function analysis and task analysis criteria by which information requirements are defined should be 

clearly documented. 

Discussion: In developing a higher-level display, such as one based on EID principles, a clear definition is 

needed of what higher-level functions and goals are addressed, e.g., electric production and safety. For the 

latter goal, the NRC established five critical safety functions needed in NPPs to achieve overall safety: reactivity control, reactor core cooling and heat removal, RCS integrity, radioactivity control, and containment 

conditions (NUREG-1342 - NRC, 1988). These should be addressed in high-level safety displays; Tables 2 

and 3 of NUREG-1342 give sample parameters for both BWRs and PWRs. 

(2) The information requirements established in advanced graphical displays should be based on the operators' 

cognitive tasks (i.e., situation assessment, monitoring and detection, response planning and response 

implementation).

9-1 NUREG/CR-6633
9 DESIGN PROCESS REVIEW GUIDANCE

Discussion: Advanced graphical displays are integral and configural displays that use novel graphic forms to 

convey plant information, such as the Rankine cycle display. Information defined by requirements analyses, 

such as task analysis and abstraction-hierarchy analysis, should consider the operator's use of the information 

and the specific cognitive tasks it will be used for. This will help ensure that the information is specified at the 

appropriate level for the operators' needs. 

(3) The analysis of information requirements should consider the different needs of individual staff in the control. 

Discussion: Roth and O'Hara (1998) identified the different information needs of the shift supervisor, reactor 

operator, balance-of-plant operator, and shift technical advisor. They vary both in terms of specific 

information requirements, as well as in the level of abstraction needed. 

9.3 Human-System Interface Design 

(1) Explicit guidance should be available defining the relationship between the physical form of the display and 

its meaning with respect to the plant's status. 

Discussion: NUREG-07 11 specifies the need for the HSI design to reflect HFE guidelines in supporting 

standardization and consistency of design. Explicit guidance defines the relationship between the physical 

form and functions of the display and its meaning with respect to the plant's functions and states. These 

rules should consider the instrumentation and the data processing driving the display's format. If a single 

display can have more than one interpretation, it is ambiguous and can be more easily misunderstood. 

(2) The information presented and its organization into display pages should be based on considering operators' 

tasks when using the displays. Display pages should include as much information as can be efficiently 

represented and interpreted to minimize the need for operators to retrieve additional pages. 

Discussion: Display pages should reflect operators' information needs. Performance can suffer when 

information is missing (Fujita, 1992; Elm and Woods, 1985) or when the interface management workload 

makes operators reluctant to retrieve information (O'Hara, Stubler, and Nasta, 1997). 

(3) When more than one display format is used on a display page, an evaluation should determine whether the 

user's perception of one format is negatively impacted by the presence of the other one(s). 

Discussion: There has been little research on interactions between formats; however, some evidence 

indicates that unwanted interactions can occur (Payne and Lang, 1995). For example, including two 

graphic formats may create an unintended emergent feature. An emergent feature is a high-level, global 

perceptual feature generated by the interactions among individual parts or graphical elements (e.g., lines, 

contours, and shapes) to produce perceptual properties, such as symmetries, closure, and parallelism (see 

Figure 5.17). 

(4) The density of information on a display page should be evaluated to ensure that important information is 

readily perceived, and needed information is rapidly identified. 

Discussion: It is difficult to precisely specify the information density of graphic displays. Operators may 

prefer displays of greater density to minimize the need to retrieve information from other pages (Roth and 

O'Hara, 1998; Thomas et al., 1998). However, there are trade-offs between density and perceptual clutter, 

and a greater chance of making errors when information is not found. For example, operators experienced 

difficulty in understanding plant situations depicted in the Toshiba EID displays (Itoh, Sakuma, and Monta, 

1995), partly because the pages were too crowded with information. This tradeoff needs to be evaluated in 

designing pages. 

(5) The organization scheme of display pages within the network should be readily apparent to operators. 

NUREG/CR-6633 9-2

I I
9 DESIGN PROCESS REVIEW GUIDANCE

Discussion: In a computer-based HSI, there can be many displays; thus, the organization of pages is 

important. Operators need to know the organizational scheme of display pages to locate and retrieve the 

appropriate ones. Consideration may be given to providing the operators with variable, operator-selectable, 

schemes which could enable them to adapt the organization of data to meet their current needs (Woods and 

Roth, 1988). 

(6) The effects of instrumentation failures on graphic displays should be analyzed. Potential failure problems 

should be evaluated with respect to the following: 

"* Can operators detect a failure of instrumentation? 

"* Can instrument failures result in representations that are interpreted by operators as real process failures; 

perhaps more importantly, can real process failures be misinterpreted as instrument failures? 

"* If operators detect a failure, should use of the display be suspended? 

"* Since the display integrates many parameters into a single display, what effect does its loss have on 

operations and how effectively can operators transition to backup displays? 

Discussion: The failure modes of the display and their effects on plant operations must be carefully 

examined, especially with complex graphical displays. The effects of instrumentation failures need to be 

understood to ensure that they do not lead to incorrect situation assessment; i.e., operators mistakenly 

interpreting a graphic change due to an instrument failure as a change in process state. 

(7) Access to displays within a network should be evaluated to ensure rapid, efficient retrieval of information 

needed to support operators' tasks. 

Discussion: Operators have experienced difficulty in gaining access to information to support their tasks. 

Heslinga and Herbert (1993, 1995) found that information organized by systems was sometimes problematic 

for operators, and required excessive interface management. 

(8) Unwanted effects of integrating a new, novel graphic representation into a conventional HSI (other displays, 

other control room HSIs, and environmental considerations such as lighting levels) should be evaluated and 

minimized. 

Discussion: Just as graphic displays can interact within display pages, they can interact with other 

information sources in the control room; such interactions are not fully understood. Since there is no 

technical basis for guidance on these interactions, they should be examined case-by-case during the design 

process. 

(9) The following aspects of information system design should be carefully analyzed and evaluated: 

"* Number of VDUs - to ensure that the display area is sufficient to show the important information needed 

by operators without them having to perform extensive interface management 

"* Interface management functions - to ensure that the HSI features are easy to use and provide explicit 

interface management support 

"* Flexibility of HSI and display features and functions - to ensure that the flexibility of the system does not 

unduly burden operators, nor increase the chance of misunderstandings and errors.

9-3 NUREG/CR-6633
9 DESIGN PROCESS REVIEW GUIDANCE

Discussion: The keyhole effect resulting from the operator's limited view of plant information through 

VDUs, the interface management tasks, and computer-system flexibility all significantly impact 

performance. These factors have also affected the design of display pages and network organizations. 

Digital instrumentation and control systems typically provide much more information than do conventional 

systems. Operators must know what information and controls are available in the virtual workspace, where 

they are, and how to navigate and retrieve them. Information is viewed through the limited display area of 

workstation VDUs; this is the so-called "keyhole effect" (Woods et al., 1990). Consequently, most 

information is hidden from view. Operators may have to frequently repeat interface management and 

navigation tasks which can distract them from their primary task of monitoring and controlling the plant. 

For example, a problem related to the keyhole effect is that access to controls and displays tends to be serial, 

i.e., only a few can be accessed at one time. This contrasts with the parallel presentation of controls and 

displays in conventional systems, where HSIs are predominantly spatially dedicated and their form and 

function does not change. Advanced HSIs are flexible; they can be configured and can function in various 

operating modes. 

The flexibility of computer-based HSIs and their lack of spatial dedication can be problematic for operators. 

As discussed in O'Hara, Stubler, and Nasta (1997), one reason may be the reduction of the automaticity of 

operators' performance. When there is a good match between the information available and the operator's 

mental model of the task, performance becomes somewhat automatic, i.e., it requires little attention and 

cognitive resources. When the situation is unclear, more cognitive resources are needed to assess it and to 

plan responses. With spatially dedicated HSIs, operators can use automatic information processing 

capabilities, such as scanning and pattern recognition, to rapidly assess situations. Flexibility and lack of 

spatial dedication increases the dependence of interface management tasks on controlled information 

processing. Flexibility also makes it easier for the operator to mistake one display for another. 

(10) Methods should be specified for assuring that plant modifications (such as changes in instrumentation or 

systems) are incorporated into the display and do not introduce inconsistencies in how they correspond to 

plant situations, or lead to technical inaccuracies and, possibly, invalid displays. 

Discussion: Methods should be specified for assuring that plant changes do not introduce errors or changes 

in the relationship between displays and their representations of plant situations. For example, the 

information system may depend upon specific sensor data to generate a graphic form. To maintain the 

integrity of the information system, it is critical that the implications of changing any plant feature or 

software calculation is traceable and controllable. 

(11) If display formats are developed for a generic plant design or as an "off-the-shelf' product, any plant

specific inputs to display characteristics need to be analyzed to ensure that the display correctly reflects the 

relationship between changes in the display format and the changes in the specific plant it is intended to 

represent. 

Discussion: Plant-specific features may affect the relationship between the appearance of the display format 

and the plant conditions being represented in it. It is important to make sure the intended relationship 

between the two is preserved once plant-specific details are incorporated into the information system. 

9.4 Training Program Development 

(1) The knowledge, skills, and abilities that the operators need to use and understand the information system 

should be specified. 

Discussion: Research on the use of complex displays, such as the Rankine cycle display and DURESS EID 

display, illustrated the importance of training (Brehmer, Hill, and Brehmer, 1995). Itoh, Sakuma, and 

NUREG/CR-6633 9-4

I I
9 DESIGN PROCESS REVIEW GUIDANCE

Monta (1995) found that operators experienced difficulty understanding the plant situations depicted in the 

Toshiba EID displays. These studies demonstrated the importance of evaluating the relationship between 

the representation in the display and the operator's mental model of the domain, on the implications for 

training. 

Where the information is sophisticated and a large amount is consolidated into one or a few graphics, 

training requirements become very important (Itoh, Sakuma and Monta, 1995). Operators need to 

understand all aspects of the display, and how it reacts to various operational transients, accidents, and 

instrument failures. Christoffersen, Hunter, and Vicente (1995) stated that "...to experience the benefits of 

EID, it seems likely that operators need to be trained to think fimctionally rather than procedurally. It would 

seem that this would require a fundamental shift in NPP operation philosophy" (p. 143). They further 

suggest that "...it may be that operators have to possess certain types of cognitive characteristics that may not 

be considered in the traditional selection process in the nuclear industry" (p. 143). 

(2) Operators should be trained on the relationship between the display form and the plant states it is intended 

to represent, including failure modes and their effect on graphical representation. 

Discussion: Where large amounts of information at various levels of abstraction are consolidated into one or 

more graphic formats, training requirements become very important. Operators need to understand all of 

the aspects of the display and how it reacts to various operational transients, accidents, and instrument 

failures. Training should cover design features of the information system, relationships between graphic 

elements and plant status, and the effects of failures. 

(3) Users should be trained in using the interface management features of the information system, including 

navigation within and between displays, manipulation of on-screen features such as windows, and use of 

user-definable characteristics and features. 

Discussion: Successful use of complex information systems depends to a great extent on the user's ability to 

understand the characteristics of the interface and effortlessly use its features (O'Hara et al., 1997). These 

aspects of HSIs are often neglected in training, so they are used inefficiently. 

(4) Users should be trained to an acceptable level of proficiency with unfamiliar graphic displays before using 

them in the control room to ensure that operators are sufficiently familiar with them to correctly assess their 

meaning. 

Discussion: The issue of operators' acceptance of a new and different type of display is important. 

Introducing the displays initially in training and then gradually into the plant will help ensure their 

acceptance and proper use. 

9-5 NUREG/CR-6633
10 REVIEW GUIDELINES FOR THE INFORMATION SYSTEM 

HFE DESIGN 

The guidelines presented in this section follow the characterization of information systems described in Section 4 

of this document. They also reflect the results of the literature review discussed in Section 5. Following the HSI 

design review procedure described in Part 1 of NUREG-0700, Rev. 1, the first step is to select the subset of 

guidelines relevant to the unique aspects of the particular design being evaluated. It is recognized that there is a 

wide range of information system designs, and that some may not include all of the characteristics and functions 

addressed in these guidelines. The reviewer will have to determine, case-by-case, the importance of information 

system features that are covered in the guidelines, but that are not part of the system being reviewed. 

Guidelines were developed from the findings and source materials that were reviewed in Section 5. They are 

constructed in the standard format adopted in NUREG-0700, Rev. 1 (see Section 6.2 of this report). The 

guidelines are organized into the following sections: 

* General Display Guidelines 

* Display Format 

- Integral Formats 

- Display Formats 

These new guidelines for review of information system designs will be integrated with the information system 

guidance in NUREG-0700, Rev. I. 

Guidelines may specify that some identified aspect of an information system needs to be "evaluated." General 

approaches to evaluation methods and criteria development are defined in NUREG-0700. Since the guidance in 

this section will eventually be incorporated into that document, the methods by which such evaluation should be 

performed and by which criteria should be identified are not repeated here. 

10.1 General Display Guidelines 

10.1-1 Correspondence Mapping 

There should be an explicit mapping between the characteristics and functions of the system to be 

represented and the features of the display representation, i.e., changes in the appearance of the display 

form should have a one-to-one relation with the plant states it represents. These changes should result 

from explicit rules relating the physical form of the display and its meaning to the plant state represented. 

ADDITIONAL INFORMATION: Correspondence mapping addresses how well the display 

communicates meaningful information about the plant to operators. The physical form and functions of 

the display should be explicitly tied to the plant's functions and states. The display form and function 

must consider the instrumentation and the data processing that drive the display format. If a single 

display can lead to more than one interpretation, it is ambiguous and can be easily misunderstood. 

Changes in the graphic display should be unambiguously related to the plant's state. The same graphic 

change should not be associated with more than one interpretation. 

Discussion: Woods and Roth (1988) conceptualized displays as involving three essential parts of an 

overall cognitive system triad that includes the world to be acted on, the agents who will act on the world, 

and the representation through which the agents act on the world. The success of a representation is a 

function of two types of mapping (Bennett et al., 1997): correspondence and coherence. (Coherence is 

addressed in Guideline 10.1.3.) The importance of providing consistent one-to-one mapping between the

10- 1 NUREG/CR-6633
10 HFE DESIGN REVIEW GUIDELINES

work domain's constraints and the features of the display has been identified as an important principle 

derived from research on representing complex systems (Bennett, Nagy, and Flach, 1997; Vicente and 

Rasmussen, 1992). 

10.1-2 Levels of Abstraction 

Displays should provide information at the levels of abstraction necessary to meet the operators' 

requirements relative to their task goals. 

ADDITIONAL INFORMATION: Information should be presented in accord with the operator's goals 

and the information needed to address them. 

Discussion: This guideline reflects considerations of the information.processing basis of situation 

awareness support (Endsley, 1988, 1995). In addition, to support knowledge-based behavior, Vicente and 

Rasmussen (1992) suggest that the information system should represent the work domain as an 

abstraction hierarchy to serve as an externalized mental model. Further, to do this, Endsley and others 

(Heslinga and Herbert, 1993 and 1995) noted that consideration of tasks involves more than simply the 

physical layout of the system. 

10.1-3 Coherence Mapping 

The characteristics and features of the display used to represent the process should be readily perceived 

and interpreted by the operator. 

ADDITIONAL INFORMATION: Coherence mapping addresses how comprehensible the representation 

is to the operator. Unambiguous relationships between the display and the process are of little value if 

they also are not readily perceived by the operators and easily understood. 

Discussion: See discussion of Guideline 10. 1-1. 

10.1-4 Understandability of Higher-Level Information 

The methods by which lower-level data are analyzed to produce higher-level information and graphical 

elements should be understandable to users. 

ADDITIONAL INFORMATION: Users must be able to judge the acceptability of higher-level 

information and how it relates to lower-level information. 

Discussion: To maintain their role as system supervisors, operators need to be able to understand and 

evaluate the appropriateness and validity of the information being displayed. 

10.1-5 User Verification of Higher-Level Information 

Operators should have access to the rules or computations that link process parameters and graphical 

features, and to an explanation of how the information system produces higher-level information. 

ADDITIONAL INFORMATION: When graphical features change in ways not completely understood by 

operators, they should be able to access the rules that produce the graphic forms. Operators should be able 

to review any analysis performed by the information system. 

Discussion: To maintain their role as system supervisors, operators need to be able to access information 

enabling them to determine the appropriateness and validity of information displayed. Access to the rules 

and computations that relate the system and the display (the correspondence map) should be available to 

operators to support situation assessment. In general, computer-based systems often are not sufficiently 

observable. That is, they do not make clear their analysis basis and do not have adequate communication 

facilities to enable operators to verify system performance (Dien and Montmayeul, 1995; IAEA, 1994; 

Malin et al., 1992a; Roth, Bennet, and Woods, 1987). 

10.1-6 Salience Levels 

The salience of graphic features should reflect the importance of the information. 

NUREG/CR-6633 10-2

I I
10 HFE DESIGN REVIEW GUIDELINES

ADDITIONAL INFORMATION: The most salient features of a graphic display should be those aspects 

of the representation that are most important. Less important information should not be more 

perceptually salient than more important information. 

Discussion: This guideline reflects considerations of the information processing basis of situation 

awareness support (Endsley, 1988, 1995), and research by Bennett, Nagy, and Flach (1997) on the 

salience of emergent features. Situation assessment depends on activating the appropriate mental 

representation for the situation; thus, the critical cues to such activation should be determined, and 

displays should make these cues salient. 

10.1-7 Display of Goal Status 

The information system should provide for global situation awareness (i.e., an overview of the status of all 

the operator's goals at all times) as well as supplying details'about the current specific goal. 

ADDITIONAL INFORMATION: Situation assessment can suffer when operators focus on some 

information and fail to attend to other important data. 

Discussion: This guideline reflects considerations of the information processing basis of situation 

awareness support (Endsley, 1988, 1995). 

10.1-8 Display of Future Status 

The information system should support the user's ability to project future states of the system when this is 

required to safely operate the plant. 

ADDITIONAL INFORMATION: Situation assessment involves not only understanding the current state 

of the plant, but projecting its future state.. Displays such as trend graphs can support these projections. 

Discussion: This guideline reflects considerations of the information processing basis of situation 

awareness support (Endsley, 1988, 1995). 

10.1-9 Minimize Attention Shifts 

Information needed by the operator to accomplish a task should be grouped and perceptually related, when 

possible. 

ADDITIONAL INFORMATION: To minimize the disadvantages of divided attention, the number of 

attention shifts should be minimized, both within a display page and between them. 

Discussion: This guideline reflects considerations of the information processing basis of situation 

awareness support (Endsley, 1988, 1995). Interface management tasks necessary to retrieve displays and 

to shift between them are distracting, and can degrade performance. Such distractions should be 

minimized. 

10.1-10 Analytical Redundancy 

Analytical redundancy should be considered to help ensure the appropriateness of displayed values. 

ADDITIONAL INFORMATION: Analytical redundancy is the calculation of expected parameter values 

using a model of system performance. The expected value is then represented in the display, along with 

the actual value. Deviations between the two indicate some disturbance or abnormality of the system. 

Discussion: Vicente and Wang (1996) found that fault detection time was reduced and the ability of study 

participants to compensate for the fault was improved when a display had analytical redundancy. 

10.1-11 Failure Recognition 

Information system failures (due to sensors, instruments, and components) should result in distinct display 

changes which directly indicate that depicted plant conditions are invalid. 

ADDITIONAL INFORMATION: The information system should be designed so that failures in 

instrumentation are readily recognized by operators. This may be more difficult to determine in complex 

graphics, and thus, should be carefully evaluated.

10-3 1 NUREGICR-6633
10 HFE DESIGN REVIEW GUIDELINES

Discussion: Moray et al. (1994) noted that graphical interfaces are "extremely vulnerable" to this type of 

failure and that understanding their failure modes is as necessary as understanding their design for normal 

conditions. 

10.1-12 Navigational Links to Related Information 

Navigational links to and from high-level and lower-levels of information and to reference and supporting 

information should be provided when needed for operators' tasks. 

Discussion: Navigation to and from related information can be time consuming, distracting, and error 

prone. Computer support, such as hyperlinks, for these activities can reduce the workload. 

10.2 Display Format 

10.2.1 Integral Formats 

10.2.1-1 Appropriate Use of Integral Displays 

Integral formats should be used to communicate high-level, status-at-a-glance information where users 

may not need information on individual parameters to interpret the display. 

ADDITIONAL INFORMATION: Since integral displays do not display individual parameters, they are 

most appropriate for general status monitoring. 

Discussion: NPP operators almost always need information on individual parameters, except for general 

monitoring. Integral displays best serve this purpose because they do not show individual parameters. 

10.2.1-2 Reference Aids for Object Displays 

A perceptually distinct reference aid should be provided in an object display to support operators in 

recognizing abnormalities in the object's characteristics. 

ADDITIONAL INFORMATION: When a change in an object's characteristics (e.g., its shape) is the 

perceptual feature that indicates a fault or abnormal condition, perceptual cues can assist operators in 

detecting the change. If shape is used, the graphic display should include the normal reference point to 

which operators can compare the current shape. Reference points are especially useful when the 

abnormality is slow to evolve, and the integral object is changing slowly. 

Discussion: There is little research guidance on how much a graphic form must change to be perceptually 

recognized. Thus, if a referent is not present, it may be difficult to ensure that a change is recognized. 

Recognition is supported by including the comparison referent. 

10.2.2 Configural Display Formats 

10.2.2-1 Appropriate Use of Configural Displays 

Configural formats should be used when operators must rapidly transition between high-level functional 

information and specific parameter values. 

ADDITIONAL INFORMATION: Configural displays provide lower-level information, such as parameter 

values, and higher-level information conveyed through emergent features. Since both are present in a 

single display, operators can easily move between them. 

Discussion: The value of configural features is their ability to convey multiple levels of information in a 

single display. Therefore, when operators must be cognizant of more than one level of abstraction of 

information, configural displays are preferred. 

NUREG/CR-6633 10-4

I I
10 HFE DESIGN REVIEW GUIDELINES

10.2.2-2 Representation of Emergent Features 

The display elements should be organized so that the emergent features that arise from their interaction 

correspond to-meaningful information about the process or system, e.g., when the aspect of the system 

represented by the emergent is disturbed, the disturbance is visible in the emergent feature. 

ADDITIONAL INFORMATION: An emergent feature is a high-level, global perceptual feature 

generated by interactions among individual parts or graphical elements of a display (e.g., lines, contours, 

and shapes) to produce perceptual properties, such as symmetries, closure, and parallelism. Displays 

cannot always be organized to provide emergent features, but they should be considered where feasible. 

Discussion: This guideline is was developed by Bennett, Nagy, and Flach (1997) from reviewing research 

on configural displays. 

10.2.2-3 Levels of Emerging Features 

The emergent features or patterns within the display should be nested (from global to local) in a way that 

reflects the hierarchical structure of the process. 

ADDITIONAL INFORMATION: High-order aspects of the process (e.g., at the level of functional 

purpose or abstract function) should be reflected in global display features; lower-order aspects of the 

process (e.g., functional organization) should be reflected in local display features. 

Discussion: This guideline is was developed by Bennett, Nagy, and Flach (1997), who reviewed research 

on configural displays. 

10.2.2-4 Salience of Emerging Features 

Each emergent feature should be clearly distinguishable for other emergent features and from information 

on individual parameters. 

Discussion: This guideline is was developed by Bennett, Nagy, and Flach (1997) based on a review of 

research on configural displays. 

10.2.2-5 Reference Aids for Configural Displays 

A perceptually distinct reference aid should be provided in a configural display to support operators in 

recognizing abnormalities in emergent features. 

ADDITIONAL INFORMATION: When a change in an object's characteristics (e.g., its shape) is the 

perceptual feature that indicates a fault or abnormal condition, perceptual cues can assist operators in 

detecting the change. If shape is used, the display graphic should include the normal reference point 

against which operators can compare the current one. Reference points are especially useful when the 

abnormality is slow to evolve, and the integral object is slowly changing. 

Discussion: See discussion of Guideline 10.2.1-2. 

10.2.2-6 Representation of Individual Parameters 

Each relevant process parameter should be represented by a perceptually distinct element within the 

display. 

Discussion: Since configural displays provide both high- and low-level information, all relevant lower

level parameters should be given. This will minimize the need for operators to retrieve supplemental 

information directly relevant to the configural display from another one. 

10.2.2-7 Use of Lower-Level Information 

The display should support the user in performing tasks requiring lower-level information. 

ADDITIONAL INFORMATION: When the operator must perform tasks using lower level information, 

the display should provide such support. For example, if precise information about a variable is desirable, 

then a scale or digital information should be provided.

10-5 NUREG/CR-6633
10 HFE DESIGN REVIEW GUIDELINES

Discussion: Providing information to support detailed use of lower-level information will minimize the 

chance for error in deriving the lower-level information from the display, and minimize the need to 

retrieve supplemental information from another display. 

10.2.2-8 Complexity 

The emergent features and their interactions should not be so complex as to be susceptible to 

misinterpretation. 

ADDITIONAL INFORMATION: The value of emergent features is that they provide a direct perception 

of higher-level information. They substitute perception for mental calculation. The shift toward 

perceptual cognition requires careful design, so that misunderstandings are unlikely to occur. 

Discussion: Configural displays can get complex and can be misunderstood by operators (Itoh, Sakuma, 

and Monta, 1995).

NUREG/CR.•6633 10-6
GLOSSARY 

Analytical Redundancy - The calculation of expected parameter values using a model of system performance. 

Coherence Mapping - A map between the features in the representation and the physical and cognitive 

characteristics of the operator (how comprehensible the representation is to the operator). 

Configural Display - A display in which information dimensions are uniquely represented, but where new 

emergent properties are created from interactions between the dimensions. Configural display representations 

often use simple graphic forms, such as a polygon. 

Correspondence Mapping - A map between the properties and characteristics of the system to be represented and 

the features in the representation (how well the display communicates meaningful information about the plant to 

operators). 

Display Devices - The media used to present information to personnel, including meters, gauges, VDUs, and hard

copy display devices (printers and plotters). 

Display Element - The basic building blocks used to make up display formats, such as abbreviations, labels, icons, 

symbols, coding, and highlighting. 

Display Format - The general class of information presentation. Examples of general classes are continuous text 

(such as a procedure display), mimics and piping and instrumentation diagram (P&ID) displays, trend graphs, and 

flowcharts. 

Display Network - An group of display pages within an information system and their organizational structure. 

Display Page - A defined set of information that is intended to be displayed as a single unit. Typical NPP display 

pages may combine several different formats on a single VDU screen, such as putting bar charts and digital 

displays in a graphic P&ID format. Display pages typically have a label and designation within the computer 

system so they can be accessed by operators as a single "display." 

Emergent Feature - A high-level, global perceptual feature produced by the interactions among individual parts or 

graphical elements- of a display (e.g., lines, contours, and shapes). 

Information System - Those aspects of the HSI that provide information on the plant's systems to the operator. 

Integral Display - A display that depicts the integration of information in such a way that the individual 

parameters used to generate the display are not explicitly represented in it. 

Level ofAbstraction - A hierarchy consisting of levels increasing in abstraction: 

- Physical form - the appearance and spatial location of the components 

- Physical function - the characteristics of the components and their interconnections 

- Generalized function - the basic functions a system was designed to achieve 

- Abstract function - the causal structure of the process in terms of mass, energy, information or value flows - Functional purpose - the purpose for which the system was designed; the functional characteristics of the 

plant as opposed to physical characteristics.

G-1I NUREG/CR-6633
GLOSSARY

Object Display - A type of integral display that uses a geometric object to represent parameter values graphically, but where the individual information dimensions or data contributing to the object are not displayed. 

Separable Display - Each process parameter is presented individually and no relationships between the parameters 

are shown by the representation itself. The key aspect of separable displays is not that individual parameters are 

presented, but that no interaction or relationship between them is perceived.

NUREG/CR-6633 G-2

I I
NRC FORM 335 U.S. NUCLEAR REGULATORY COMMISSION 1. REPORT NUMBER 

(2-89) (Assigned by NRC. Add VoL, Supp., Rev., 

NRCM 1102. and Addendum Numbers, if any.) 

3201. 3202 

BIBLIOGRAPHIC DATA SHEET NUREG/CR-6633 

(See instructions on, reverse) BNL-NUREG-52563 

Z TITLE AND SUBTITLE 

Advanced Information Systems Design: Technical Basis and Human Factors Review 3. DATE REPORT PUBLISHED 

Guidance MONTH YEAR 

March 2000 

4. FIN OR GRANT NUMBER 

J-6012 

S. AUTHOR(S) 6. TYPE OF REPORT 

John M. O'Hara, James C. Higgins, and Joel Kramer Technical 

7. PERIOD COVERED (Indusive Dates) 

8. PERFORMING ORGANIZATION -NAME AND ADDRESS (CNRC, provid Division, Office or Region, U.S. Nuclear Regulatory Commission, and mailing address; if contractor, provide 

name and mailing address.) 

Department of Advanced Technology 

Brookhaven National Laboratory 

Upton, NY 11973-5000 

9. SPONSORING ORGANIZATION -NAME AND ADDRESS (if NRC, type 'Same as above'; if contractor, provide NRC Division, Office or Region, U.S. Nuclear Regulatory Commission, 

and maifing address.) 

Division of System Analysis and Regulatory Effectiveness 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

10. SUPPLEMENTARY NOTES 

J. Kramer, NRC Project Manager

11. ABSTRACT (200 words orless) 

Plant information systems provide operators with information supporting situation assessment, monitoring and detection of 

disturbances, response planning, and response implementation. The importance of the design of information systems for human 

performance and reliability has long been recognized. Recent advances in design go beyond the 'one sensor - one indicator' 

display systems of conventional plants. Computer-based systems provide a variety of ways to process and present data. These 

characteristics of design represent a trend toward making displays more immediately meaningful to personnel by mapping display 

representations to important aspects of plant processes and to underlying cognitive mechanisms, such as perceptual processes and 

mental models. The objective of this study was to develop guidance for human factors review of advanced information systems 

based on a technically valid methodology for developing guidance. To support this objective, we developed a characterization 

framework for describing key design characteristics of information systems. The characterization includes the following major 

components: information requirements, representation systems, interface management functions, and display devices. 

Representation systems were further hierarchically divided into display elements (basic building blocks of displays, such as axes, 

alphanumeric elements, and icons), display formats (such as mimic displays, configural displays, and novel graphics), display 

pages (a defined set of information that is intended to be displayed as a single unit), and display networks (the organization of 

display pages). Then, we examined research in the following areas: (1) generic cognitive tasks that an information system must 

support, (2) information requirements analysis, (3) information representation, and (4) information organization. This research was 

used to provide the technical basis on which guidelines for review of design were developed. These guidelines address both the 

design process and the implementation of advanced information systems. However, there were aspects of information systems for 

which the technical basis was insufficient to support the development of guidance. These were identified as issues to be addressed 

in future research.

12 KEY WORDS/DESCRIPTORS (List vwords or phrases that vwg assist researchers in locating the report.) 13. AVAILABILITY STATEMENT 

Control rooms, human factors engineering, human-system interface, man-machine systems, Unl.imied 

reactor safety, reactor operators, test and evaluation, human-factors review criteria 14. SECURITY CLASSIFICATION 

(This Page) Unclassified 

(This Report) Unclassified 

15 NUMBER OF PAGES 

16. PRICE

NRC FORM 335 (2-89)
Federal Recycling Program
ADVANCED INFORMATION SYSTEMS DESIGN: TECHNICAL BASIS AND 

HUMAN FACTORS REVIEW GUIDANCE

MARCH 2000

UNITED STATES 

NUCLEAR REGULATORY COMMISSION 

WASHINGTON, D.C. 20555-0001

SPECIAL STIANDARD MAIL 

POSTAGE AND FEES PAID 

USNRC 

PERMIT NO. G-67

NUREG/CR-6633