Cognitive Basis for Human

Reliability Analysis

Office of Nuclear Regulatory Research

NUREG-2114
AVAILABILITY OF REFERENCE MATERIALS

IN NRC PUBLICATIONS

NRC Reference Material

As of November 1999, you may electronically access 

NUREG-series publications and other NRC records at 

NRC’s Library at www.nrc.gov/reading-rm.html. Publicly 

released records include, to name a few, NUREG-series

publications; Federal Register notices; applicant, 

licensee, and vendor documents and correspondence; 

NRC correspondence and internal memoranda; bulletins 

and information notices; inspection and investigative 

reports; licensee event reports; and Commission papers 

and their attachments.

NRC publications in the NUREG series, NRC regulations, 

and Title 10, “Energy,” in the Code of Federal Regulations

may also be purchased from one of these two sources.

1. The Superintendent of Documents

U.S. Government Publishing Office

Mail Stop IDCC

Washington, DC 20402-0001

Internet: bookstore.gpo.gov

Telephone: (202) 512-1800

Fax: (202) 512-2104

2. The National Technical Information Service

5301 Shawnee Rd., Alexandria, VA 22312-0002

www.ntis.gov

1-800-553-6847 or, locally, (703) 605-6000

A single copy of each NRC draft report for comment is

available free, to the extent of supply, upon written

request as follows:

Address: U.S. Nuclear Regulatory Commission

Office of Administration

Publications Branch

Washington, DC 20555-0001

E-mail: distribution.resource@nrc.gov

Facsimile: (301) 415-2289

Some publications in the NUREG series that are posted 

at NRC’s Web site address www.nrc.gov/reading-rm/

doc-collections/nuregs are updated periodically and may 

differ from the last printed version. Although references to 

material found on a Web site bear the date the material 

was accessed, the material available on the date cited 

may subsequently be removed from the site.

Non-NRC Reference Material

Documents available from public and special technical 

libraries include all open literature items, such as books, 

journal articles, transactions, Federal Register notices, 

Federal and State legislation, and congressional reports. 

Such documents as theses, dissertations, foreign reports 

and translations, and non-NRC conference proceedings 

may be purchased from their sponsoring organization.

Copies of industry codes and standards used in a

substantive manner in the NRC regulatory process are 

maintained at—

The NRC Technical Library

Two White Flint North

11545 Rockville Pike

Rockville, MD 20852-2738

These standards are available in the library for reference 

use by the public. Codes and standards are usually 

copyrighted and may be purchased from the originating 

organization or, if they are American National Standards, 

from—

American National Standards Institute

11 West 42nd Street

New York, NY 10036-8002

www.ansi.org

(212) 642-4900

Legally binding regulatory requirements are stated only in 

laws; NRC regulations; licenses, including technical specifications; or orders, not in NUREG-series publications. The 

views expressed in contractorprepared publications in this 

series are not necessarily those of the NRC.

The NUREG series comprises (1) technical and administrative reports and books prepared by the staff (NUREG–

XXXX) or agency contractors (NUREG/CR–XXXX), (2) 

proceedings of conferences (NUREG/CP–XXXX), (3) reports 

resulting from international agreements (NUREG/IA–XXXX), 

(4) brochures (NUREG/BR–XXXX), and (5) compilations of 

legal decisions and orders of the Commission and Atomic 

and Safety Licensing Boards and of Directors’ decisions 

under Section 2.206 of NRC’s regulations (NUREG–0750).

DISCLAIMER: This report was prepared as an account 

of work sponsored by an agency of the U.S. Government. 

Neither the U.S. Government nor any agency thereof, nor 

any employee, makes any warranty, expressed or implied, 

or assumes any legal liability or responsibility for any third 

party’s use, or the results of such use, of any information, 

apparatus, product, or process disclosed in this publication, 

or represents that its use by such third party would not 

infringe privately owned rights.
Cognitive Basis for Human

Reliability Analysis

Manuscript Completed: June 2015

Date Published: January 2016

Prepared by:

April M. Whaley,1 Jing Xing2

Ronald L. Boring,1 Stacey M. L. Hendrickson,3 

Jeffrey C. Joe,1 Katya L. Le Blanc,1 Stephanie L. Morrow2 

1Idaho National Laboratory

2U.S. Nuclear Regulatory Commission

3Sandia National Laboratories

Erasmia Lois and Jing Xing, NRC Project Managers 

Office of Nuclear Regulatory Research

NUREG-2114

 

iii

ABSTRACT

This report documents the results of a literature review to synthesize human cognition research

into a technical basis for human reliability analysis. The U.S. Nuclear Regulatory Commission 

(NRC) organized a team of researchers to review literature in psychology, cognition, behavioral 

science, and human factors and apply it to human performance in nuclear power plant 

operations. The project team synthesized the results into a cognitive framework that consists of 

five macrocognitive functions: (1) detecting and noticing, (2) understanding and sensemaking, 

(3) decisionmaking, (4) action, and (5) teamwork. For each macrocognitive function, the team 

identified proximate causes for why the cognitive function may fail, cognitive mechanisms 

underlying the failures, and factors that influence the cognitive mechanisms and may lead to 

human performance errors. Moreover, the project team used the information in the literature to 

infer causal relationships and links between different types of human failures and factors that 

influence human performance. This report provides a cognitive basis for human performance 

and a structured framework to assess how human performance may contribute to errors in the 

context of an evolving event scenario. The information can serve as the technical foundation for 

the NRC’s human reliability analysis methods and human factors engineering guidance.

v

TABLE OF CONTENTS

ABSTRACT.....................................................................................................................................iii

TABLE OF CONTENTS...................................................................................................................v

LIST OF FIGURES .........................................................................................................................ix

LIST OF TABLES............................................................................................................................xi

EXECUTIVE SUMMARY.............................................................................................................. xiii

ACKNOWLEDGMENTS ...............................................................................................................xix

ACRONYMS AND ABBREVIATIONS..........................................................................................xxi

1. INTRODUCTION................................................................................................................... 1

1.1 Overview of the Literature Review .............................................................................. 2

1.1.1 Developing a Technical Basis for IDHEAS ..................................................... 2

1.1.2 Scope and Goals of the Literature Review ..................................................... 3

1.1.3 Literature Review Process............................................................................... 5

1.2 Overview of this Report ............................................................................................... 6

2. MACROCOGNITION............................................................................................................. 9

2.1 Introduction .................................................................................................................. 9

2.2 Overview of Macrocognition ...................................................................................... 10

2.2.1 Macrocognitive Functions by Klein et al........................................................ 11

2.2.2 The Overlapping Macrocognitive Function Model ........................................ 12

2.3 Macrocognition for NPP Operations.......................................................................... 14

2.3.1 IDA and IDAC ................................................................................................ 14

2.3.2 O’Hara’s Model of NPP Operator Generic Tasks ......................................... 17

2.3.3 Integrating Macrocognition with NPP Model................................................. 20

2.4 The Macrocognition Model for HRA.......................................................................... 21

2.4.1 Definitions of the Macrocognitive Functions Used in this Approach............. 21

2.4.2 Relationship between the Macrocognitive Functions.................................... 23

2.5 Development of the Cognitive Framework................................................................ 24

2.5.1 Identification of Cognitive Mechanisms......................................................... 25

2.5.2 Identification of Proximate Causes................................................................ 26

2.5.3 Performance Influencing Factors .................................................................. 27

2.5.4 Creating the Cognitive Framework: Connecting Proximate Causes, 

Cognitive Mechanisms, and PIFs.................................................................. 28

2.6 Summary.................................................................................................................... 31

3. DETECTING AND NOTICING............................................................................................ 33

3.1 Introduction ................................................................................................................ 33

3.1.1 Sensation and Perception ............................................................................. 33

3.1.2 Detecting and Noticing across Sensation and Perception............................ 33

3.2 Cognitive Mechanisms for Detecting and Noticing ................................................... 36

3.2.1 Behavioral Aspects of Detecting and Noticing.............................................. 36
vi

3.2.2 Overview of Mechanisms for Detecting and Noticing ................................... 37

3.2.3 Cognitive Mechanisms Involved in Detecting and Noticing.......................... 38

3.3 Proximate Causes for Detecting and Noticing .......................................................... 40

3.4 Performance Influencing Factors impacting Detecting and Noticing........................ 43

3.5 Summary.................................................................................................................... 44

4. UNDERSTANDING AND SENSEMAKING ........................................................................ 47

4.1 Introduction ................................................................................................................ 47

4.1.1 Overview of the Understanding and Sensemaking Function........................ 47

4.1.2 Understanding and Sensemaking with Procedures in the NPP 

Domain........................................................................................................... 48

4.1.3 An Overview of the Dynamic Data/Frame Theory of Sensemaking............. 52

4.1.4 Working Memory and the Neurological Basis of Sensemaking.................... 53

4.2 Major Cognitive Models of Sensemaking.................................................................. 54

4.2.1 Detailed Discussion of the Data/Frame Theory of Sensemaking................. 55

4.2.2 The Product of Sensemaking: Situation Awareness..................................... 58

4.2.3 The Proximate Causes for the Understanding and Sensemaking 

Function ......................................................................................................... 65

4.2.4 Cognitive Mechanisms for Proximate Causes .............................................. 66

4.2.5 The Effects of PIFs on Understanding and Sensemaking............................ 69

4.3 Summary.................................................................................................................... 76

5. DECISIONMAKING............................................................................................................. 77

5.1 Introduction ................................................................................................................ 77

5.2 Historical Background of Decisionmaking................................................................. 77

5.3 Naturalistic Decisionmaking ...................................................................................... 78

5.4 Decisionmaking in a Nuclear Power Plant ................................................................ 80

5.5 Proximate Causes of Failure of Decisionmaking ...................................................... 82

5.6 Summary.................................................................................................................... 84

6. ACTION ............................................................................................................................... 85

6.1 Introduction ................................................................................................................ 85

6.2 Cognitive Mechanisms of Action Failure................................................................... 86

6.2.1 Overview of Psychological Causes of Action Errors..................................... 86

6.2.2 Overview of Action Execution in the Brain .................................................... 89

6.2.3 Summary of Cognitive Mechanisms Underlying Action Failures.................. 92

6.3 Identification of Proximate Causes............................................................................ 93

6.4 Summary.................................................................................................................... 94

7. TEAMWORK ....................................................................................................................... 95

7.1 Introduction ................................................................................................................ 95

7.1.1 Teamwork in NPP Control Rooms ................................................................ 95

7.1.2 The Emergent Aspect of Teams.................................................................... 96

7.2 Models and Theories on Teamwork.......................................................................... 96

7.2.1 Research on NPP Crews............................................................................... 96

7.2.2 Team Errors: Definition and Taxonomy by Sasou and Reason (1999)........ 98

7.2.3 Crew Resource Management........................................................................ 99
vii

7.2.4 Team Sensemaking..................................................................................... 100

7.2.5 Macrocognition in Teams ............................................................................ 101

7.3 Cognitive Mechanisms and Proximate Causes of Failure of Teamwork................ 103

7.3.1 Communication............................................................................................ 104

7.3.2 Leadership ................................................................................................... 106

7.4 Summary.................................................................................................................. 107

8. REFERENCES.................................................................................................................. 109

Appendix A Cognitive Mechanism Tables...................................................................................A-1

Appendix B Cognitive Framework Diagrams ..............................................................................B-1

ix

LIST OF FIGURES

Figure 2-1. Macrocognitive functions and supporting processes for individuals and 

teams (Klein, et al., 2003)........................................................................................ 11

Figure 2-2. The overlapping function model of macrocognition (Patterson & Hoffman, 

2012). ....................................................................................................................... 13

Figure 2-3. IDAC operator cognitive flow model........................................................................ 15

Figure 2-4. Nested IDA structure concept.................................................................................. 17

Figure 2-5. O’Hara’s cognitive model of NPP control room operations, adapted from 

O’Hara et al. (2008). ................................................................................................ 18

Figure 2-6. The five macrocognitive functions........................................................................... 24

Figure 3-1. The relationship between the macrocognitive functions of Detecting and 

Noticing and Understanding and Sensemaking...................................................... 34

Figure 3-2. Diagram of cognitive processes for the Detecting and Noticing function. .............. 37

Figure 4-1. High-level overview of sensemaking....................................................................... 50

Figure 4-2. The sensemaking process (Klein, et al., 2006)....................................................... 53

Figure 4-3. Cognitive processes behind sensemaking (Xing et al., 2011)................................ 54

Figure 4-4. A more detailed view of the sensemaking process (Klein, et al., 2007)................. 57

Figure 4-5. Endsley’s model of situation awareness. ................................................................ 59

Figure 4-6. The Perceptual Cycle model of situation awareness (Salmon, et al., 2008; 

Smith & Hancock, 1995).......................................................................................... 62

Figure 4-7. The Functional Model of Orienting Activity (Bedny & Karwowski, 2004; 

Bedny, et al., 2004; Bedny & Meister, 1999)........................................................... 64

Figure 5-1. Integrated version of recognition-primed decision model (Klein, 1993). ................ 80

Figure 5-2. Integrated NDM model (Greitzer, et al., 2010)........................................................ 81

Figure 6-1. Pathways for motor execution. ................................................................................ 89

Figure 7-1. Sasou & Reason error taxonomy with mapping to PSFs........................................ 99

Figure 7-2. The CRM input-process-output model of crew performance................................ 100

Figure 7-3. Macrocognitive model of team collaboration (Letsky, et al., 2007)....................... 102

Figure 7-4. Knowledge building process within team macrocognition (Fiore, Rosen, et 

al., 2010). ............................................................................................................... 103

xi

LIST OF TABLES

Table 1-1. Search terms used in literature review...................................................................... 6

Table 2-1. Comparison of the macrocognitive and NPP models. ............................................ 20

Table 2-2. Complete PIF taxonomy as organized by Groth & Mosleh (2012). ........................ 28

Table 2-3. Excerpt from Table A.1.1 listing cognitive mechanisms and PIFs for the 

proximate cause Cues/information not perceived................................................... 30

Table 3-1. Key literature according to cognitive mechanisms for Detecting and 

Noticing. ................................................................................................................... 38

Table 3-2. Explanations and examples for the proximate causes of the Detecting and 

Noticing function ...................................................................................................... 42

Table 4-1. Cognitive mechanisms for the proximate causes. .................................................. 69

Table 7-1. Team sensemaking behavioral markers (Klein, Wiggins, and Dominguez, 

2010). ..................................................................................................................... 101

Table 7-2. Communication error types (Lee, Ha, and Seong, 2011). .................................... 105

xiii

EXECUTIVE SUMMARY

The U.S. Nuclear Regulatory Commission (NRC) sponsored and led the work documented in 

this report as part of a larger project to update the technical basis for human reliability analysis 

(HRA), and develop new HRA methods to address limitations of existing methods. The NRC 

frequently uses HRA results and insights to support risk-informed regulatory decisionmaking, 

and HRA is an important component of probabilistic risk assessment (PRA). As a result, the 

NRC has a vested interest in the quality of HRA and continues to seek to improve the 

robustness of HRA. 

One key step toward improving HRA is to build an HRA method based on state-of-the-art 

knowledge about human performance and human errors. Therefore, the NRC project team 

conducted an extensive review and synthesis of literature in psychology, cognition, behavioral 

science, and human factors to consolidate current understanding of human performance in 

operating environments. The ultimate goal of the literature study was to develop a technical 

basis for understanding human errors in NPP internal, at-power events to support the 

development of a new HRA method called the Integrated Human Event Analysis System 

(IDHEAS).

The project team reviewed thousands of scientific papers and technical reports spanning from 

laboratory studies, simulations of task performance by operating crews, and field studies in 

nuclear and non-nuclear (e.g., aviation, oil production) domains. The team then developed a 

cognitive framework to organize the relevant results from the literature. The framework 

assumes that human tasks are accomplished through five macrocognitive functions: (1)

Detecting and Noticing, (2) Understanding and Sensemaking, (3) Decisionmaking, (4) Action, 

and (5) Teamwork. For each function, the team synthesized the information about proximate 

causes for why the function may fail, cognitive mechanisms underlying the failures, and factors 

that influence the cognitive mechanisms and may lead to human performance errors. 

Overview of the Literature Review

The first stage of the literature review focused on identifying research related to operator 

performance in internal, at-power NPP events, in which experienced crews are trained to use

operating procedures to perform tasks. Since NPPs are operated by crews—groups who work 

together, are highly trained, supervised, and who work in a regulated, procedure-driven 

environment—the review focused on literature related to both individual and team aspects of 

human performance, performance when working with procedures, and research related to 

working with highly trained or expert personnel, rather than novices.

After reviewing various models of human cognition in complex supervised tasks, we adapted a 

cognitive model for NPP tasks. The model assumes that cognitive tasks are achieved through 

five macrocognitive functions: (1) Detecting and Noticing, (2) Understanding and Sensemaking, 

(3) Decisionmaking, (4) Action, and (5) Teamwork. We then reviewed information about how 

each of the functions are processed, how the function fails, and what leads to the failure. The 

following terms are used throughout this report to describe the various elements of the cognitive 

model:

Macrocognitive functions – These are the high-level functions through which a cognitive task 

is accomplished. The macrocognitive functions work together in a loop; these functions 

relate to and support each other, and they share some common cognitive mechanisms, yet 

each function is achieved by a unique set of cognitive mechanisms and can lead to unique 

types of human errors. 
xiv

Cognitive mechanisms – Cognitive mechanisms are the processes by which macrocognitive 

functions work. They are the processes by which cognition takes place in the work 

environment, and are thus crucial to successful performance. If part of the process fails, this 

failure may manifest itself as a macrocognitive function failure (e.g., a decisionmaking

failure). To understand why humans make cognitive errors, it is necessary to understand 

how human cognition successfully operates. In other words, it is important to identify the 

processes or mechanisms by which a cognitive function works. Some psychological 

literature may discuss these processes on a behavioral or descriptive level, while other 

psychological research may provide a more profound understanding of the contingencies 

and boundary conditions with which humans can reliably perform a function. 

Proximate causes – A macrocognitive function can fail in many ways due to failure of its

various cognitive mechanisms. We identified the outcomes when a cognitive mechanism

fails, and grouped the outcomes into types of failures. We refer to these types of failures as 

the proximate causes. While cognitive mechanisms are scientific findings described in the 

literature, the set of proximate causes is merely a classification scheme for grouping 

cognitive mechanisms. There can be different ways to classify the mechanisms. Our goal 

was to develop a defined set of proximate causes that have distinct non-overlapping 

definitions, and are observable, identifiable, or inferable in a practical manner. 

Performance influencing factors (PIFs) – These are circumstances or contextual factors that 

contribute to human performance in a work environment. They are commonly used in HRA 

methods to quantify human error probabilities. They are also used to identify root causes of 

errors and areas for improvement. Typical PIFs in HRA include task complexity, available 

time for performance, human-system interfaces, procedures, stress, etc. To understand 

how PIFs influence task performance and contribute to human errors, we identified the PIFs 

relevant to every macrocognitive function and compiled some example studies that 

demonstrated the effects of certain PIFs on the success or failure of the macrocognitive 

functions. 

We organized information about each of the macrocognitive functions into separate chapters of 

this report. Together the information in these chapters forms a structured understanding of 

human errors. Since the information is largely taken from cognitive and psychological literature,

it is generic to human performance. However, the scope of the review was limited to research 

relevant to NPP control room tasks in internal, at-power events. Research that did not fit one or 

more assumptions of the tasks (e.g., distributed decisionmaking, action execution without 

procedures) was not fully covered or integrated into this framework. 

Stage two of the review was to infer links between the cognitive mechanisms identified and the 

relevant performance influencing factors. However, the information needed to make these 

connections was not always readily available in the literature. We established these links 

through analysis and inference from the information available in the literature. For every 

cognitive mechanism under each proximate cause, we link it to known cognitive theories or 

models explaining how the mechanism works. We then went through known performance 

influencing factors to analyze whether the factor affects the mechanism. Furthermore, we listed 

some examples of how the factor affects the mechanism. The information about the linkages is 

documented in the tables in Appendix A. HRA analysts should use these tables as a tool to 

determine types of potential failures in a human event and relevant factors that contribute to the 

failure. Note that due to the incompleteness in the literature and limitations in our review, we 

had to infer many of the relationships in the tables based on our understanding of the existing 

information. The inferences are considered expert judgment and thus have inherent 

subjectivity. In the future, information in the tables should be validated and updated with new 

findings in cognitive research. 
xv

Overview of Macrocognition

Literature on human information processing and human performance spans many research 

domains such as cognitive psychology, cognitive neuroscience, psychophysics, behavioral 

science, and human factors. The level of detail at which each research effort addresses human 

information processing also varies from neural responses, cognitive system activities, to 

behavioral measures. Macrocognition is a term to describe cognition in real-world settings. It 

focuses on the nature of human performance “in the field,” where decisions are often very 

complex, have to be made quickly, by domain experts, in risky or high-stakes situations. That 

is, macrocognition focuses on what humans do with their brains, rather than on the 

fundamentals of how the brain works. Consequently, macrocognition is a useful way to model

human cognition in HRA because it is at a sufficiently high level so that it is practical to use in

predictive analyses. At the same time, it is also detailed enough to yield a coherent 

understanding of how humans perform tasks and what makes humans fail to perform the tasks. 

By synthesizing existing macrocognitive models, we adapted a macrocognitive model for human 

performance in NPP operations. In the model, a cognitive task is accomplished through the 

following five macrocognitive functions: 

Detecting and Noticing is the function to detect and become aware of important information in 

the work environment. This macrocognitive function allows humans to perceive large amounts 

of information and focus selectively on those pieces of information that are pertinent to present 

activities. The cognitive processes associated with Detecting and Noticing include sensing, 

attending, perceiving, and recognizing the key information. Sensation has a large capacity for 

input, but sensory information can only be retained for a short interval before being replaced by 

new sensory information. Moreover, a person can only attend to a limited amount of information 

at a time. Attention determines which pieces of sensed information are processed and enter 

into the human’s awareness. Raw information must be filtered or selected, and meaningful 

information is processed and extracted for further cognitive processing. When there is too much 

meaningful sensory information, the individual may not be able to detect and notice all of that 

information, resulting in sensory overload. Conversely, a lack of salient sensory information 

may cause important plant information to go undetected or unnoticed. Finally, individuals’ past 

experiences and training may affect the meaning associated with a particular percept. That is, 

the raw sensory stream is imbued with meaning, thus meaning is perceived and is subject to the 

cognition of the individual beholding it. Therefore, the outcome of Detecting and Noticing is 

determined by the physical sensory inputs, the process of filtering and perceiving information, 

and the memory that recognizes the meanings of the information. 

Understanding and Sensemaking is the function to bind the meaning of individual pieces of 

information that has been detected and form a coherent understanding of the information. This 

function allows people to question what is known, evaluate what is conjectured, hypothesize 

and diagnose, and integrate facts with theories. The outcome of understanding can be situation

awareness, evaluation, diagnosis, and resolution of conflicts. The process of achieving this 

function involves interpreting pieces of information with existing mental models of the 

information, integrating the information, and generating the output of understanding. The 

central theme of the process is its dynamic aspect (i.e., the process iterates until a satisfactory 

outcome is achieved). This can be best explained by Klein’s data-frame theory. The theory 

posits that information coming into the sensemaking process is data. This data is integrated 

with an existing frame, which is a mental representation that links data with other elements to 

explain and describe the relationship of the data with other entities. A frame encompasses the 

concepts of a mental representation, a mental model, a story, a map, a schema, a script, or a 

plan, and serves as a structure for explaining the data and guiding the search for more data. 
xvi

The data identify or construct the frame, and the frame determines which data are attended to. 

Neither the data nor the frame comes first; rather, it is an iterative process. 

Decisionmaking is to make decisions or plan action scripts to achieve given task goals. The 

process involves managing goals, planning, re-planning and adapting, evaluating options, and 

selecting an option as the final decision outcome. This process describes how an experienced 

operator makes a decision with or without explicit procedures. Decisionmaking within an NPP is 

characterized as involving experts and being largely driven by procedures. Yet, although 

procedures usually dictate the actions of the operators, operators still maintain a mental model 

of the situation and plan their course of action while adhering to procedures. That is, they will 

have an idea of what it is that must be accomplished and how that should be done and will look 

to the procedures to confirm these beliefs. Furthermore, situations may arise that procedures 

do not cover. In these instances, operators must rely on their expert knowledge to solve the 

problem and implement the appropriate decision.

Action is defined as implementing an action intended to achieve a particular goal on the level of 

a single manual action (such as operating a valve) or a predetermined sequence of manual 

actions. The action(s) must involve the manipulation of the hardware or software that would 

consequently alter plant status. The process of Action includes initiating the action, executing 

action steps, and verifying the action outcome. Errors of action execution can be classified into 

two major forms: slips and lapses. Slips are errors where actions executed are “not as 

planned.” They typically occur in the performance of largely automatic tasks performed by a 

skilled individual in familiar conditions, especially when attention is diverted (e.g., because of 

distraction or preoccupation). Lapses are errors in executing the planned action, such as 

missing steps of an action or executing the action at the wrong time. Lapses involve failures of 

memory, where an individual may forget to perform an intended action. 

Teamwork is defined as crew interactions with each other to accomplish a task. Teamwork 

typically involves coordination, collaboration, and communication among the crew members.

For NPP control room operation, the crew has been trained to work together and crew 

coordination and collaboration is defined in operating procedures; therefore, the teamwork 

process mainly involves communication and leadership. NPP control room crews are 

hierarchical and have a distinct leadership structure. Errors in either communication or 

leadership can lead to failure of the Teamwork function.

Operators typically have to engage in all of these macrocognitive functions to accomplish a task. 

Some tasks, such as diagnosing an alarm, may involve more detecting and understanding than 

decisionmaking or action, whereas other tasks, such as implementing a reactor coolant system 

(RCS) depressurization, rely heavily on action and teamwork. Additionally, the flow of human 

thought does not follow a linear path through the macrocognitive functions. Rather, there is 

parallel thought and iterations of macrocognitive functions as operators conduct their work. 

Cognitive Framework

One major outcome of the literature review is a synthesis of information about cognitive 

mechanisms, proximate causes, and PIFs for each macrocognitive function. The other outcome 

is a cognitive framework that takes all four of these elements—macrocognitive functions, 

proximate causes, cognitive mechanisms, and PIFs—and organizes them into a tree structure 

that illustrates how macrocognition may fail and describes the reasons why. Each 

macrocognitive function is represented with one tree. Such a causal tree is similar in 

appearance to a fault tree tipped sideways; however, there are no logic operators in the 

cognitive framework, nor is there an assumption of orthogonality throughout the tree branches. 
xvii

The generic structure for each tree in the cognitive framework is shown in the diagram below. 

The tree is written in failure terms because the purpose of the tree is to identify how a 

macrocognitive function may fail. Starting from the left to right, the box in columns represents 

the macrocognitive function, the proximate causes of failure of the function. The cognitive 

mechanisms associated with each proximate cause, and PIFs that contribute to the failure of 

each mechanism. 

This cognitive framework provides explanation about why PIFs are important and how PIFs 

influence human cognition errors. It is important to note that the framework is a tool to identify 

which proximate causes, mechanisms, and PIFs to consider or investigate for the situation or 

human failure event (HFE) under analysis. In addition, while the framework identifies which 

factors are likely to be relevant given the psychological research reviewed, the authors make no 

claim that the factors listed are the only potentially relevant factors. Other factors may influence

a particular mechanism; it is also plausible that a mechanism may fail even in the absence of 

contextual factors. These trees simply show the factors that have been identified as relevant by 

psychological research.

The content of the framework was developed to apply universally to any HRA methodology. As 

such, it does not provide guidance for assigning a relative importance to each performance 

influencing factor, evaluating its degree of goodness or badness, or a methodology for 

quantification of human error probabilities. Guidance for those elements of the analysis process 

depends on details of the particular HRA methodology and its associated quantification models. 

In summary, this report presents the outcome of an effort to build an up-to-date cognitive basis 

for HRA. We conducted a literature review to synthesize our understanding of the cognitive 

aspects of NPP crew behavior in response to plant upsets, based on research in various 

behavioral science disciplines. We developed a cognitive framework to organize information 

related to human performance in NPP operations and identify relevant PIFs leading to crew 

failure. This framework presents the links between the PIFs, cognitive mechanisms, proximate 

causes of failure, and ultimately the macrocognitive functions. 

The information in this report can serve as the technical basis for new HRA methods and a tool 

for improving existing HRA practices. The structured cognitive framework can assist HRA 

analysts with making judgments on what can go wrong with human performance and how 

various performance influencing factors may contribute to errors. The cognitive basis enables 

analysts to understand and systematically identify the reasons why humans make errors. 

Lastly, although the cognitive basis was initially developed for improving HRA, it can also 

support addressing human performance issues identified in human factors applications.

xix

ACKNOWLEDGMENTS

The authors wish to express their sincere gratitude to a number of people who have made 

significant contributions to this literature review: Ali Mosleh of the University of Maryland, who 

provided the inspiration that launched this effort and provided extensive guidance throughout 

the review process; Amy D’Agostino, Julie Marble, and James Chang at the U.S. Nuclear 

Regulatory Commission and Emilie Roth of Roth Cognitive Engineering, who all provided 

valuable comments and feedback on various drafts of this report; Katrina Groth of Sandia 

National Laboratories, for her performance influencing factor research and taxonomy, which has 

provided us a standard set of performance influencing factors from which to work; John 

Forester, of Idaho National Laboratory and formerly Sandia National Laboratories, for his 

guidance throughout this project; Bruce Hallbert, at Idaho National Laboratory, and Harold 

Blackman, of Boise State University and formerly Idaho National Laboratory, for conducting an 

internal peer review and providing valuable insights and recommendations; and the late Dr. 

Dana Kelly, who championed this effort and provided inspiration and guidance throughout the 

review process.

xxi

ACRONYMS AND ABBREVIATIONS

ACRS Advisory Committee on Reactor Safeguards

ACT-R Adaptive Control of Thought—Rational

ADS accident dynamics simulator

AT Activity Theory

ATHEANA A Technique for Human Event ANAlysis

BOP balance-of-plant operator 

CAP corrective action program

CRM crew resource management

D-I-K Data-Information-Knowledge

EUT Expected Utility Theory

GEM Guidance-Expertise Model

HEP human error probability

HERA Human Event Repository and Analysis

HFE human failure event

HRA human reliability analysis

HSI human-system interface

IDA Information, Decision, Action

IDAC Information, Decision, Action, Crew

IDHEAS IntegrateD Human Event Analysis System

LOCA loss of coolant accident

LTM long-term memory

LT-WM long-term working memory

MS mental state

NDM naturalistic decisionmaking

NPP nuclear power plant

NRC U.S. Nuclear Regulatory Commission

PIF performance influencing factor

pMFC posterior medial frontal cortex

PRA probabilistic risk assessment

PSF performance shaping factor

R/M recognition/metacognition

RCS reactor coolant system

RCT Rational Choice Theory
xxii

RO reactor operator

RPD recognition-primed decision

SA situation awareness

SACADA Scenario Authorization, Classification, Analysis, and Debriefing Applications 

SAMG severe accident management guidelines

SG steam generator

SGTR steam generator tube rupture

SRM staff requirements memorandum

SRO senior reactor operator

SS shift supervisor

STA shift technical advisor

THERP Technique for Human Error Rate Prediction

WM working memory
1

1. INTRODUCTION

Many human reliability analysis (HRA) methods are available for use as part of probabilistic risk 

assessments (PRAs) in modeling risk in nuclear power plants (NPPs). HRA is a two-stage 

process: qualitative analysis and quantification. Qualitative analysis involves evaluating the 

scenario or event involved in the analysis, working with pre-defined human failure events 

(HFEs) and/or identifying and defining new HFEs, and analyzing plant conditions, procedures, 

operator tasks, and other contextual information associated with the HFEs. The quantification 

process takes the qualitative analysis as input and estimates the probability of failure of the 

HFE, referred to as the human error probability (HEP), based on the context and effects of 

performance influencing factors (PIFs).1

However, there is evidence that the results associated with a particular HFE analysis could vary 

depending on the HRA model/method used, and/or the analyst applying the method (Lois 

et al., 2009). Because the U.S. Nuclear Regulatory Commission (NRC) frequently uses HRA 

results and insights to support risk-informed regulatory decisionmaking, the NRC has a vested 

interest in the quality of HRA and continues to improve the robustness of PRA/HRA through 

targeted activities (e.g., supporting and endorsing PRA standards developed by professional 

societies).

In a staff requirements memorandum (SRM), SRM-M061020, to the Advisory Committee on 

Reactor Safeguards (ACRS), the NRC Commissioners directed the ACRS to “work with the staff 

and external stakeholders to evaluate the different human reliability models in an effort to 

propose a single model for the agency to use or guidance on which model(s) should be used in 

specific circumstances” (2006). The NRC then instituted a research effort in response to 

SRM-M061020 to address issues related to the robustness of HRA, with a particular emphasis 

on improving traceability and consistency.

The NRC project team convened expert workshops to identify desirable features for HRA 

models. The features included a sound underlying technical basis to model human 

performance, completeness, reliability, repeatability, and transparency (Hendrickson 

et al., 2012). The team used these features as the criteria against which they examined existing 

models. The team concluded that, while each model has its strengths, no model meets all the 

desirable features identified above. Furthermore, evidence accumulated through several 

studies, including the International Empirical Study (Lois, et al., 2009) as well as from a 

U.S. domestic empirical study, has shown that all methods have some general limitations 

contributing to variability in HRA results for the same HFE (Forester et al., in press):

Cognitive Basis. All of the methods have limitations in modeling and quantifying human 

performance under various conditions. At least part of the variability within a method between 

analysts can be attributed to a lack of an adequate underlying theoretical basis to guide the 

analysis, particularly with respect to the cognitive activities associated with understanding 

complex situations and deciding how to respond.

Qualitative Analysis. The HRA methods need systematic and thorough guidance for performing 

a qualitative assessment to support HRA quantification (Hendrickson, et al., 2012). The 

differences in the qualitative analysis required by the different methods (and those performed by 

different analysts) appear to be a major driver of the variability in the results obtained by the 

different applications. Improved guidance for performing the qualitative analysis should 

contribute to improving the consistency of HRA results.

 

1. Some HRA methods refer to contextual factors as performance shaping factors (PSFs). The term performance 

influencing factor (PIF) is used in the IDHEAS HRA methodology, and thus is used throughout this document.
2

Tie between Qualification and Quantification. Many methods focus on identifying failure 

mechanisms and the contextual factors that contribute to the mechanisms; these methods 

generally produce a superior qualitative analysis (rich in content and quality operational stories).

However, superior qualitative analysis itself does not necessarily produce HEPs that are more 

accurate. The information gathered during the qualitative analysis should be used to produce a 

HEP. However, with some exceptions (such as the Technique for Human Error Rate Prediction 

(THERP)), many methods have inadequate guidance on how to use the information from 

qualitative analysis to determine HEPs (i.e., translating the information into HEPs). For 

example, in empirical HRA studies, some analyst teams had difficulty using the information 

obtained during the qualitative analysis in an effective way during quantification (c.f., Lois et al., 

2009).

PIF Judgments. Different HRA methods identify different PIFs as important to consider (e.g., 

high versus low workload, adequacy of indications) for a given situation. Yet, the methods 

generally do not provide justification regarding why and how a PIF contributes to human errors; 

therefore, it is difficult for HRA analysts to judge the presence and effects of a PIF when 

quantifying HEPs. 

The first limitation, cognitive basis, contributes largely to other limitations. An adequate 

technical basis is needed to identify the relevant domains of cognitive activity for a given human 

action, identify potentially important failures and failure modes that are most likely while 

performing in those cognitive domains, and guide the collection of information that analysts use 

to perform an HRA. Without a strong cognitive basis, it is unclear what information should be 

included in qualitative analysis, how analysts should identify such information, and how analysts 

should use such information for HEP estimation. In addition, PIFs affect human failures by 

affecting cognitive processes involved in the human failure events. Without a strong cognitive 

basis, judgments of PIFs and their effects on human failures on a given situation can be 

arbitrary or solely rely on expert judgment.

Based on this evidence, the NRC staff decided to address SRM-M061020 by developing an 

integrated HRA method that would incorporate the strengths of the existing methods and 

develop new features to address the limitations of existing methods. The integrated method is 

thus referred to as an Integrated Human Event Analysis System (IDHEAS). Among the new 

features should be an updated technical basis for human reliability, grounding the method in 

current science. Therefore, the first step of the method development was to develop a technical 

basis that synthesized the current understanding of human performance and human errors from 

cognitive sciences and operational experience. The technical basis will serve as the foundation 

for developing the qualitative analysis guidance and a quantification model for estimating human 

error probabilities in IDHEAS. We developed the technical basis by performing a 

comprehensive cognitive literature review, synthesizing the information, and developing a 

framework to organize the information for use in HRA. This report presents the technical basis 

we developed, including the psychological framework and literature review results. The use of 

these outcomes in IDHEAS will be described in separate reports. 

1.1 Overview of the Literature Review

1.1.1 Developing a Technical Basis for IDHEAS

We recognize that establishing a solid cognitive foundation for HRA is critical to reduce 

variability of HRA. Current HRA methods generally use descriptive human behavioral models.

Yet, those models are not tied to the mechanisms underlying human errors. A mechanism, by 

definition, means a natural or established process by which something takes place or is brought 

about. We used the term mechanism or cognitive mechanism here to refer to the 

neurophysiological, psychological, or behavioral processes by which human cognitive tasks are
3

accomplished. Therefore, we decided to perform a thorough review of current psychological, 

cognitive, human factors, and operational research to develop the technical basis for IDHEAS.

Research in the behavioral sciences has accumulated knowledge about the mechanisms 

underlying human performance, including how human performance may be affected by 

situational factors. This knowledge should be used to develop the technical foundation for HRA. 

Such a cognitive basis can guide qualitative analysis, tie information from qualitative analysis to 

quantitative analysis, elucidate the effects of PIFs on human failure, and define the strength of 

PIFs with respect to cognitive mechanisms and human vulnerabilities.

However, this information is scattered throughout the broad fields of cognitive psychology, 

behavioral psychology, neuropsychology human factors, and human performance; even review 

articles in the literature typically only focus on one narrow aspect of human performance 

(e.g., attention, situation awareness (SA)) without systematically documenting the available 

information for the full range of human performance. Moreover, the majority of literature 

focuses the description of results and conclusions on how humans can successfully perform 

given tasks without explicitly delineating the conditions under which humans would fail the tasks 

(i.e., it often needs analysis and inference to identify information about human failures from 

literature). Therefore, our task was to mine current state-of-the-art behavioral sciences 

research for information to establish direct links (causal relationships) between performance

influencing factors, mechanisms of human cognition, and human performance. Such links 

would enhance the basis for expert judgment and improve both qualitative and quantitative 

aspects of HRA.

The use of explicit and causal links allowed us to adapt PRA practices (such as event trees/fault 

trees to model equipment performance) and apply them in an analogous approach to the 

qualitative analysis part of HRA. One of the most significant findings of the International HRA 

Empirical Study (Lois, et al., 2009) is that variability in estimation of HEPs is driven to a great 

extent by differences in the quality and depth of the qualitative analysis. By adopting a 

structured causal cognitive framework to inform the qualitative analysis, we aim to enable 

analysts to appropriately collect information and identify potential failure paths and associated 

causes and influencing factors, thus reducing inter- and intra-analyst variability. This aim took 

the literature review a step further—the development of causal linkages in a structured cognitive 

framework that provides analysts a tool that will assist them in the qualitative analysis process.

1.1.2 Scope and Goals of the Literature Review

Using a systems engineering approach, we first identified the scope, goals, and requirements 

for the literature review, and then developed an implementation plan to perform the literature 

review and synthesize the information from the literature. The ultimate goal of the literature 

review was to develop a technical basis for IDHEAS by providing a profound understanding of 

human errors in NPP internal, at-power events. This understanding should include the following 

aspects:

• cognitive tasks in NPP internal, at-power events 

• cognitive functions that support the tasks

• cognitive mechanisms underlying the functions

• performance influencing factors that affect the mechanisms

To accomplish this, the specific goals of the literature review were to:

• Identify the proximate causes for failure of cognitive tasks that operators may perform in 

NPP control rooms that result in negative consequences for plant safety. 
4

• Identify cognitive mechanisms underlying the cognitive function’s proximate cause (i.e., 

identify how and why cognitive errors occur).

• Identify factors that influence human performance and, where possible, identify how those 

factors affect the chance of failures.

• Develop, based on the literature, a structured cognitive framework that can serve as a 

psychological foundation for IDHEAS.

We subsequently developed the following requirements for conducting the literature review to 

achieve these goals. 

Requirement 1: Review literature and operational experience relevant to NPP operation.

Because the purpose of the literature review was to improve the technical foundation of HRA for 

the NPP domain, the information on human cognition identified from the literature review should 

be relevant to NPP operation. We decided that this requirement included research related to 

human performance for internal at-power events, which typically involves control room operator 

performance in response to design-basis initiating events that are part of a PRA. Yet, the 

psychological foundation developed from the literature review should be as generic and broadly 

applicable as possible to allow for application outside of control room activities and for events 

outside the scope of Level 1 design basis events. For example, the cognitive framework should 

be informative to such applications as event recovery performance that involves activities 

outside the control room. 

Requirement 2: Identify taxonomy of cognitive functions required for performance in NPP 

operation.

NPP operator activities are predominately driven by cognition; accordingly, one task of the 

literature review was to identify the cognitive functions that operators use to accomplish NPP 

operation. Another characterization of NPP operation is teamwork; operating crews are groups 

of people who are highly trained, supervised, and who work together in a regulated, 

procedure-driven environment to control the plant. Therefore, we specified that the literature 

review include both cognition at an individual level and at the crew level, and that the review 

focus on research related to highly trained or expert personnel rather than novices. 

Furthermore, the literature review should identify a taxonomy that describes the cognitive 

functions that mediate operator performance in NPPs. Section 2 defines the cognitive functions 

(called macrocognitive functions) used in the literature review and cognitive framework.

Requirement 3: Identify information on why and how failure of a macrocognitive function can 

occur.

To understand why humans make cognitive errors, it is necessary to understand how human 

cognition successfully operates. In other words, it is important to identify the mechanisms by 

which human cognition operates. Some psychological literature may discuss these 

mechanisms on a behavioral or descriptive level, while other psychological research may 

provide a more profound understanding of the contingencies and boundary conditions with 

which humans can reliably perform a function. Both lines of information are necessary for the 

specific goals of determining why operators make failures under various situations and 

explaining how the PIFs affect the chance of failures.

Requirement 4: Identify information about which PIFs are relevant for each type of failure, and 

why and how the PIFs affect the chance of failures.

Of primary importance for this requirement is to identify the PIFs relevant to a particular cause 

of failure; for example, an HRA analyst would find it very valuable to have information that 

allows her/him to narrow down which PIFs to consider when evaluating the likelihood that 
5

operators will misdiagnose a situation in a particular HFE under analysis. Regarding how and 

why PIFs influence the chance of failure, it is desirable that the technical basis should include 

information addressing the following specific issues: 

• how a PIF affects human performance in general (i.e., fatigue increases reaction time) 

• how PIFs influence specific task performance (i.e., increasing traffic complexity leads to 

higher workload for air traffic controllers) 

• how PIFs relate to mechanisms (i.e., a sustained high level of stress can impair a human’s 

attention shift to new targets) 

• how PIFs affect human errors.

Requirement 5: Produce a structured tool that organizes all of the above information into a 

cognitive framework that serves as the psychological foundation of the IDHEAS HRA method.

Each of Requirements 1-4 above represents a part of the desired outcome, while 

Requirement 5 represents the integrated outcome. The task was to sort, synthesize, integrate, 

and organize information in these different forms from the literature review into a structure that 

can be used to support the IDHEAS HRA method. Thus, the literature review should connect 

failures of macrocognitive functions with cognitive mechanisms, and PIFs. However, the 

information needed to make these connections may not be readily available in the literature. As 

mentioned previously, research in the areas of cognitive psychology and human performance 

may not have focused on fully elucidating the relationships between PIFs, cognitive 

mechanisms, and types of proximate causes. Establishing these links requires analysis, 

inference, and development based on information available. Due to the limited information, the 

links can be limited and subjective. Therefore, we determined that we would organize the 

cognitive framework structure in such a way that it may be updated as new research becomes 

available and validated or modified for application to HRA. Furthermore, the cognitive 

framework should be a usable tool with adequate documentation for analysts to employ it in 

their analyses. The cognitive framework is thus the ultimate product of the literature review.

1.1.3 Literature Review Process

The literature review team conducted the literature review in two major rounds. The first round 

was exploratory with the purpose to identify the scope of the literature review (i.e., what major 

models and research domains to include), and to identify a model for organizing the literature 

into a usable structure. Based on the results of the first round, the literature review team 

adopted a macrocognitive model from a number of existing cognitive models for structuring the 

literature review and cognitive framework. The model includes five macrocognitive functions 

that underlie cognitive tasks. Section 2.2 of this report discusses macrocognition in general and 

provides an overview of various macrocognitive models; Section 2.4 details the macrocognitive 

structure used in this approach.

We then commenced the second round of the literature review to identify the causes of failure of 

human cognition in the macrocognitive functions included in the model. The focus was on 

identifying the mechanisms, or the cognitive processes, that, in certain circumstances, may fail 

and lead to proximate causes. We then began organizing the mechanisms obtained from the 

references included in the review, and began the process of building the cognitive framework.

Section 2.5 of this report discusses this process in detail. Table 1-1 provides the search terms 

and concepts that we included when conducting the literature review.
6

Table 1-1. Search terms used in literature review.

Search Domain Search Terms and Concepts

Cross-Cutting 

Areas

• Macrocognition

- and performance

- and team performance

• Attention

• Working memory/central 

executive

• Personality 

theory/individual 

differences

• Learning

• Resilience

• Workload/task load/other 

loads

Detecting and 

Noticing

• Vigilance

• Monitoring

• Problem detection

• Sensation

• Perception

• Pattern 

processing/matching 

(context effects, 

expectancies, complexity, 

recognition)

• Desensitization

• Engagement

• Information foraging

• Change blindness

• Situation awareness

Understanding 

and Sensemaking

• Situation awareness

• Situation assessment

• Sensemaking

• Mental model

• Frames

• Schemas

• Scripts

• Mental maps

Decisionmaking • Complex problem solving

• Team problem solving

• Planning

• Re-planning/adaptation/ 

adaptability

• Naturalistic decisionmaking

• Recognition-primed 

decisionmaking

• Cognitive biases

• Goal selection

• Prioritization/prioritization 

errors

• Procedure following 

errors

Action • Multitasking

• Dual tasks

• Execution errors 

• Slips 

• Lapses 

• Performance and contextual 

errors

• Task/action switching

• Simultaneous action goals

• Automaticity 

• Stimulus-response 

compatibility 

• Error monitoring and 

correction 

• Motor control

Teamwork • Communication

• Teamwork

• Team/crew collaboration

• Team/crew performance

• Team/crew 

decisionmaking

• Team/crew problem 

detection

• Team/crew problem 

solving

• Team/crew dynamics

• Team sensemaking

• Team situation awareness

• Groupthink

• Group collaboration style

• Leadership styles

• Team cohesion

• Shared resource 

management

• Crew resource 

management

• Group macrocognition

• Team cognition

• Distributed cognition

1.2 Overview of this Report

The current chapter briefly discusses above the literature review process, boundaries, and 

scope; the majority of the report focuses on the macrocognitive models, macrocognitive 

functions, and cognitive mechanisms that achieve the functions. These together comprise the 

cognitive framework for IDHEAS. This document serves as an introduction to the psychological 

concepts that relate to human performance in NPPs, and provides users with enough 

background information to use the cognitive framework (Appendix A and Appendix B). 

A separate report discusses the development of IDHEAS and instructions for the practical 

application of the method (Forester, et al., in press).

Chapter 1 describes macrocognition and macrocognitive models and functions. From there, 

each of the remaining chapters presents literature findings that support each macrocognitive 

function. Chapter 0 discusses the macrocognitive function Detecting and Noticing, Chapter 0

describes Understanding and Sensemaking. Chapter 5 reviews Decisionmaking, Chapter 6

describes Action, and Chapter 7 discusses Teamwork. The results of the literature review are 

summarized in Appendices A and B. Appendix A provides tables that list all of the cognitive

mechanisms for failure for each macrocognitive function, and identifies PIFs that influence the 
7

cognitive mechanisms and proximate causes of failure, while Appendix B depicts the 

information from the tables in Appendix A graphically in the cognitive framework diagrams.

Because of limited time and resources, this report, while serving as a generic technical basis for 

HRA, has several limitations: 

1. While the information presented in the report is largely taken from cognitive and 

psychological literature and therefore it is generic to human performance, the work was 

directed by the chosen cognitive framework, which in turn was strongly influenced by the 

assumptions of the IDHEAS method. Review of the cognitive mechanisms was primarily 

focused on those relevant to control room tasks. Research in psychology, sociology,

and human factors of relevance for NPP control room operation that do not fit one or 

more assumptions have not been covered or only marginally integrated into the 

framework. This includes research from team decisionmaking, situated cognition, 

distributed cognition, and cognitive systems engineering. 

2. The focus of the report is on cognitive mechanisms of the macrocognitive functions. 

While the report presents information from the literature on how various performance 

influencing factors may contribute to proximate causes, the models for performance 

influencing factors and their relations to various cognitive mechanisms were not 

thoroughly reviewed. Thus, many factors, especially organizational factors that may 

enhance human performance and recover human errors were not thoroughly covered in 

the report.

3. The tables in Appendices A and B delineated the potential relationships between 

proximate causes, mechanisms, and related PIFs. These tables can serve as useful 

references for developing HRA methods and practicing HRA. Yet, notice that due to the 

incompleteness in the literature and limitations in our review, we had to infer many of the 

relationships in the tables based on our understanding of the existing information. The 

inferences are considered to be expert judgment and thus have inherent subjectivity. 

Overall, the report provides a thorough literature review and technical foundation for human 

reliability analysis. Moreover, the literature review conducted for each of the macrocognitive 

functions provides broad coverage of the relevant literature and synthesis of the key points to be 

drawn from the literature relative to the factors influencing human performance and human 

reliability. Therefore, the report also serves as a technical basis for research and practice in 

human factors engineering that is oriented to ensure safety. 

9

2. MACROCOGNITION

2.1 Introduction

Models of human cognition developed circa the 1970s–1980s used a “human as computer” 

metaphor to describe information processing. Information was input to the mind (or “black box” 

of the human) through the sensory systems, then the information was processed by the mind

and a response was generated. This was also consistent with the behavioral tradition that had 

dominated experimental psychology to that point, in which the stimulus-response paradigm was 

the basis of much psychological thought. The “black box” metaphor was used because the 

cognitive processing could not be seen; therefore, it was hidden from view. 

As neural imaging technology and the sophistication of psychological research advanced, 

scientists began to shed light on the internal workings of the “black box”. It gradually became 

clear that an input-processing-output information processing metaphor was inadequate to 

describe the true complexity and dynamics of human cognition. Nevertheless, the information 

processing perspective of human cognition has been quite popular in applied fields like

engineering and human reliability analysis (HRA), as evidenced by the information processing 

models used in various HRA methods. The simplification of human thought into serial or linear 

stages made the information processing models very useful for applied purposes. However, we 

now know that human thought is not entirely serial or linear—a great deal of simultaneous, 

parallel, and circular processing occurs. Also, information processing approaches cannot 

adequately account for the creativity, insight, illogical thinking, instinct, and moments of 

brilliance that people are prone to have (Anderson, 2000).

Models of human cognition have subsequently become more dynamic as a result of this 

research. Research on human cognition spans many domains such as cognitive psychology, 

cognitive neuroscience, psychophysics, behavioral science, and human factors. The level of 

detail at which each research effort addresses human information processing varies from neural 

responses, cognitive system activities, and behavioral measures. Psychological research has 

moved on to models that describe human cognition as several systems that work together to 

process information (Baddeley, 2000; Baddeley & Hitch, 1974; Baddeley & Larsen, 2007). 

Work done in real-world settings, such as naturalistic decisionmaking and situation awareness 

(SA), combined with more narrowly focused laboratory research, has done much to elucidate 

the complexity of human cognition. Particularly for applied research in naturalistic settings, it 

has become clear that a macrocognitive perspective is imminently more useful for 

understanding human cognition.

Macrocognition is a term originally coined by Cacciabue and Hollnagel (1995) to describe 

cognition in real-world settings, rather than in research laboratories. Macrocognition focuses on 

the nature of human performance “in the field,” where decisions are often very complex and 

must be made quickly, by domain experts, in risky or high-stakes situations (Klein et al., 2003). 

Microcognition, on the other hand, is typically the focus of tightly controlled laboratory research, 

with a goal of elucidating the building blocks that underlie cognition that is more complex. There 

are a large number of microcognitive models, all focused on different aspects of human 

cognition and how the brain works. Macrocognition integrates the more narrowly focused 

microcognition laboratory research findings together into a larger picture that describes what 

people actually do with their brains in applied, complex settings. 

Given that the present project is focused on human cognition in a nuclear power plant domain, 

using macrocognition as the organizing construct for the literature review and cognitive 

framework is more appropriate for several reasons: 

• It organizes microcognitive models into a manageable set of functions.
10

• HRA analysts can easily understand macrocognition.

• Macrocognition is useful for predictive analyses of human performance in complex 

scenarios. For example, it is more appropriate for an analyst to predict crew performance at 

detecting a critical cue than it is for the analyst to predict whether the operator will pay 

sufficient attention to an incoming stimulus (cue) that it is drawn into active working memory. 

• Macrocognition is detailed enough to synthesize psychological research findings into a 

structure that yields a coherent understanding of how human cognition functions and what 

can lead it to fail. 

• It integrates state-of-the-art psychology and cognitive science into a foundation for HRA, 

providing a more advanced technical basis for HRA than previous information processing 

models.

This chapter provides an overview of several macrocognitive models and then shows how 

macrocognition was adapted to apply to nuclear power plant (NPP) operations. The second half 

of the chapter describes the development of a macrocognitive framework for use in HRA.

2.2 Overview of Macrocognition

There are a number of macrocognition models in the literature describing macrocognitive 

functions (i.e., high level mental activities to accomplish a task or achieve a goal in a naturalistic 

environment; Letsky, 2007). Each approach divides the spectrum of cognition up into slightly 

different chunks; however, there is general consensus that the macrocognitive functions 

relevant to human performance in complex, dynamic, high-risk domains include but are not 

limited to (Roth, 2010a, 2010b):

• Detecting stimuli and noticing problems (detecting and noticing).

• Understanding information and making sense of situations (understanding and 

sensemaking).

• Planning responses and making decisions about what to do (planning and decisionmaking).

• Taking action and monitoring effectiveness of the action (action).

2

• Communicating and coordinating with team members (communication and coordination).

Notice that from the point of view of the individual, team communication and coordination 

activities could be seen both as resources for achieving other macrocognitive functions or as 

goals on their own. One could also consider other resources (like memory aids, support tools) 

in an extra function like “cognitive tools use”. This might be an important addition in a NPP 

setting where there is extensive use of, for example, operating procedures. On the other hand, 

from the perspective of a team such as the ones in NPP control rooms, communication, 

collaboration, and coordination together can be viewed as a macrocognitive function with which 

the team achieves its task goals. 

A key difference between macrocognitive models and traditional information processing models 

is that macrocognition recognizes that human cognition is not strictly linear or serial, but it also 

involves a great deal of parallel and cyclic processing. Macrocognitive functions occur in a 

continuous loop (Roth, 2010b) and overlap (Patterson & Hoffman, 2012). People working in 

 

2. Roth (2010a, 2010b) clarified that most models of macrocognition do not typically include action, but included 

action in her list of macrocognitive functions based on the history in the NPP HRA community of analyzing action 

implementation errors.
11

naturalistic settings typically have to accomplish most or all of these functions, often at the same 

time (Klein, et al., 2003).

The next two sections review two well-recognized macrocognition models to elucidate how the 

macrocognitive functions work together to accomplish complex tasks.

2.2.1 Macrocognitive Functions by Klein et al.

Klein et al. (2003) proposed a macrocognition model with an initial set of primary macrocognitive 

functions and supporting macrocognitive processes3 (see Figure 2-1) to investigate how 

cognition actually functions in real world complex tasks. The primary macrocognitive functions 

included:

• naturalistic decisionmaking, or how experts make decisions in real-world environments

• sensemaking and situation assessment, or how people understand and make sense of the 

situation

• planning, or how people develop plans for responding to the situation

• adaptation and re-planning, or how people adapt their plans and strategies given the 

dynamic nature of the situation

• problem detection, or how people detect anomalies or abnormal conditions that need to be 

addressed

• coordination, or how people coordinate their behavior with others to achieve common goals

Figure 2-1. Macrocognitive functions and supporting processes for individuals and 

teams (Klein, et al., 2003).

 

3. Klein et al. (2003) expressly stated that they intended this list of primary macrocognitive functions to encourage 

research at the macrocognitive level rather than to advocate an “official,” validated list.
12

In Figure 2-1, the blocks in the middle represent the macrocognitive functions; the items in the 

surrounding circle represent the macrocognitive processes that support the functions. The 

macrocognitive processes are means for achieving the functions. Notice that all of the functions 

share all of the macrocognitive processes. Therefore, these functions and processes work 

together in a continuous loop, with the acknowledgment that some or all of these functions have 

to occur simultaneously.

At the time when Klein et al. (2003) published their model, they decided that it was premature to 

attempt to diagram the relationships between the different functions and supporting processes.

There is no assumption of serial or linear flow between the functions, nor is there an assumption 

about which function occurs first. Rather, the flow is dynamic, continuous, and dependent on 

the situation, with much parallel or simultaneous cognitive activity.

2.2.2 The Overlapping Macrocognitive Function Model

Patterson and Hoffman (2012) adapted the Klein et al. model (2003) for space shuttle missions. 

They proposed an integrated macrocognitive framework for primary functions, which are the 

most important functions that a work system must conduct to achieve its primary goals 

(Patterson & Hoffman, 2012). Patterson and Hoffman identified five primary macrocognitive 

functions: detecting problems, sensemaking, re-planning, deciding, and coordinating, each of 

which is described briefly below. 

Detecting Problems. The function of detecting problems involves noticing that events may be 

taking an unexpected turn. This change, whether positive or negative with respect to the goal of 

the system, requires explanation and may indicate a need to reframe the understanding of the 

situation (sensemaking) and/or revise ongoing plans (re-planning) or actions.

Sensemaking. Sensemaking involves activities such as collecting, verifying, and integrating 

information, and making assessments about how the detected information maps onto potential 

scenarios or explanations. It includes generating hypotheses to consider and potentially 

revisiting discarded hypothesis when faced with new evidence.

Re-planning. This function involves adaptively responding to changes in the situation or 

objectives from a variety of sources. It also includes adapting pre-made default plans to the 

specific situation within a window of opportunity. When default plans are not available or 

applicable, re-planning involves creating a new strategy for building one or more desired end 

states or goals. According to Patterson and Hoffman (2012), “this function includes adapting 

procedures, based on possibly incomplete guidance, to an evolving situation where multiple 

procedures need to be coordinated, procedures that have been started may not always be 

completed, or when steps in a procedure may occur out of sequence or interact with other 

actions” (p. 2).

Deciding. Patterson and Hoffman state that making a decision is a complex activity that is not 

simply the act of committing to some course of action to reach certain fixed goals (2012). 

Instead, it is far more complex. It involves a number of questions and issues, some of which are 

more or less important in different contexts. It might also involve questioning the 

appropriateness of default courses of action, considering trade-offs in ongoing plan trajectories, 

or reversing previous decisions or commitments. Patterson and Hoffman view decisionmaking

as more often the activity of a team, rather than a single individual; deciding can be a 

consensus activity involving the accommodation of different positions toward decisions (2012).

Coordinating. According to Patterson and Hoffman, the function of “Coordinating” consists of 

managing activity, dependencies, and communication across individuals who have roles that 

may have common, overlapping, interacting, or even conflicting goals (2012).
13

Patterson and Hoffman developed a representation of the dynamic overlaps and interactions of 

the five primary functions (see Figure 2-2). They posit that any of the five functions can involve 

events in the environment, communication between people, or reasoning within an individual. 

All of the functions are continually ongoing (consider each ring as continuously rotating about its 

center), and over time, a function overlap more or less with the others functions as they engage 

or interact with each other. “Coordinating” may be a function that continuously “wraps” the 

others. As the situation evolves, the work system may increase the emphasis on a particular 

function, which could be illustrated in Figure 2-2 by increasing the radius of the corresponding 

circle and altering the overlaps (Patterson & Hoffman, 2012). Thus, Patterson and Hoffman’s 

integrated framework illustrates that macrocognitive functions are parallel, continuous, 

interacting, and overlapping.

Figure 2-2 also clearly shows that the macrocognitive functions are not independent of each 

other. The function of detecting problems, for example, involves the functions of understanding 

the perceived information, making a decision about whether it is important or relevant, and 

adapting plans to the new information. Similarly, there are aspects of deciding in all of the other 

macrocognitive functions, and all of the functions require coordination with other members of the 

crew. The Patterson and Hoffman model depicts human cognition in a “big picture” view, 

integrating microcognitive areas together and demonstrating the interrelationships with other 

areas.

Figure 2-2. The overlapping function model of macrocognition (Patterson & Hoffman, 

2012).

Both the Klein model and the Patterson and Hoffman model incorporate “coordination” as the 

teamwork aspect of macrocognition. Other studies focused more explicitly on team 

macrocognition (Fiore, Smith-Jentsch, Salas, Warner, & Letsky, 2010; Letsky, 2007; Letsky, 

Warner, Fiore, & Smith, 2008). For example, Letsky et al. proposed that team coordination is 

achieved through four stages: knowledge construction, team collaborative problem solving, 

team consensus, and outcome evaluation. They also identified that team coordination is 

achieved through verbal and nonverbal communication. In summary, while the authors of the 

various macrocognitive models intended the main theoretical concepts to apply across domains, 
14

they developed their models based on their own experience in specific domains of application. 

This might result in the subtle differences between the models. Given that these models have 

no fundamental differences in their capturing the nature of human cognition, we should value all 

these models and integrate their strengths to come up with a generic cognitive framework for 

NPP applications.

2.3 Macrocognition for NPP Operations

Nuclear power operations have some aspects in common with aviation and military operations, 

such as hierarchical command and control, and real-time management of complex interactive 

systems. However, NPP operations are highly proceduralized. Various types of procedures 

(e.g., alarm response procedures, abnormal operating procedures, and emergency operating 

procedures) provide pre-defined goals, criteria to assess conditions and situations, and detail 

specific actions to be taken in response to abnormal or accident conditions. Additionally, plants 

have severe accident management guidelines (SAMGs) that provide high-level guidance for the 

goal of preventing or mitigating radioactive release to the environment, and set functional 

priorities for optimal management of an accident. In the United States, the general work 

practice for NPP operators is to adhere to procedural instructions. The unique nature of the 

nuclear power plant domain tailors a macrocognitive model to the specific features of the 

nuclear environment. For example, procedure-based operation eliminates many challenging 

and uncertain aspects in decisionmaking: the team structure and working protocols in the 

control room do not demand many cognitive aspects of communication and coordination 

activities. 

Several cognitive models have been used in the NPP human factors engineering and HRA 

domains: Information-Decision-Action (IDA) and Information-Decision-Action-Crew (IDAC), and 

O’Hara’s model of generic operator tasks. These models provide information about the 

important aspects of human cognition from an NPP perspective. In addition, this report includes 

a descriptive model of teamwork in NPP control room operations.

2.3.1 IDA and IDAC

The IDA model (Shen & Mosleh, 1996, Smidts, Shen & Mosleh, 1997) is an engineering 

approximation of human cognition developed for modeling operator performance in the nuclear 

power plant domain. It provides an error taxonomy for event analysis (Smidts, Shen & Mosleh, 

1997). The model applies to environments similar to NPP operations where behavior is 

influenced by extensive training and requirements to follow procedures. IDA divides human 

information processing into three main blocks: 

• I – Information (collecting information): the operator’s perception of plant conditions

• D – Decision (diagnosis and making a decision about what to do): the operator’s goals and 

strategies for handling the situation, as well as memories (including memories of expected 

plant responses, current activities, related plant symptoms from earlier experience, and 

knowledge of plant system and operation)

• A – Action (implementing the chosen action): taking action in the plant, based on the 

decision made in the D phase

IDA’s error taxonomy consists of internal errors and external errors. An internal error would be 

failure of any of the I, D, or A phases. External errors are mismatches between three elements: 

the plant’s needs, procedures, and crew decisions. IDA also accounts for certain PIFs such as 

time stress that may affect behavior.

In a series of five papers, Chang and Mosleh (2007a, 2007b, 2007c, 2007d, 2007e) expanded 

and improved upon the IDA model to create a model called IDAC. IDAC is best described as a 
15

computer simulation-based cognitive model designed specifically for simulating an operating 

crew’s behavior when responding to abnormal NPP scenarios in the Accident Dynamics 

Simulator (ADS) simulation environment. ADS includes five functional elements that interact 

with each other in close frequency to generate multiple scenarios following an initial plant 

malfunction:

• a plant behavior simulator

• a control room panel simulator

• a crew behavior simulator (i.e., IDAC)

• a hardware behavior simulator (i.e., to model potential hardware failures)

• a simulation sequence controller

IDAC’s role in the ADS simulation environment is to probabilistically model potential operators’ 

responses based on up-to-date plant information and available resources (e.g., procedures and 

memory). The most recent ADS-IDAC implementation can be found in Coyne (2009).

The IDAC crew behavior simulator consists of two functional elements: the cognitive processor 

and the psychological processor. Together, they interact with memory and procedures to form 

the IDAC individual cognitive process model, shown in Figure 2-3. 

Figure 2-3. IDAC operator cognitive flow model.

The cognitive processor consists of three blocks taken from IDA: Information pre-processing (I), 

problem solving and decisionmaking (D), and action execution (A). These three blocks 

dynamically interact with the psychological processor, also known as mental state (MS). 

Changes in any of the cognitive processor blocks (IDA) may result in changes to mental state, 

which in turn may affect the subsequent cognitive elements’ behavior, as shown in Figure 2-3. 

Because IDAC assumes a crew operation environment, the “External World” in Figure 2-3

includes the plant, the procedures, and the other crew members. IDAC’s Action execution block 

includes not only physical action taken on the plant but also communication to other crew 

members. Each of these blocks in the operator cognitive process model is summarized below 

and discussed in more detail in Chang & Mosleh (2007c).
16

Information pre-processing (I): Block (I) of the cognitive process model refers to a person’s 

highly automatic process of handling incoming information. This involves information filtering, 

comprehension and memory, grouping and categorizing, and prioritizing, but stops short of 

further inference and conclusions (Chang & Mosleh, 2007a, 2007c).

Problem solving and decisionmaking (D): This stage is also referred to in IDAC as “diagnosis 

and decisionmaking.” This block refers to what happens after information perception—what 

people do with the information they have perceived. This block includes cognitive activities 

such as “situation assessment,” “diagnosis,” and “response planning.” Information that has 

been brought to an operator’s attention is translated into a problem statement or a goal that 

requires resolution. The process of resolving a problem or goal involves selection of a method 

or strategy. Goals may be broken into sub-goals, in such a way that complex problems are 

broken down into simpler ones, and solved using an appropriate strategy (Chang & Mosleh, 

2007a, 2007c).

Action execution (A): Block (A) of the cognitive process model refers to the execution of the 

action that was decided upon in Block (D). Actions are typically skill-based, highly practiced, 

and require little cognitive effort (Chang & Mosleh, 2007a, 2007c).

Mental State (MS): The MS block interacts with each of the IDA blocks dynamically. The mental 

state influences the activities in each of the IDA blocks (e.g., by affecting the filter used in I, or 

by shaping the choice of strategy in D), and is in turn updated by the results of the activities in 

each block. The MS block explains how cognitive activity starts and continues, and provides 

reasons for why a goal is selected or abandoned (Chang & Mosleh, 2007a, 2007c).

The IDAC model takes the classical information processing concept, i.e., framing cognition as a 

continuous series of IDA loops. IDAC posits a linear, serial flow from I to D to A, and an 

assumption of success of the previous phase when evaluating any one phase. In other words, 

when evaluating the D phase, for example, an analyst assumes the information gathered in the I 

phase is correct, necessary, and sufficient for proper decisionmaking. Otherwise, the error 

would be in the I phase, not the D phase. Likewise, when analyzing the A phase, an analyst 

assumes success in both the I and D phases. If an error occurs in action implementation that is 

due to incorrect decisionmaking, then the error is actually in the D phase, not A. 

Additionally, IDAC adds fine-grain information processing by conceptualizing that these IDA 

loops are nested,4 as shown in Figure 2-4. What this means is that for any phase of IDA, there 

are sub-loops of IDA. For example, in the process of perceiving information (I), an operator 

recognizes information (I-in-I), makes decisions about the relevance of that information (D-in-I), 

and takes action based on that decision, such as discarding or integrating with other information 

(A-in-I). Therefore, there are decision elements in the information gathering process. The 

Nested IDA loop structure would identify the decision related to information gathering that 

belongs to the high-level I element rather than the D element. This nested structure has the 

benefit of pinpointing the specific error. IDAC states that this nested structure can continue for 

as many layers as necessary to decompose complex tasks into simple subtasks (Chang & 

Mosleh, 2007a).

 

4. For human failure analysis, IDAC uses the nested I-D-A loop error taxonomy to identify the cognitive root cause 

that failed a task. A task in PRA typically requires multiple I, D, and A interactions with different levels of 

granularities within an operator and between operators. Using a single layer of I-D-A for root cause identification 

has a tendency to blur the scope of each individual cognitive element. IDAC’s two-level nested I-D-A loop (see 

Figure 2-4) intends to clarify the boundaries. 
17

Figure 2-4. Nested IDA structure concept.

A key difference between IDA and IDAC is the crew component (C) of IDAC. IDAC models 

crew behavior by creating a simplified crew structure with three generic types of operators 

based on the roles in an NPP crew: the decision maker, action taker, and consultant (Chang & 

Mosleh, 2007a). These roles can be organized in different crew structures based on the type of 

crew organization at a plant. Two main features characterize crew interactions: communication 

and coordination. Communication occurs through formal and informal channels. Roles specify 

formal communication channels, such as the formal communication between the decision maker 

and the action taker. IDAC considers all other communication as informal. Crew coordination 

can be highly dynamic and complex, and is dependent on the culture of the crew. Coordination 

as modeled in IDAC includes supporting behaviors such as performance monitoring, error 

correction, and workload and responsibility redistribution (Chang & Mosleh, 2007a).

The IDAC crew model includes crew interactions with the system through actions of individual 

members. For each member in the crew model, there are:

• defined responsibilities and tasks

• defined formal communication channels

• defined experience and knowledge bases

• individualized psychological and physical characteristics

Crew influences on individual operator response are accounted for through the IDAC 

team-related performance influencing factors (PIFs) (discussed in Chang & Mosleh, 2007b).

While the simulation aspects of IDAC are not relevant to the present effort, IDAC provides a 

structure with underlying models of both individual cognition and team behavior of an NPP 

operating crew. These aspects of the IDAC model make it an appropriate choice for adapting 

macrocognition to the NPP domain. The next section discusses another NPP-specific model 

that is also informative to the task of applying macrocognition to NPP operations: O’Hara’s 

model of NPP operator tasks.

2.3.2 O’Hara’s Model of NPP Operator Generic Tasks

John O’Hara and colleagues have developed a generic characterization of NPP operator 

performance that has been applied in many NRC human failure event (HFE) guidance 

development efforts (O’Hara et al., 2008). This model does not use the term “macrocognition,” 

but it describes the basic categories of operator activities to accomplish control room tasks. 
18

According to O’Hara, operators perform two types of tasks: primary tasks and secondary tasks.

Primary tasks include activities such as monitoring plant parameters, following procedures, 

responding to alarms, and operating equipment (e.g., starting pumps and aligning valves). The 

secondary tasks of interest are “interface management tasks.” Primary tasks have a number of 

common cognitive elements: monitoring and detection, situation assessment, response 

planning, and response implementation. O’Hara referred to these common cognitive elements 

as generic primary tasks. The generic primary tasks are similar to those macrocognitive 

functions in the Klein et al. (2003) model. Breakdowns in any of these generic primary tasks 

can cause a human error. Figure 2-5 shows the diagram of O’Hara’s model. Each of these 

generic primary tasks is discussed in more detail below.

Figure 2-5. O’Hara’s cognitive model of NPP control room operations, adapted from 

O’Hara et al. (2008).

Monitoring and Detection: According to O’Hara et al. (2008), the task of monitoring and 

detection involves extracting information from the environment, such as checking the 

parameters on a control panel, monitoring parameters displayed on a computer screen, 

obtaining verbal reports from other personnel, and sending operators to areas of the plant to 

check on system components. From this information, operators determine if the plant is 

operating as expected. In a highly automated plant, much of what operators do involves 

monitoring. Detection is the operator’s recognition that something has changed (e.g., a 

component is not operating correctly), or the value of a parameter has increased or decreased.

Situation Assessment: Situation assessment is the evaluation of current conditions to determine 

if they are within acceptable limits, or to identify the underlying causes of any abnormalities. 

Operators actively try to construct a coherent, logical explanation to account for their 

observations. According to O’Hara et al. (2008), this cognitive activity involves two related 

concepts: the situation model and the mental model. The mental model consists of the 

operator’s internal representation of the physical and functional characteristics of the plant and 

its operation, as they understand it should be. This model rests upon formal education, training, 

and experience. Situation assessment occurs when operators use their mental model to 

understand information they obtain from the human-system interfaces (HSIs) and other sources.

The cognitive representation resulting from situation assessment is termed the “situation 

model,” which refers to the understanding that personnel have of the plant’s current situation, 

(i.e., their current situation model). The alarms and displays serve to generate information 

supporting situation assessment. The HSIs may provide additional support to situation 

assessment in the form of operator-support systems.
19

To construct a situation model, operators use their mental model, i.e., the general knowledge 

and understanding about the plant and its operation, to interpret their observations and to 

extract its implications. Limitations in knowledge or in current information may entail incomplete 

or inaccurate situation models. The mental model consists of the operator’s internal 

representation of the physical and functional characteristics of the plant and its operation as 

they understand it should be. The mental model rests on formal education, training, and 

experience.

Response Planning: According to O’Hara et al. (2008), response planning refers to deciding 

upon actions to resolve the current situation. In general, it involves operators using their 

situation model to identify goal states and the transformations required to achieve them. The 

goal state may vary, such as identifying the proper procedure, assessing the status of back-up 

systems, or diagnosing a problem. To meet their goals, operators generate alternative 

response plans, evaluate them, and select the one most appropriate to the current situation 

model. Response planning can be as simple as selecting an alarm response or it may involve 

developing a detailed plan when existing procedures proved incomplete or ineffective.

In an NPP, procedures usually aid response planning. The need to generate a response plan in 

real time largely may be eliminated when operators trust that the procedures are suitable to 

meet the current problem. However, even with good procedures, operators will undertake some 

aspects of response planning. For example, they still need to (1) identify goals based on their 

own situation assessment, (2) select the appropriate procedure(s), (3) evaluate whether the 

procedure-defined actions are sufficient to achieve those goals, and (4) adapt the procedure to 

the situation, if necessary.

Response Implementation: This entails performing the actions specified by the response plan, 

which may include such actions as selecting a control, providing control input, and monitoring 

the responses of the system and processes. Several types of errors are associated with 

controls. One example is mode errors, a new type associated with digital technology. A mode 

error occurs when operators take an action thinking the control system is in one mode when 

actually it is in another. Consequently, the system’s response to the action is not what the 

operator expected.

While the O’Hara et al. model does not describe “coordination” as one of the generic tasks, it 

does point out the importance of teamwork in NPP operations. As O’Hara et al. (2008) noted:

Individual operators typically do not undertake these tasks alone; they are 

accomplished by the coordinated activity of multi-person teams. 

Operators share information and work in a coordinated fashion to 

maintain the plant’s safe operation as well as to restore it to a safe state 

should a process disturbance arise. Crew members may perform a task 

cooperatively from one location, such as the main control room, while in 

other cases a control room operator may have to coordinate tasks with 

personnel in a remote location, such as at a local control station. 

Important human factors engineering aspects of teamwork include having 

common, coordinated goals, maintaining shared situation awareness, 

engaging in open communication, and cooperative planning. Successful 

teams monitor each other’s status, back each other up, actively identify 

errors, and question improper procedures” (p. 7).

O’Hara’s model suggests that the four generic primary tasks, at least partially, are processed in 

serial:

Monitoring/DetectionSituation AssessmentDecision and PlanningAction Implementation
20

O’Hara’s model did not address the relationship between the generic primary tasks and how 

they work together to accomplish complex tasks. The descriptive model of NPP control room 

teamwork presented next was intended to fill this gap. 

2.3.3 Integrating Macrocognition with NPP Model

This chapter discussed three macrocognitive models and three cognitive models used in the 

NPP domain. We evaluated these models and analyzed their applicability to NPP operator 

performance. Through the analysis, we decided to adapt the strengths of these models and 

reconcile them into a single macrocognition model for HRA use. Table 2-1 presents a 

side-by-side comparison of the models.

Table 2-1. Comparison of the macrocognitive and NPP models.

Model Roth (2010a, 

2010b)

Klein et al. (2003) Patterson & 

Hoffman (2012)

Chang & Mosleh 

(2007a)

O’Hara et al. 

(2008)

Macrocognitive 

Functions/ 

Information 

Processing 

Stages/ 

Detecting/noticing Problem detection Detecting problems Information 

pre-processing

Monitoring and 

detection

Understanding and 

Sensemaking

Sensemaking/ 

situation assessment

Sensemaking N/A Situation 

assessment

Planning /deciding Planning; 

Adaptation/ 

replanning; and 

Naturalistic 

decisionmaking

Deciding; 

Replanning

Diagnosis/problem 

solving and 

decisionmaking

Response 

planning

Action N/A N/A Action Response 

implementation

Communicating/ 

coordinating 

(teamwork 

functions)

Coordination Coordinating Crew coordination 

and interactions

Crew coordination

Relationship of 

the Functions/ 

Stages

Processing occurs 

in a continuous 

loop. It can be 

serial, but it also 

can start at any 

function and move 

in any direction

All the functions are 

in a dynamic, 

continuous loop, with 

many or all of the 

functions occurring 

at the same time, 

depending on the 

situation

Functions are 

overlapping and 

interact dynamically, 

depending on the 

situation

Phases are in a 

serial loop. Cognition 

flows in a series of 

IDA loops with 

nested sub-loops

The main stream 

of cognitive 

processing is 

serial from 

monitoring/ 

detection to 

response 

implementation

As shown in Table 2-1, a notable difference between the macrocognitive models and the NPP 

models is that the NPP models assume linear, serial flow, while the macrocognitive models do 

not. The macrocognitive models acknowledge that human cognition is more complex than what 

serial information-processing models can account for. As discussed by Roth (2010a, 2010b), 

Klein et al. (2003), and Patterson and Hoffman (2012), the cognitive process can start in any of 

the macrocognitive functions and move in any direction, involving parallel and cyclical 

processing in addition to serial processing.

The nested IDA concept in IDAC recognizes that the cognitive functions are not independent of 

each other. It is similar to Patterson and Hoffman’s overlapping function model in that IDAC 

recognizes that perceiving information involves some aspects of decisionmaking and action; 

decisionmaking involves aspects of gathering information, diagnosis, and action; and that taking 

action involves aspects of detecting information and decisionmaking. 

Another difference between the models is in dividing the continuous span of cognition into 

discrete macrocognitive functions. Roth (2010a, 2010b) identified five functions by combining 

planning with deciding. Klein et al. identified six functions (2003), and Patterson and Hoffman 

identified five (2012). Both the Klein et al. model and the Patterson and Hoffman model 
21

separated planning or re-planning from decisionmaking, and neither included action. The NPP 

models, on the other hand, have fewer granularities. IDAC has only four functions, with 

diagnosis and problem solving combined with decisionmaking instead of standing as a separate 

function. O’Hara’s model only contains four generic primary tasks, though the model does 

discuss the importance of teamwork.

Despite the differences, there are remarkable similarities between the macrocognitive models 

and the NPP models, given that they were developed from psychological and engineering 

perspectives, respectively. The four generic primary tasks plus crew coordination used in the 

O’Hara et al. model are essentially the same as the functions generalized by Roth, and if one 

combines planning, re-planning and decisionmaking in the Klein et al. (2003) and Patterson and 

Hoffman (2012) models, this produces comparable categories, with the exception of action.

IDAC has only four functions; however, IDAC combines diagnosis (or forming an understanding 

of the situation) and problem solving with decisionmaking. If one separates that part out, it 

produces five functions that look nearly identical on the surface to O’Hara et al. (2008) and Roth 

(2010a, 2010b).

Several key features were generalized from these models. Primarily, these include an 

understanding that the macrocognitive functions overlap, and that they dynamically interact with 

each other in a continuous, non-linear loop, involving parallel and cyclical processing. Given 

that the purpose of the literature review is to develop a cognitive framework that serves as a 

foundation for HRA in the NPP domain, and given that both of the NPP models have five 

functions (having separated diagnosis from decisionmaking in IDAC), we decided to adapt these 

functions into the macrocognitive model for Integrated Human Event Analysis System 

(IDHEAS):

• Detecting and Noticing

• Understanding and Sensemaking

• Decisionmaking

• Action

• Teamwork

The next section provides basic definitions of each of these five macrocognitive functions, and 

subsequent chapters in this report discuss each function in detail.

2.4 The Macrocognition Model for HRA

As discussed in the previous section, we adapted a macrocognition model that consists of five 

macrocognitive functions as the cognitive framework for IDHEAS. This section first provides 

brief descriptions of each macrocognitive function, and then describes the relationship between 

the functions.

2.4.1 Definitions of the Macrocognitive Functions Used in this Approach

Detecting and Noticing 

Detecting and Noticing is the process of perceiving important information in the work 

environment. Emphasized in this macrocognitive function are the sensory and perceptual 

processes that allow humans to perceive large amounts of information and focus selectively on 

those pieces of information that are pertinent to present activities.

The cognitive processes associated with Detecting and Noticing are those related to the 

psychological processes of sensation, perception, and attention. Sensation has a large capacity 

for input, but sensory information can only be retained for a short interval before being replaced 
22

by new sensory information (Atkinson & Shiffrin, 1968; Baddeley, 2000; Baddeley & Hitch, 

1974). Moreover, a person can only attend to a limited amount of information at a time. 

Attention determines which pieces of sensed information are processed and enter into the 

human’s awareness. Raw information must be filtered or selected, and meaningful information 

is processed and extracted for further cognitive processing. When there is too much meaningful 

sensory information, the individual may not be able to detect and notice all of that information, 

resulting in sensory overload. Conversely, a lack of salient sensory information may cause 

important plant information to go undetected or unnoticed. 

Understanding and Sensemaking 

The macrocognitive function Understanding and Sensemaking is the process of understanding 

the meaning of the information that has been detected. Models and processes that fit into this 

function include the data/frame theory of sensemaking, situation awareness (SA), interpretation 

of pieces of information, and integrating multiple pieces of information together into a diagnosis. 

Cognition in this function ranges from automatic, effortless recognition and understanding to 

more effortful thinking and deliberate attempts to make sense of multiple pieces of information.

The model used to integrate all of the various approaches to understanding for the purposes of 

this effort is the data/frame theory of sensemaking (Klein, Moon, & Hoffman, 2006; Klein, et al., 

2007). This model is an extension of Neisser’s (1967) perceptual cycle theory, in which a 

person’s sampling of the environment updates their cognitive map of the environment, which in 

turn directs further exploration. The data-frame theory of sensemaking posits that information 

coming into the sensemaking process is data. This data is integrated with an existing frame, 

which is a mental representation that serves as a structure for explaining the data and guiding 

the search for more data (Klein, et al., 2007). The data identify or construct the frame, and the 

frame determines which data are attended to. Neither the data nor the frame comes first; 

rather, it is a constant process of moving back and forth from data to frame. This dynamic 

aspect is the central theme of the model.

This model is ideal for the cognitive framework because it identifies the three primary sources of 

failure of Understanding and Sensemaking: the data (e.g., the operator has the wrong 

information), the frame (e.g., the operator is using an incorrect system model to understand the 

situation), or the integration of the two (e.g., new information is not properly integrated with the 

frame).

Decisionmaking

Decisionmaking involves goal selection, planning, re-planning and adapting, evaluating options, 

and selection. Decisionmaking within an NPP is characterized as involving experts and being 

largely driven by procedures. NPP control room decisionmaking typically involves routine 

(unconscious) decisions, such as deciding a piece of information is irrelevant, deciding between 

two gauges to obtain information from, or deciding to ask a question; it is not limited to making a 

decision about what action to take in response to the event. Although procedures usually 

dictate the actions of the operators, Roth (1997) explains that the operators still maintain a 

mental model of the situation and will plan their course of action semi-independently of the 

procedures. That is, they will have an idea of what must be accomplished and how it should be 

done and will look to the procedures to confirm these beliefs. Furthermore, situations may arise 

that procedures do not cover. In these instances, operators must rely on their expert knowledge 

to solve the problem and implement the appropriate decision.

Two models that are particularly useful when examining decisionmaking within an NPP are a 

recognition-primed decision (RPD) model (Klein, 1993), and an integrated naturalistic 

decisionmaking model proposed by Greitzer et al. (2010). RPD was primarily developed to 
23

explain decisionmaking in stressful situations and under time pressure. The integrated 

naturalistic model includes concepts from situation awareness theories, and 

recognition/metacognition. This integrated model well describes the processes an experienced 

operator goes through in making a decision with or without explicit procedures. 

Action 

Action is defined as implementing an action on the level of a single manual action (such as 

operating a valve) or a predetermined sequence of manual actions. The action(s) must involve 

the manipulation of the hardware and/or software that would consequently alter plant status. It 

is assumed that the other macrocognitive functions (e.g., Detecting and Noticing, Understanding

and Sensemaking, and Decisionmaking) have been carried out successfully.

Operator actions can take the form of individual control actions (e.g., turning a switch to a 

particular position; turning a pump on or off) or a sequence of actions intended to achieve a 

particular goal. An example of a sequence of actions is realigning a set of valves to change a 

flow path. In some cases, all that is required is a single, discrete, control action to achieve the 

goal. For example, manually tripping the plant generally requires a single button press (or turn 

of a switch). However, in other cases more sustained control is required. For example, in many 

situations operators are required to make continuous adjustments to maintain a parameter 

within a specified set of limits. 

Reason (1990) divided errors of execution into two major forms: slips and lapses. Slips are 

errors where actions executed are “not as planned.” They typically occur in the performance of 

largely automatic tasks performed by skilled individuals in familiar conditions, especially when 

attention is diverted (e.g., because of distraction or preoccupation). Lapses involve failures of 

memory, where an individual may forget to perform an intended action. This is often due to a 

failure in prospective memory—memory for intended actions in the future.

Teamwork

Teamwork is the macrocognitive function that focuses on how people interact with each other to 

coordinate as the individual or crew works on a task. In the present effort, we are using this 

macrocognitive function primarily to include coordination, collaboration, and communication 

between individuals, as well as to address crew interaction issues such as command and 

control.

This macrocognitive function focuses on the emergent aspects of team coordination to avoid 

duplicating cognitive functions already described by previous macrocognitive functions. Building 

on the team sensemaking work of Klein, Wiggins, and Dominguez (2010), these emergent 

aspects are unique to and only emerge when people work together in teams. For example, 

even in a team setting, individual macrocognitive functions like Understanding and 

Sensemaking occur in parallel in all team members, such as a crew’s response to an alarm 

annunciation. Given this context of independent parallel processing, the essence of teamwork 

is the combination of these independent process efforts via communication for purposes of 

facilitating team coordination. Using communication to combine the individual macrocognitive 

processes is one emergent aspect of Teamwork; the other is leadership. NPP control room 

crews are hierarchical and have a distinct leadership structure. Errors in either communication 

or leadership can lead to failure of the Teamwork macrocognitive function.

2.4.2 Relationship between the Macrocognitive Functions

The macrocognitive functions described above represent a complete span of NPP operators’ 

cognition when performing complex tasks. Klein’s model states that the macrocognitive 

functions work together in a loop; Patterson and Hoffman’s model suggests that the functions 

are overlapping, while O’Hara’s model hinted that at least one of the main streams of 
24

information processing is from Monitoring/Detection to Response Implementation (also referred 

to as Action). Each of these models represents one perspective of how the functions work 

together. All the models acknowledged that these functions relate to and support each other,

and they share some common cognitive mechanisms, yet each function is achieved by a unique 

set of cognitive mechanisms and can lead to unique types of human errors. 

Figure 2-6. The five macrocognitive functions.

Figure 2-6 depicts the relationship among the macrocognitive functions. The functions are

parallel and cyclical, and the functions overlap and interact with each other. Each of the 

macrocognitive functions operates in the context of team interaction. Like the Patterson and 

Hoffman (2012) model, the representation shown in Figure 2-6 recognizes that each 

macrocognitive function may involve aspects of the other functions. The figure shows that 

individual cognitive functions play out in the context of team interaction. We recognized that it is 

important to consider cognition at the level of the crew. The macrocognitive functions work 

simultaneously in all operators in the NPP crew, and the crew members must communicate and 

coordinate with each other to ensure that they make the appropriate response to the plant 

conditions. 

2.5 Development of the Cognitive Framework

This section documents the process we took to transform the literature review into a cognitive 

framework. The purpose of the cognitive framework is to identify connections between 

proximate causes, cognitive mechanisms, and influences for the failures. The goal is to identify 

how failure occurs, (i.e., the cognitive mechanisms for human errors and the context (PIFs) that 

may activate those mechanisms).

Because the purpose of HRA is to understand human error and predict the likelihood of errors, 

we structured the literature review and developed the cognitive framework in terms of human 

failure. This meant that the focus was on identifying information that can explain human error.

This required reinterpretation and inferences of the major psychological literature reviewed 
25

because many findings in the psychological literature are focused on how to optimize successful 

human performance. For example, research on attention may discuss factors that are crucial 

for successful performance solely from the perspective of optimal performance. This does not 

necessarily explain what can lead to failure of attention. In such cases, we had to interpret, 

supplement, or make inferences about how the information could apply to failures, based on our 

expertise and other research.

This section describes the genesis of the cognitive framework, including identification of the 

cognitive mechanisms, proximate causes, links of the causes and mechanisms with PIFs, and 

construction of this information into the cognitive framework.

2.5.1 Identification of Cognitive Mechanisms

Cognitive mechanisms are the processes by which macrocognitive functions work. They are 

the processes by which cognition takes place in the work environment, and are thus crucial to 

successful performance. If part of the process fails, this failure may manifest itself as a 

proximate cause of the macrocognitive function failure. An example of a cognitive mechanism 

is working memory, the ability to retain information in completing a task. It is important to note 

that the cognitive mechanisms are vulnerable to fail under certain external or internal factors.

Thus, the mechanisms are the substrates of human performance successes and failures. 

Using this definition, we reviewed the literature in depth to extract knowledge about the 

macrocognitive functions and then identify the underlying processes and explain how errors or 

mistakes can occur, or ways in which psychological models would help describe failures. Some 

models call out specific types of errors. Other models specifically discuss processes that are 

required for success; transforming this information into a mechanism required framing the 

description of the process in failure terms. We often added terms such as “not” or “failed to” to 

the processes that make the functions work.

For example, the following excerpts from Endsley (1995) illustrate a cognitive mechanism for 

the Understanding and Sensemaking function:

“Operators of complex systems frequently employ a process of 

information sampling to circumvent this [attention] limit. They attend to 

information in rapid sequence following a pattern dictated by the portion 

of long-term memory concerning relative priorities and the frequency with 

which information changes.... Working memory also plays an important 

role, allowing one to modify attention deployment on the basis of other 

information perceived or active goals...” (p. 41).

“Failures in information sampling are commonplace, however, and may 

result from the lack of an adequate strategy or internal model for directing 

sampling. Wickens...has also noted that humans have several general 

failings in sampling, including misperception of the statistical properties of 

elements in the environment and limitations of human memory (forgetting 

what has already been sampled)” (p. 55).

The excerpts describe the process of using a mental model to guide information sampling and 

direct attention toward important information. However, when an incorrect mental model is 

used, attention may be directed toward information that is not important and contribute to an 

incorrect understanding of the situation. Consequently, we identified a cognitive mechanism for 

failing the Understanding and Sensemaking function as, “incorrect, inappropriate, or inadequate 

frame used to search for, identify, or attend to information.” Appendix A lists all of the cognitive

mechanisms identified through this process, sorted by macrocognitive function and proximate 

cause.
26

2.5.2 Identification of Proximate Causes

Many cognitive mechanisms were identified for each macrocognitive function. A cognitive 

mechanism can fail to achieve its intended function in different ways. For example, working 

memory is a mechanism for understanding the situation. Failure of working memory, such as 

memory overload, can lead to misunderstanding or incomplete understanding of the situation. 

We identified the outcomes of failure of the mechanisms, and grouped the outcomes into readily

identifiable types of failures of the cognitive functions. We refer to these type of failures as 

proximate causes. Hence, proximate causes are the result of failure of cognitive mechanisms. 

A proximate cause can be associated with several cognitive mechanisms, and vice versa. 

Furthermore, one or several proximate causes may lead to human failure. 

Whereas cognitive mechanisms are scientific findings described in the literature, the set of 

proximate causes is merely a classification scheme used to group the cognitive mechanisms. 

Therefore, there can be different ways to classify the mechanisms. Our goal was to develop a 

defined set of proximate causes that have distinct non-overlapping definitions, and are 

observable, identifiable, or inferable in a practical manner. With many rounds of exploration and 

pilots, we decided to use the major process steps of a macrocognitive function as the framework 

of proximate causes. For example, per the literature, achieving the Detecting and Noticing

function requires the cognitive mechanisms of attending to the cue/information and perceiving 

the cue/information correctly. Therefore, the proximate causes were identified as: 

cue/information not attended to, cues/information not perceived, and cues/information 

misperceived. We developed an initial list of proximate causes, then gradually revised the list 

as we collapsed together items that seemed similar (i.e., that produced similar or overlapping 

effects).

The set of proximate causes identified is listed below.

• Failure of Detecting and Noticing

- Cues/information not attended to

- Cues/information not perceived

- Cues/information misperceived

• Failure of Understanding and Sensemaking

- Incorrect data

- Incorrect frame

- Incorrect integration of data, frames, or data with a frame

• Failure of Decisionmaking

- Incorrect goals or priorities set

- Incorrect pattern matching

- Incorrect mental simulation or evaluation of options

• Failure of Action

- Failure to execute desired action

- Execute desired action incorrectly

• Failure of Teamwork 

- Failure of team communication

- Failure in leadership/supervision
27

We describe the cognitive mechanisms and proximate causes in generic terms from a 

psychological perspective to ensure applicability over a wide range of situations, including 

human cognition within and outside of the control room. Yet, since the initial scope of the 

literature review was primarily limited to cognitive mechanisms that support cognitive activities 

of NPP operators in internal, at-power events, some cognitive mechanisms that are pertinent to 

situations outside the scope, such as those supporting dynamic, distributed decisionmaking, 

were not included in the review. Subsequently, the proximate causes, while a good coverage 

for NPP internal, at-power events, may not cover the entire operation span of NPP events such 

as several accident management or low-power shutdown. 

2.5.3 Performance Influencing Factors

The circumstances or contextual factors that contribute to human performance in a work 

environment are referred as to performance influencing factors (PIFs), or performance shaping 

factors. PIFs can either reduce or increase the likelihood of error. PIFs are commonly used in 

human reliability analysis (HRA) methods to adjust the human error probability (HEP) depending 

on the context of the situation, and they are also commonly used to identify root causes of 

errors and areas for improvement. We defined PIFs as contextual factors (including plant 

factors) that influence the likelihood that the cognitive mechanisms “activate” and lead to the 

proximate causes of macrocognitive function failure. 

Every HRA method has its own set of PIFs. At present, there is not a standard way of defining 

PIFs. Groth (Groth, 2009; Groth & Mosleh, 2012) assembled a list of PIFs from a number of 

HRA methods. The PIFs are sorted into five factors or categories: 

• Organization-based factors: factors that are under the control of the organization and 

include the organization’s attitudes and behaviors that affect human performance.

• Team-based factors: factors related to the team that is working together to achieve a 

common goal and the quality of their interactions.

• Person-based factors: factors internal to each individual and encompass a person’s state of 

mind, temperament, and other intrinsic characteristics.

• Situation/stressor-based factors: characteristics of the scenario that are external to the 

person and to the system, and that are likely to influence human performance. Stressor 

PIFs are the demands of the situation that the person perceives as tension or pressure that 

can disrupt or facilitate performance. 

• Machine-based factors: factors related to the system as designed by the vendor or 

manufacturer, including the building structure, mechanical and electrical components, 

hardware, and software.

Each of these categories consists of many specific factors, as shown in Table 2-2. In our 

literature review, we identified empirical findings about how cognitive mechanisms may fail 

under various experimental manipulations or operational contexts. Based on these findings, we 

used the list in Table 2-2 to make inferences about what PIFs may affect a given cognitive 

mechanism. The links are summarized in Appendix A. 
28

Table 2-2. Complete PIF taxonomy as organized by Groth & Mosleh (2012).

Organization-Based Team-Based Person-Based

Situation/

Stressor-Based

Machine-Based

Training Program Communication Attention External Environment HSI

• Availability • Availability • To Task Conditioning Events • Input

• Quality • Quality • To Surroundings Task Load • Output

Corrective Action Program Direct Supervision Physical and Time Load System Responses

• Availability • Leadership Psychological Abilities Other Loads

• Quality • Team member • Alertness • Non-task

Other Programs Team Coordination • Fatigue • Passive Information

• Availability Team Cohesion • Impairment Task Complexity

• Quality Role Awareness • Sensory Limits • Cognitive

Safety Culture • Physical Attributes • Task Execution

Management Activities • Other Stress

• Staffing Knowledge/Experience Perceived Situation

o Number Skills • Severity

o Qualifications Familiarity with Situation • Urgency

o Team composition Bias Perceived Decision

• Scheduling Morale/Motivation/Attitude • Responsibility

o Prioritization • Impact

o Frequency o Personal

Workplace Adequacy o Plant

Resources o Society

• Procedures

o Availability

o Quality

• Tools

o Availability

o Quality

• Necessary Information

o Availability

o Quality

2.5.4 Creating the Cognitive Framework: Connecting Proximate Causes, 

Cognitive Mechanisms, and PIFs

One requirement for the literature review was to build a framework that connects failures of 

macrocognitive functions with proximate causes, cognitive mechanisms, and PIFs. The 

purpose of such a tool is to identify how failure occurs. For the possible proximate causes of 

failure of a macrocognitive function, the cognitive framework identifies the potential cognitive

mechanisms for human error, and what contexts (PIFs) may activate those mechanisms. We

took all four of the elements discussed above—macrocognitive functions, proximate causes, 

cognitive mechanisms, and PIFs—and organized them into the cognitive framework, a tree 

structure that illustrates how macrocognition may fail and describes the reasons why. Each 

macrocognitive function is represented with one tree, yet the tree branches are not necessarily 

orthogonal to each other. Specifically, we have endeavored to make the proximate causes as 

independent from each other as possible. The generic structure for each tree in the cognitive 

framework is shown in Figure 2-7. Starting from the left in the figure, the first box represents the 

macrocognitive function that the tree is analyzing. Boxes in the next column represent the 

proximate causes of failure for the function. Each proximate cause is then linked to a number of 
29

cognitive mechanisms. Each cognitive mechanism is connected to the relevant PIFs for that 

mechanism. The flow moves from left to right: for a given cognitive function, the tree allows one

to identify the proximate causes of the failure of the function, the cognitive mechanisms leading 

to the causes, and the PIFs contributing to the causes through the mechanisms. Note that the 

diagram depicts an ideal situation: a given set of PIFs only contribute to one cognitive

mechanism, and a given set of mechanisms only contribute to one proximate cause. In reality, 

different causes can be associated with some common mechanisms, and the same cognitive

mechanism may lead to more than one proximate cause. The same is true for the connections 

between PIFs and the cognitive mechanisms. 

Figure 2-7. Generic cognitive framework: links between PIFs, cognitive mechanisms, 

proximate causes, and macrocognitive functions.

Each tree of the cognitive framework (in Appendix B) corresponds to a set of detailed tables (in 

Appendix A) that provide supporting information for the psychological basis of each node in the 

tree. Together, Appendix A and Appendix B are the outcomes of the literature review: a 

cognitive framework tool that serves as the psychological foundation for the qualitative and 

quantitative analysis methodology of the IDHEAS HRA method. 

In addition, the cognitive framework can be used more generally to identify causes, 

mechanisms, and PIFs to consider for any situation involving human error. The framework 

gives analysts a structured tool, based on psychological research, for identifying the types of

factors likely to be relevant for a given human failure event. As a result, the cognitive framework

may prove useful to other HRA methods or human factors applications.

An excerpt from Table A.1.1 in Appendix A (the Detecting and Noticing macrocognitive function) 

is shown in 

Table 2-3. This excerpt contains two cognitive mechanisms that explain the proximate cause of 

cues/information not perceived. The tables in Appendix A are organized by macrocognitive 

function and proximate cause. The first column in each table contains the cognitive mechanism 
30

that can lead to the proximate cause. The second column provides a discussion of how the 

mechanism works. The third column provides an example. Next, the table lists the relevant 

PIFs for the cognitive mechanism, followed by an explanation of how or why the PIF is relevant.

The last column provides references.

Table 2-3. Excerpt from Table A.1.1 listing cognitive mechanisms and PIFs for the 

proximate cause Cues/information not perceived.

Mechanism Discussion Example Relevant PIF(s) Explanation References

Working 

memory

Broadbent’s “filter 

theory” explains sensory 

bottlenecks—the alarm 

might be missed 

because humans have a 

limited ability to take in 

stimulus inputs and 

perceive them. It is 

possible to reach 

sensory bottlenecks, 

which can result in 

sensory overload.

Filter theory has direct 

implications for plant 

environments like 

control rooms, where 

concurrent tasks and 

multiple simultaneous 

alarms may lead to 

sensory overload and 

cause operators to 

miss some relevant 

information.

• Task load Sensory 

bottlenecks 

dictate ability to 

perceive 

information.

Broadbent (1958)

Cue content The human’s ability to 

detect or sense stimuli is 

a function of how clearly 

that stimuli is present 

over and above existing 

background noise (i.e., 

the cue’s signal to noise 

ratio).

The cue is not 

presented to the 

operator with 

sufficient 

strength/energy to 

distinguish itself from 

existing background

noise such that it 

activates a sensory 

response in the 

operator.

• HSI

• Task load

• Task complexity

• Stress

The signal to 

noise ratio is too 

low for the 

operator to be 

able to perceive 

the cue (i.e., 

distinguish it 

from noise).

Bustamante 

(2008)

The two cognitive mechanisms included in the excerpt in 

Table 2-3 are working memory and cue content. People have limits to the amount of 

information they can process at once. Information overload occurs when more information is 

sent to a person than that person is capable of taking in, or when the person is processing other 

information and as a result cannot take in anything new (Broadbent, 1958). When this happens, 

important data may be missed. Because of this, task load is a PIF that can influence the 

likelihood of information overload occurring.

The cue content cognitive mechanism explains that if the cue is not salient enough to be 

distinguished from the background noise, it is more likely to be missed (Bustamante, 2008). 

Therefore, the amount and nature of the output from the HSI is important—is the critical 

information buried in flashing lights and noise? Workload and stress are also issues, because

with increasing task load, complexity, and stress, the amount of mental resources that are 

occupied with the task increases and leaves less available to attend to cues. With a heavy 

workload, cues that are not particularly salient, which would normally be noticed in low workload

situations, may be missed. Stress is known to cause attentional narrowing (Nikolic, Orr, & 

Sarter, 2004), or to decrease the amount of information one can attend to. This also can lead to 

missing a cue if the cue is not particularly salient.

As shown by the example in 

Table 2-3, one of the most important developments in the cognitive framework is that there are 

explicit connections between PIFs, cognitive mechanisms, and proximate causes of failure. 

Specifically, the tables provide explanation about which PIFs are important and why, provide 

information about how PIFs influence cognitive errors, and put this information in a simple tool 

that can inform HRA and other applications. 
31

2.6 Summary

This chapter provided an overview of macrocognition and explained the process we used to 

identify a macrocognitive model for the IDHEAS HRA method. We reviewed major models of 

macrocognition and two NPP-specific models. Comparing the models demonstrated that they 

were largely consistent, though they use slightly different terminology for the same concepts. 

Furthermore, the models are complementary. We adapted these models into a generalized 

cognitive framework for use in the IDHEAS HRA method. This framework has five 

macrocognitive functions:

• Detecting and Noticing

• Understanding and Sensemaking

• Decisionmaking

• Action

• Teamwork

We conducted a review of the behavioral sciences literature related to each of the above 

macrocognitive functions to identify cognitive mechanisms, causes of failure (proximate 

causes), and the effects of PIFs. We then organized the information gleaned from the review 

into tables (Appendix A) and the cognitive framework (Appendix B). Together, the cognitive 

framework and the accompanying tables form the technical basis for the IDHEAS HRA method, 

and can also be useful to other HRA methods or human factors applications.

33

3. DETECTING AND NOTICING

3.1 Introduction

The macrocognitive function of Detecting and Noticing refers to perceiving important information 

in the work environment. Detecting and Noticing emphasizes how humans selectively process 

the large bulk of information that they have sensed and focus on the information that is pertinent 

to tasks. This chapter first introduces Detecting and Noticing as a macrocognitive function in 

general and its specific involvement in performing nuclear power plant (NPP) tasks. The 

chapter then summarizes the cognitive mechanisms underlying Detecting and Noticing, followed 

by proximate causes of failure in Detecting and Noticing. Lastly, the chapter presents examples 

from the literature that demonstrate the impact of certain performance influencing factors (PIFs)

on the success or failure of the Detecting and Noticing function. We generalized the 

psychological literature findings to provide a cognitive basis of what leads to human failures in 

detecting and noticing critical task-relevant information. This information is summarized in 

Appendix A.1.

3.1.1 Sensation and Perception

The cognitive processes associated with Detecting and Noticing include sensation, perception, 

and attention. Sensation, broadly defined, involves the input of environmental stimulus 

information into the sensory organs. This information may encompass the traditional five 

senses and may be visual, aural, olfactory (related to smell), gustatory (related to taste), or 

tactile. Additional sensory information includes proprioception (related to the position of parts of 

the body), kinesthesia (related to the sensation of motion), equilibrioception (related to the 

body’s balance), chronoception (related to perception of time), thermoreception (related to 

sense of temperature), and nociception (related to sense of noxious stimuli and pain). For the 

purposes of nuclear power plant operations, the main sensory modalities are visual and aural, 

although some consideration is given to tactile inputs.

Perception is the point at which sensory information is first given meaning. Sensory information 

can be seen as raw input information. Sensation is largely a biological process and holds 

relatively constant across people. Different normal-sighted individuals, for example, do not 

sense the color blue differently from each other. However, at the point at which blue is imbued 

with meaning, that meaning is subject to the cognition of the individual beholding it. Blue may 

be a favorite color for one individual yet a disliked color for another individual. Moreover, the 

experience of particular individuals may impact their perception of that color. One individual 

may have been trained to associate blue with a positive flow condition, while another individual 

may have no functional associations with the color. Perception is a byproduct of memory—the 

individual’s past experiences and training will affect the meaning associated with a particular 

perception. Memory infuses sensation with meaning to become perception.

3.1.2 Detecting and Noticing across Sensation and Perception

Human cognition is not a strictly serial process of inputting information, giving it meaning, and 

then making sense of it. Throughout information processing, meaning is created by merging 

sensory information with cognition and memory. The flow of information is bidirectional, and 

meaning may be imbued early in the sensory process to enable sensory filtering. As described 

earlier in Chapter 2, the five macrocognitive functions are overlapping and support each other, 

yet each function has its own focus and roles in achieving the tasks.

Figure 3-1 illustrates the different focuses of sensation and perception across the Detecting and 

Noticing and Understanding and Sensemaking functions. In terms of cognitive processes, cues 

and information from the environment are initially sensed. At this stage, the environmental cues 
34

and information are represented as raw sensory signals. This sensory information is perceived 

by giving it initial meaning. Essentially the sensory signals are categorized into perceptual

objects. As additional meaning is placed on these perceptual objects and they are synthesized, 

cognition is said to take place. At this stage, the objects are semantic in the sense that they 

represent meaningful entities that are understood. The progression of information can be seen 

in the example from spoken speech at the bottom of the figure. Speech is initially sensed as 

sound waves and encoded neurologically. As meaning is extracted from the raw speech signal, 

the sound emerges phonetically as /sh/. /sh/ is still a relative abstraction, but it is a meaningful 

sound or phoneme in the English language. These phases of sensing and perceiving are the 

areas covered in Detecting and Noticing. Additional meaning is attached to the phoneme, and it 

takes on word or word-like meaning. The sound /sh/ in isolation has a meaning (“Quiet!”) in 

English. As that meaning is deduced, the corresponding phase is captured by Understanding 

and Sensemaking.

Figure 3-1. The relationship between the macrocognitive functions of Detecting and 

Noticing and Understanding and Sensemaking.

The relationship between sensation, perception, and cognition can be seen from a biological 

perspective. The sensory receptors such as the rod and cone cells found within the retina 

provide the raw sensory input to the visual system. As this information reaches the visual 

cortex, it is perceived. Finally, as information in the visual cortex travels to the associative 

cortex, it becomes cognition.

A further example helps to illustrate the distinction between sensation, perception, and 

cognition. When a control room operator sees an annunciator illuminate, the operator initially 

senses the light from the light box. In parallel, the operator sees the shapes and forms of the 

surrounding, unilluminated alarm tiles. Both illuminated and unilluminated alarms contain equal 

sensory information (although illumination results in higher sensory intensity). However, 

illuminated and unilluminated alarms do not have equal meaning. The illuminated alarm tile has 

a different meaning from the unilluminated alarm tiles surrounding it. The operator perceives 

the illuminated versus unilluminated alarm tiles and associates meaning with them. This 

semantic difference is the essence of perception. Even though the operator has perceived the 

“alarmness” of the alarm, he or she still has not made sense of the alarm or understood its 
35

explicit meaning. Its perceptual meaning is that it is a lighted light box needing attention—a 

composite mental representation of feature detection, object recognition, and sensory salience. 

Its cognitive meaning—imbued with focused sensemaking and understanding beyond the 

generic representation of perception—is that of a specific alarm, and the operator responds 

according to the specific alarm.

Sensation has a large capacity for input, but sensory information can only be retained for a short 

interval before being replaced by new sensory information (Atkinson & Shiffrin, 1968; Baddeley, 

2000; Baddeley & Hitch, 1974). Moreover, only a limited amount of sensed information is 

processed by the brain’s information processing systems and enters into the human’s 

awareness or percepts. For this reason, raw information must be filtered or selected and 

meaningful information is processed and extracted for further cognitive processing. This 

function is referred to as Detecting and Noticing. While sensation is a process that occurs at the 

biological or hardware level, perception is analogous to control systems where hardware 

processing can be modulated by software algorithms. When there is too much meaningful 

sensory information, the individual may not be able to detect and notice all of that information, 

resulting in sensory overload. In summary, Detecting and Noticing represents the process of 

detecting meaningful signals from a large bulk of information received by the sensory organ.

Literature has broadly suggested the following purposes of Detecting and Noticing:

• detecting salient cues in the environment

• detecting changes or new cues onset in the environment

• detecting weak but mission-critical cues from an information-rich environment

• noticing abnormal cues

• searching for specific cues pertinent to a task

These purposes are applicable to operators’ tasks in NPPs as well and actually define a 

significant portion of the operators’ monitoring tasks in the main control room (Morray, 1986). 

NPP operators monitor plant state during normal, abnormal, and urgent operating conditions.

The control room provides a series of stimulus cues to the operators—instrumentation and 

control states, alarms, and written procedures draw the attention of the operators and are 

detected and noticed accordingly (Vicente, Mumaw, and Roth, 2004). A failure to detect and 

notice important cues in the control room can lead to delays in diagnosing plant states, 

misdiagnosis of plant states, or even failure to diagnose plant states requiring response. As 

such, Detecting and Noticing is a crucial part of the control room operators’ contribution to plant 

safety in the main control room.

Mumaw et al. (1994, 2000) reported that monitoring demands in an NPP are often different from 

those in the types of monitoring tasks that have been examined in laboratory settings. An NPP 

has thousands of parameters that operators can potentially monitor. Further, monitoring is not 

the only activity in which power plant operators are engaged; monitoring activity is interweaved 

with other ongoing responsibilities of managing day-to-day tasks for generating power. Mumaw 

et al. (2000) performed cognitive field studies to understand NPP operators’ monitoring task. 

They found that what makes monitoring difficult is not the need to identify subtle abnormal 

indications against a quiescent background, but rather the need to identify and pursue relevant 

findings against a noisy background. Operators devised proactive strategies to make important 

information more salient or reduce meaningless change, create new information, and off-load 

some cognitive processing onto the interface. These findings emphasize the active 

problem-solving nature of monitoring, and highlight the use of strategies for knowledge-driven 

monitoring and the proactive adaptation of the interface to support monitoring.
36

The scope of the literature review was limited to those relevant to monitoring/detecting activities 

in NPP control rooms during internal, at-power events. We made the following assumptions to 

the activities: (1) operators always know what to monitor/detect, per their training and 

procedures, (2) the cue/information to be detected has no uncertainty; that is, operators always 

know and can recognize the meaning of a cue/information as long as the cue is correctly 

perceived, and (3) the information is effortlessly used for the next cognitive function, 

Understanding and Sensemaking. These assumptions set forth the broadness of the literature 

search. As for the levels of details and depth of literature search, the literature reviewed in this 

chapter excludes psychological literature in a number of well-documented areas in sensation 

and perception:

• sensory and perceptual impairments such as colorblindness or prosopagnosia (impaired 

facial recognition), which are presumed not to be relevant to control room operators or 

operations;

• neuroimaging studies, which provide insights into the neurobiological underpinnings of 

sensation and perception, but do not typically provide functional accounts of processes that 

map to proximate causes; and

• psychophysical research, which primarily reviews the intersection of sensation, perception, 

and the use of psychological scales, which is not a typical control room activity.

It must also be noted that almost no literature on sensation and perception is specific to the 

domain of nuclear energy. The findings from other domains such a basic psychology, human 

factors, and aviation psychology must be extrapolated to account for control room phenomena. 

As such, the examples involved make use of subject matter expertise in psychology and nuclear 

power operations to generalize the psychological findings. However, sensation and perception 

are largely invariant across humans and across applications. Sensation and perception occur at 

the physiological level, and the processes are consistent across domains. Therefore, it has 

been unnecessary to perform such research specifically for nuclear power applications. Nuclear 

regulatory guidance on those factors that affect sensation and perception (e.g., lighting levels or 

font size for readability) has been derived from research in other areas (NUREG-0700; O’Hara, 

Brown, Lewis, & Persensky, 2002) and the generalization to nuclear has been straightforward 

and widely accepted.

3.2 Cognitive Mechanisms for Detecting and Noticing

3.2.1 Behavioral Aspects of Detecting and Noticing

Sensory and perceptual information may be passively detected or actively sought. This 

distinction is explained through information foraging theory (Pirolli, 2007), stating that sensory 

and perceptual information gathering is akin to foraging for food. In the wild, organisms 

alternate between grazing in fruitful patches of food and actively foraging for food. By analogy, 

humans feed on sensory information in the environment. Humans alternate between grazing 

information that is readily available and actively foraging for new information. Control room 

operations alternate between phases when the operator is actively looking for information about 

key plant parameters and periods where sufficient information can be found within a limited area 

of focus (Boring, 2011). During a period of passive information grazing, the operator receives 

salient information that pops out of the scene and captures his or her attention. During a period 

of active information foraging, the operator seeks information that is important to the task goals. 

In NPP control rooms, procedures or knowledge may guide the degree to which operators 

actively versus passively seek specific information (Massaiu, Hildebrandt, & Bone, 2011). So, 

this information may guide the operator where to look and it may change “what” information to 

search for, but it cannot change how the operator senses or perceives that information. For 
37

example, experienced control room operators will actively seek specific plant information, 

whether guided by procedure or based on the operators’ plant knowledge. The operators will 

sense and perceive the same information—the only difference being whether the operators 

actively seek that information or passively receive it.

3.2.2 Overview of Mechanisms for Detecting and Noticing

Figure 3-2 shows a diagram of the cognitive processes underlying the visual Detecting and 

Noticing function. The processes for auditory information follow a very similar pattern.

Figure 3-2. Diagram of cognitive processes for the Detecting and Noticing function.

Briefly speaking, visual signals are first sensed by the retina—the sensory organ—and go 

through some preliminary signal processing (such as enhancing the signal-to-noise ratio) by 

subcortical neural structures, then reach the primary visual cortex for preliminary visual 

perceptual processing. From the visual cortex, visual information is processed in two stages: the 

pre-attentive and the attentive stages. The pre-attentive stage is vision without capacity limits. It 

captures salient cues/information at once in the whole visual field by segmenting a visual scene 

and popping out salient stimuli, and the salient information triggers visual attention and is stored 

in working memory. Next at the attentive stage, visual attention directs the fovea to salient 

information for detailed visual perception. These cognitive processes are achieved through 

three major information processing pathways: pre-attentive, bottom-up (stimulus driven), and 

top-down (memory driven):

• The pre-attentive pathway performs visual segmentation and pop-out of salient stimuli.

• The bottom-up pathway processes visual features of the attended visual area of interest, 

combines visual features into integrated patterns, and retains visual objects in working 

memory.

• In top-down processing, the individual strategically directs their attention to current goals 

and expectations due to experience. This is done through memory. The top-down pathway 

guides attention to the fovea or searches for the more interesting information and selectively 

enhances the responses of the visual cortex to specific stimuli (i.e., selective attention).

The micro-cognitive functions involved in these pathways are summarized as follows:

• visual signal processing—sense and pre-process visual signals for perception

• segmentation and pop-out—extract salient information

• visual feature perception—perform preliminary visual analysis of features such as contrast, 

color, shape, and motion
38

• pattern and object integration—integrate multi-dimensional visual features into a coherent 

pattern or object

3.2.3 Cognitive Mechanisms Involved in Detecting and Noticing

The psychological literature has reported many cognitive processes or models involved in 

Detecting and Noticing. Those cognitive processes act like software that works on the three 

pathways delineated in Figure 3-2 to ensure that the Detecting and Noticing function is reliably 

performed. The literature either hypothesized or generalized experimental findings on how 

various cognitive processes work to achieve the Detecting and Noticing function. We grouped 

the cognitive processes reported by the experimental literature into five categories of cognitive 

mechanisms. After a representative list of seminal articles and books for Detecting and Noticing

was identified through the process described earlier in this document, we sorted the references 

into meaningful clusters of related articles.5

 Table 3-1 lists the references from the reviewed 

literature that are pertinent to each category of mechanisms. Note that this list of references is, 

by design, representative but not exhaustive in terms of the psychological phenomena 

documented in the research literature.

Table 3-1. Key literature according to cognitive mechanisms for Detecting and 

Noticing.

Cue Content – Salient 

Process Vigilance in Monitoring Attention Expectation Working Memory

• Biederman (1987)

• Hendy, Farrel, & 

East (2001)

• Hitchcock et al. 

(2003)

• Levi (2008)

• Massaro & Cohen 

(1991)

• Nikolic, Orr & Sarter 

(2004)

• Orasanu & Martin 

(1998)

• Phillips (1974)

• Pirolli (2007)

• Xing (2007)

• Aarts & Pourtois (2010)

• Bustamante (2008)

• Donald (2001)

• Donald (2008)

• Eriksen & St. James 

(1986)

• Jones & Endsley (1996)

• Lavine et al. (2002)

• Lin et al. (2009)

• MacLean et al. (2009)

• Malcolmson, Reynolds & 

Smilek (2007)

• Mumaw, Roth & Burns 

(2000)

• Pirolli (2007)

• Steelman-Allen et al. 

(2009)

• Szalma et al. (2004)

• Vicente (2007)

• Vicente & Burns (1996)

• Vicente, Roth & Mumaw 

(2001)

• Beck, Levin & 

Angelone (2007)

• Bubić (2008)

• Durlach, Kring & 

Bowens (2008)

• Tales, Porter & Butler 

(2009)

• Varakin, Levin &

Collins (2007)

• Vierck & Kiesel 

(2008)

• Einhorn & Hogarth 

(1981)

• Klein (1993)

• Köhler (1947)

• Lipshitz (1993)

• Treisman (1991)

• Brewer et al. (2010)

• Broadbent (1958)

• Caggiano & 

Parasuraman (2004)

• Einhorn & Hogarth 

(1981)

• Endsley (1995)

• Kurby & Zacks (2007)

• Phillips (1974)

• Reicher (1969)

• Soliman (2010)

• Steelman-Allen et al. 

(2009)

The categories simply provide a high-level way to organize the many cognitive phenomena 

related to Detecting and Noticing. The categories represent different types of cognitive 

phenomena. It would be equally possible to categorize the articles differently according to their 

effects on human performance. These five types of cognitive processes may be viewed as 

requirements for detection—these are fundamental cognitive mechanisms that allow people to 

 

5. A formal sorting method called card sorting was employed. This method is commonly used in human factors to 

arrive at categories of related concepts.
39

perceive task-pertinent information in their environment. Each requirement for detection 

presents the opportunity for failure, which provides explanations behind the proximate causes. 

The discussion will first focus on the five types of mechanisms required for Detecting and 

Noticing, and then the discussion will address specific opportunities for failure.

The five different categories of cognitive mechanisms for Detecting and Noticing as identified in 

the research literature are as follows:

• Cue Content – Salient Process. One area that can greatly affect the operators’ Detecting 

and Noticing is the type and quality of cues with which he or she is confronted. Important 

sensory cues may be of a degraded quality, making it difficult for the operator to detect or 

notice them. Important cues may also be buried in an overabundance of other cues, making 

it difficult for the operator to detect the most salient cues. Similarly, the information scene 

the operator confronts may be complex, requiring considerable effort to find the most salient 

cues and necessitating higher cognitive functions such as Understanding and Sensemaking

to glean meaningful cues from the environment. The visual system has segmentation and 

pop-out functions to detect salient targets. Yet, the content of the cue has to be salient 

enough to be detected by these functions.

• Vigilance in Monitoring. Vigilance is marked by the ability to attend to cues over time in 

monitoring tasks. An operator’s ability to attend to or monitor cues will naturally degrade 

over time as a byproduct of fatigue. In addition, the operator’s attention can be affected by a 

number of factors, ranging from the operator’s workload, which degrades attention more 

quickly; to stress, which can interfere with the operator’s ability to attend.

• Attention. Attention is the cognitive process of selectively concentrating on one aspect of 

the environment while ignoring other things. It has also been referred to as the allocation of 

processing resources. Attention guides searching for and monitoring the information of 

interest, focusing visual processing on it, filtering out the unwanted information, and binding 

visual features into an integrated pattern/object. Salient cues can automatically draw one’s 

attention. Attention plays a crucial role in determining what sensory information is 

detected/noticed. As such, attention is a central part of perception and one of the areas that 

determines the success or failure of the Detecting and Noticing macrocognitive function. 

Therefore, attention serves to direct cognitive resources to the most important semantic 

information. Similarly, when the individual is engaged in tasks requiring high levels of 

attention (i.e., high workload), the attentional focus may not allow reapportioning of novel 

sensory information, resulting in missed information.

An important part of control room operations is being able to detect changes to cues. 

Typically, salient changes in cues can pop-out of the visual scene and automatically capture 

one’s attention. However, a number of factors can intervene to make change detection 

difficult. Inattentional blindness describes such a failure. It results when a person fails to 

detect a meaningful cue in the environment, such as when first gazing at a control board, 

even when the cue is salient enough. The factors leading to inattentional blindness include 

not paying attention (e.g., attention is focused on something else), perceptual overload, or

memory overload. A phenomenon related to inattentional blindness is change blindness, 

pursued as a technique to study inattentional blindness. In change blindness, a person will 

fail to notice that a key cue has changed. Typically, this occurs during a visual saccade, in 

which the person briefly averts his or her gaze to look at another cue. Upon return to the 

original cue, the person fails to notice a change because his or her attention was away from 

the change when it occurred.

• Expectation. How operators perceive their environment is subject to certain expectancies 

and biases. Experience primes the operators to expect certain cues in the environment. In 
40

most circumstances, specific cues are the correct ones, and the result is that such cues are 

reinforced for greater reliance in the future. However, cues can be deceiving, and operators 

may come to over-rely on particular cues while missing important contradictory cues needed 

for proper situation assessment. Perceptual biases result when operators over-rely on 

specific cues to the detriment of proper perception.

• Working Memory. Working memory represents memory held for a very brief period. Such 

memory is transitory and is constantly refreshed by new sensory information. Working 

memory represents the focus of attention. As information in working memory is processed, 

it interacts more directly with long-term memory stores. Information that is not processed or 

rehearsed in working memory is lost as it is refreshed with new sensory information. 

Working memory is considered to have a finite amount of information that can be attended 

to or processed at any given time. As more information is added to this, the ability to 

maintain items in working memory degrades. Information-rich circumstances may overload 

the working memory buffer and make it difficult for the operators to maintain information that 

is important to attend to and process. For example, if an operator is comparing a set point 

value with an actual value on a component, the ability of the operator to retain the two 

values in working memory can be greatly hampered if other, salient information from the 

control room is competing for working memory resources. Likewise, the operator may fail to 

take in new information into working memory if he or she is otherwise concentrating on 

specific items in working memory. Information in working memory must be segmented, and 

errors in segmenting are often a cause of incorrect retention in long-term memory (i.e., 

incorrect eyewitness testimony, because details of an event have been incorrectly 

segmented and missing information has been filled in).

There is clearly a degree of interaction between these cognitive mechanisms. For example, 

perceptual expectations in the form of biases may exacerbate change blindness, or working 

memory overload can affect attention allocation. Table 3-1 provides a list of the key literature 

identified for each cognitive mechanism discussed in this section. The specific operator 

performance effects of these categories are discussed in terms of the proximate causes further 

below. The literature sources identified in Table 3-1 are referenced at the end of this chapter for 

further reading in Appendix A-1.

3.3 Proximate Causes for Detecting and Noticing

By our classification scheme, proximate causes correspond to the failure of the major steps of 

the cognitive processes underlying the Detecting and Noticing function. From Section 3.2, we 

can summarize that the major steps for the function include attending for the pertinent 

information, perceiving the information, and processing the meaning of the information (through 

pattern recognition, working memory, etc). Therefore, we identified three proximate causes that 

correspond to the failure of each step: 

• cues and information not perceived

• cues and information not attended to

• cues and information misperceived

Here cue or information refers to information in the environment that is meaningful to task 

performance. A cue is a stimulus in the environment. A salient cue serves as a direct trigger for 

a decision or action. Operator may fail to attend to important perceived information, for 

example. When an individual actively seeks or forages for meaningful stimuli, he or she is 

looking for cues. Likewise, when specific stimuli are broadcast, such as happens with an alarm, 

a cue is being pushed to the operator. A cue may take the form of a status indicator, an alarm, 

or other plant-related details that are meaningful in understanding the status of the plant. Cues 
41

may represent textual information such as the numbers on a gauge, the words on an 

annunciator, or even written procedures. Communication, whether verbal or nonverbal, 

between individuals can also serve as cues. Cues may also take the form of graphical 

(non-textual) displays, sounds such as auditory alerts, or even distinct smells (such as the smell 

of burning wire) that provide indications of the status of the plant.

Note that the definition of a cue in the present context goes beyond visual perception as 

described in the psychological literature. A cue is a feature of an object that helps the individual 

give form to the object. For example, depth perception is an important aspect of determining 

the relative size and position of objects. While some depth information comes from binocular 

disparity between the images captured between the two eyes, a significant amount of 

information can be determined by monocular cues. Monocular cues important for depth 

perception include the texture gradient (there is less detail in objects that are further away), 

interposition (an object that occludes another is closer), size (objects further away are smaller), 

linearity (parallel lines converge in the distance), and atmospheric cues (items in the distance 

tend to be “hazier”). Such features are valid in an ecological context but are seen to be of less 

importance in the carefully constrained world of the control room. Therefore, cues, as used in 

this report, refer to salient information about the plant status, such as trend displays, alarms, 

switch position, and other indicators that help the operator extract meaning about the plant.

A failure of perceiving meaningful cues results in a failure of the subsequent macrocognitive 

function of Understanding and Sensemaking. Here the meaningful cues include all the cues 

that are needed for understanding. For instance, if a level indicator is miscalibrated, the quality 

of a cue content is considered as low and the information is not perceived. In fact, if the 

operators notice that the indicator is not behaving as expected (by comparing with other cues) 

they might still infer the real level, that is, succeed in Understanding and Sensemaking. 

Therefore, the meaningful cues here include the indicator levels and its behavior or changes. 

Cues that exist but are not sensed or perceived are assumed not to trigger the operators to 

respond. The three proximate causes for Detecting and Noticing correspond to three

processing steps in which cues and information are attended, correctly perceived, and 

propagated for subsequent Understanding and Sensemaking:

• Cues and information not attended to. The cues or information may be sensed and 

perceived but not attended to—in other words, the sensory-perceptual system acts as a 

filter, and this information is not propagated for further Understanding and Sensemaking. 

For example, during an incident at the plant, a control room operator may be confronted with 

an alarm flood. The raw sensory cues correspond to the individual alarms that are present, 

but the operator experiences sensory overload and cannot consciously perceive or process 

the large number of simultaneous cues. Or, the operator’s attention may be focused on a 

particularly important part of plant recovery and choose not to respond to certain alarms until 

a more pressing issue is addressed.

• Cues and information not perceived. The cues or information may simply be missed, in 

which case they are not perceived. For example, a control room operator may not notice 

that an important indicator is trending downward.

• Cues and information misperceived. The cues or information may be sensed but 

misperceived. In other words, the sensed information is tagged with the incorrect meaning. 

For example, a control room operator may misread a pump as being on when, in fact, it is 

off.

The proximate causes can be explained in terms of cognitive mechanisms, as seen in Table 

3-2. For example, cues can fail to be perceived because of low cue quality. Cues may fail to be 

attended to because of competing sensory information resulting in too many meaningful cues. 
42

Cues may be misperceived because the cue content is too complex, requiring greater 

attentional resources than the operator has available at that time.

Table 3-2. Explanations and examples for the proximate causes of the Detecting and 

Noticing function.

Cognitive 

Mechanism Example

Proximate cause: Cues/information not perceived

[A1] Cue 

Content

Cue salience is low and not detected. For example, the operator fails to detect “low” indication 

because scale of trend display makes subtle changes difficult to see.

[A2] Vigilance 

in Monitoring

Unable to maintain vigilance. For example, the operator is fatigued at the end of the night shift 

and has difficulty monitoring the plant. Alternatively, the operator must attend to several 

indications following a plant transient, thereby missing important cues.

[A3] Attention Inattentional blindness—missing something that is in plain sight. For example, the operator may 

not perceive a plant indicator because it is not the focus of the operator’s attention.

[A4] 

Expectation 

Mismatch between expected and actual cues. The operator may not notice a plant indicator 

because he or she is not looking for it. This may occur because of the operator’s mindset—if 

the operator has a particular hypothesis about the plant condition, he or she will look for 

information to confirm that hypothesis but may not equally look for information to disconfirm it.

[A5] Working 

Memory

Working memory capacity overload. If the operator attends to multiple alarms simultaneously, 

they may reach a state of information overload in which he or she is focused on a particular 

alarm panel and may miss additional alarms or indicators elsewhere in the control room.

Proximate cause: Cues/information not attended to

[B1] Cue 

Content

Too many meaningful cues. For example, during a plant transient, the operator may have 

multiple simultaneous indications, such as an alarm flood involving hundreds of annunciators, 

and may not be able to attend to all indicators simultaneously. This mechanism focuses on the 

content and amount of the information.

[B2] Vigilance 

in Monitoring

Divided attention. For example, if the operator has to respond to multiple simultaneous alarms, 

the operator may reach a state of so-called information overload or alarm flood in which he or 

she is unable to attend to additional alarms. Also, the operator may be fatigued at end of night 

shift and have difficulty monitoring plant. Alternatively, the operator may be stressed, leading to 

shortened attentional focus. In [A2], operator simply misses important information. Here, the 

operator is aware of new cues like alarms but cannot attend to or address it.

[B3] Attention Change blindness. For example, the operator may not notice a change in a plant indicator 

because the operator has missed the prior state of the indicator. In inattentional blindness [A3], 

the operator misses detecting a change in an indicator that is not his or her current focus. In 

contrast, change blindness occurs when the operator is focused on monitoring an indicator and 

simply fails to notice the change.

[B4] 

Expectation 

Overreliance on primary indicator. The operator may be monitoring a particular plant indicator 

to the exclusion of other meaningful indicators that should or need to be monitored. This often 

occurs because of the operator’s mindset—if the operator has a particular hypothesis about the 

plant condition, he or she will look for information to confirm that hypothesis and may discard 

information to disconfirm it.

[B5] Working 

Memory

Working memory capacity overflow. This mechanism focuses on the operator’s limited working 

memory capacity. For example, if the operator has to respond to multiple simultaneous alarms, 

the operator may reach a state of so-called information overload or alarm flood in which he or 

she is focused on a particular alarm panel and may disregard additional alarms or indicators 

elsewhere in the control room. The distinction with [A5] involves the extent the operator is 

aware of information that must be attended to. In [A5], the operator misses information because 

his or her attention and working memory are elsewhere. Here, the operator is aware of the 

additional alarms, but simply cannot dedicate resources to processing additional alarms.
43

Cognitive 

Mechanism Example

Proximate cause: Cues/information misperceived

[C1] Cue 

Content

Cues are too complex. For example, during a plant transient, the operator may have multiple 

simultaneous indications that have multiple facets or dimensions, and may not be able to attend 

to all indicators simultaneously. The operator may arrive at the wrong conclusion about the 

plant state because he or she is unable to interpret the entire cue content.

[C2] Attention Inattentional blindness—The technical term for the phenomenon of missing something that is in 

plain sight. For example, the operator may misperceive obvious plant information because that 

information is not the focus of the operator’s attention. In this case, the operator may fail to 

integrate important information that would otherwise be obvious (e.g., changing trends, 

because the operator is focused on understanding different information). The distinction 

between [A3] and here deals with how inattentional blindness manifests. The operator may 

simply not perceive changed but important information in [A3], whereas here they may 

misinterpret it.

[C3] 

Expectation 

Mismatch between expected and actual cues. The operator may be monitoring a particular 

plant indicator to the exclusion of other meaningful indicators that should or need to be 

monitored, leading to a misperception of a cue about plant status. This may occur because of 

the operator’s mindset—if the operator has a particular hypothesis about the plant condition, he 

or she will look for information to confirm that hypothesis and may discard or miss information 

to disconfirm it.

[C4] Working 

Memory

Memory segmenting error. Information may be parsed or processed incorrectly, leading to a 

misperception of the plant state. This can occur especially in response to high task load or 

when the operator is distracted. Based on his or her expectations for the progression of a plant 

transient, the operator may fill in missing or expected information, leading to the wrong 

perception.

3.4 Performance Influencing Factors impacting Detecting and 

Noticing

Cognitive mechanisms are influenced by contextual factors that arise from human interactions 

with the work environment, as well as the nature of the tasks being performed. Appendix A.1 

provides summaries of the relationships between the cognitive mechanisms and performance 

influencing factors (PIFs) based upon the literature review and authors’ inferences. The tables 

link PIFs and mechanisms for each proximate cause. Selection of appropriate PIFs for a given 

human failure event (HFE) has been a source of confusion in many human reliability analysis 

(HRA) methods. The cognitive mechanisms help to delineate the correct PIFs for proximate 

causes. This delineation becomes especially important in HRA quantification, where different 

PIFs may lead to different human error probabilities for the same proximate causes. Accounting 

for the cognitive mechanisms provides the scientific basis for linking the correct PIFs with the 

proximate causes.

Note that PIFs rarely offer a simple binary (yes/no) effect on triggering proximate causes. The 

effect of context in triggering degraded performance is not an all-or-nothing effect. Rather, as 

the contextual factors increase, PIFs serve to drive up the magnitude of the degraded 

performance. Where sufficient evidence exists in the psychological literature, the cognitive 

mechanisms described in Appendix A.1 provide justification to account for the effect of the PIFs 

on the proximate causes.

Much of psychological literature reports inferential statistical results when reporting the results 

used to test a hypothesis (e.g., F-values for an analysis of variance and p-values for 

significance testing). In many cases, the inferential statistical results will be presented in an 

article, but the underlying descriptive statistical results (e.g., mean, standard deviation, or effect 

size) remain unreported and are simply depicted in a figure. For the purposes of determining 
44

the effect of a reported PIF on performance, it is particularly useful to have descriptive statistics 

that might, for example, reveal that a particular PIF decreased performance by 20 percent. 

Instead, the exact effect is not reported, making it difficult to extract the precise relationship 

between the PIF and performance. In such cases, it is necessary simply to report that a 

particular PIF caused a statistically significant difference in performance. Appendix A.1 lists 

descriptive statistics when they are reported in the source articles; otherwise, the relationship 

between a PIF and performance is discussed in general terms as can be derived from the 

articles.

The psychological literature does not typically refer to its subject matter using the term used in 

the present document—PIFs. An article may, for example, report results of performance of 

subjects using different visual display technologies and display design techniques. In this 

literature review, we may use these results to provide evidence about a ‘PIF’ pertaining to the 

human system interface and its effect on human performance. The literature review is thus 

outcome oriented; that is, studies have been reviewed with the intent of extracting insights about 

the relationships between the phenomena or subject matter of interest (e.g., display design 

technology) and the outcome of interest to HRA (e.g., the potential for display design 

technologies to influence the kinds and likelihood of human errors).

The tables in Appendix A.1 provide a crosswalk between:

• Proximate causes for Detecting and Noticing

• Categories of cognitive mechanisms that explain the proximate causes 

• Example explanations of the cognitive processes involved in the mechanisms

• PIFs associated with those proximate causes. 

The tables are further summarized graphically in cognitive framework diagrams in Appendix B.1. 

The reader may use these tables and diagrams to determine plausible relations between the 

proximate causes and PIFs. The value of using the mechanisms becomes apparent in this 

context, because most proximate causes can be explained through multiple cognitive 

processes. Without consideration of the cognitive mechanisms presented here, the selection of 

the most appropriate PIFs can be extremely subjective. The mechanisms account for why a 

particular PIF is the best fit. Moreover, different PIFs may have different underlying error rates. 

In other words, the same proximate cause may have different human error probabilities (HEPs)

depending on the cognitive processes involved. Understanding the underlying cognitive 

mechanism is therefore important not only for identifying the most relevant PIFs but also for 

calculating the correct HEP.

Note that the table provides separate paths for different types of analyses. When performing an 

analysis for which the proximate causes have been identified, it is appropriate to trace from left 

to right through the table, starting with the proximate causes and selecting the most appropriate 

PIFs based on the cognitive mechanism. However, in a prospective analysis, such as when 

attempting to anticipate potential proximate causes that could happen, the analysis should be 

performed from right to left, first identifying realistic PIFs and then tracing through the cognitive 

mechanisms to determine the most likely proximate causes. Alternately, the analyst may 

consider the mechanism at play for particular operator tasks and use this as the basis for using 

the table.

3.5 Summary

This chapter summarized the psychological research literature applicable to monitoring and 

acquiring information in nuclear power plant (NPP) control rooms. This chapter introduced three 

proximate causes that contribute to the failure of the function and explained the underlying 
45

cognitive mechanisms. Finally, this chapter highlighted the various contexts in the form of PIFs 

that may trigger failures in the cognitive mechanisms. Using this chapter in conjunction with the 

supplemental information in Appendix A.1 allows the analyst to use insights from psychological 

literature to complete an accurate and scientifically grounded human reliability analysis related 

to Detecting and Noticing.

Notice that the scope of the literature review was limited to those relevant to activities in NPP 

control rooms in internal, at-power events. The assumptions made for the scope include that (1) 

operators always know what to monitor/detect, per their training and procedures, (2) the 

cue/information to be detected has no uncertainty, and (3) the information is effortlessly used for 

the next cognitive function, Understanding and Sensemaking. For scenarios beyond these 

assumptions, additional mechanisms and proximate causes may be needed to provide a full 

coverage of the explanation to the failure. 

47

4. UNDERSTANDING AND SENSEMAKING

4.1 Introduction

Understanding and Sensemaking refers to the process of associating the meaning of the 

various pieces of information with one’s mental models to generate the understanding of the 

situation in the scenario context. It involves models and processes such as sensemaking, 

situation awareness (SA), interpretation of pieces of information, and integrating multiple pieces 

of information together with knowledge into a coherent understanding. This chapter focuses on 

elucidating the cognitive processes of how people make sense of their environment. We

reviewed neurocognitive and psychological experiments and models for how humans 

understand their environment. This chapter first introduces Understanding and Sensemaking as 

one of the macrocognitive functions; the chapter then presents an overview, at a very high level, 

of the basic processes involved in making sense of a situation, followed by the description of 

several influential models of how humans achieve Understanding and Sensemaking. 

Subsequently, the chapter reviews and categorizes a large body of literature on how 

performance influencing factors (PIFs) may affect sensemaking. Finally, the authors applied the 

mechanisms reviewed in this chapter to establish links between PIFs and proximate causes to 

provide understanding of how PIFs affect likelihood of human errors associated with 

Understanding and Sensemaking. The details of the review are summarized in Appendices A 

and B. 

4.1.1 Overview of the Understanding and Sensemaking Function

The function of Understanding and Sensemaking encompasses the cognitive activities of 

understanding and making sense of the information that a person has perceived. This ranges 

from the automatic, unconscious comprehension of words, for example, to the very conscious 

and effortful thinking, integration of information and diagnosis of a situation. The same 

psychological processes are involved across the spectrum, but the level of automaticity varies—

the immediate, unconscious recognition of information that occurs in perception is automatic, 

whereas deliberate, conscious understanding of situations has much less automatic processing. 

Cognitive activity is not one or the other, conscious or automatic, but rather a blending of both, 

with varying degrees of conscious or unconscious processing depending on the situation (Klein, 

Phillips, Rall, & Peluso, 2007).

As discussed in Chapters 2 and 3, the boundary between Detecting and Noticing function and 

Understanding and Sensemaking function is blurry, as human cognition is a continuous, 

non-linear process. An analogy from the reading comprehension literature may help distinguish 

between perception, understanding, and sensemaking. Perception that occurs in the Detecting 

and Noticing function is comparable to recognizing a familiar word—one does not piece together 

the letters to know the word; the word is simply recognized as a whole (Durso, Rawson, & 

Girotto, 2007). This process is immediate and automatic. Understanding the meaning of the 

sentence is comparable to understanding the context-specific meaning of information extracted 

from the perceptual input. This is less automatic and requires attention, but still does not 

require extensive effort to process. Integrating the information from a passage of text with word

knowledge to form a richer understanding of the text is comparable to integrating information 

from a situation with knowledge to diagnose the event under way (Durso, et al., 2007). It is a 

deliberate effort to understand the environment.

To further clarify the difference between Detecting and Noticing and Understanding and 

Sensemaking, it is possible to describe Detecting and Noticing as involving assigning basic 

meaning to a stimulus that is not context dependent, but Understanding and Sensemaking
48

involves comprehending what the information means in the context of the present situation. For 

example, identifying an alarm indicator as lit occurs in Detecting and Noticing, but assigning the 

context-specific meaning of failure of a piece of equipment occurs in Sensemaking. Another 

way to differentiate between Detecting and Noticing and Sensemaking is to consider whether 

the application of meaning to stimuli is conscious. In perception, a person imbues stimuli with 

meaning in a largely automatic, unconscious manner. However, in Understanding and

Sensemaking, applying a meaning to a red light in the context of the situation is conscious and

effortful, and people are aware of their thinking process.

To map this analogy to the macrocognitive functions, perception (as in recognizing a word, or as 

in recognizing an alarm) falls into the Detecting and Noticing function. When a person applies a 

context-specific meaning (such as understanding a sentence, or understanding that an alarm 

means a particular piece of equipment is malfunctioning) and performs deeper integration with 

knowledge (such as diagnosing an event), this falls into Understanding and Sensemaking. To 

differentiate between these ends of the spectrum, in this report, we refer to comprehending the 

context-specific meaning of perceived information as Understanding, whereas Sensemaking is 

referred to as the process of integrating the understanding of the pieces of information into a 

cohesive story or big picture to make sense of the situation based on knowledge and 

experience. 

4.1.2 Understanding and Sensemaking with Procedures in the NPP Domain

Sensemaking serves a number purposes or activities beyond simply comprehending stimuli.

Klein et al. (2007) list eight purposes of sensemaking:

• Detecting problems (e.g., to determine if a pattern is worth worrying about and monitoring 

more closely)

• Using clues to learn more about the situation

• Forming explanations

• Using anticipatory thinking to prevent problems

• Projecting future states (e.g., to prepare for them, or, as in Naturalistic Decisionmaking, 

visualizing the outcome of a potential decision)

• Finding the levers (e.g., to figure out how to think and act in a situation, or to figure out what 

strategy to employ)

• Seeing relationships (e.g., as when we use a map to understand where we are going)

• Identifying problems (e.g., describing problems in a manner that leads to finding a solution)

All of these purposes are involved in operators’ situational assessment in nuclear power plant 

(NPP) operations. In NPPs, operations are dictated by procedures. Control room operators 

have detailed procedures for normal, abnormal, and emergency operations. Field operators 

have procedures for the testing and operation of equipment on the plant floor. Technicians and 

maintenance personnel have procedures or step-by-step work orders for their maintenance 

tasks, detailing the work that is to be done. There are also procedures that instruct operators on 

how to use the operating procedures.

Control room procedures, particularly abnormal and emergency procedures, are designed to be 

diagnostic and symptom oriented. They are designed to aid operators in diagnosing events, 

help them understand the situation, and assist them in determining the appropriate course of 

action to respond to the situation. Utilities design their procedures to reduce the cognitive 

burden on operators and prevent them from being forced to rely primarily or solely on their 
49

knowledge and experience to respond to a plant upset. This is important, as operating 

experience and research has shown that errors occur more frequently when people are 

engaged in knowledge-based activity (i.e., they are relying solely on their knowledge and 

experience, without any rules to guide them) than when people are engaged in rule-based 

behavior (i.e., people are making decisions with the aid of rules or procedures) (Reason, 1990).

Nuclear power plant operators are qualified and licensed to operate the reactor and associated 

systems in the plant, yet regulations and plant policies tightly constrain operators to follow 

emergency procedures in responding to an event. Given this, one may ask why Understanding 

and Sensemaking is still a concern for human reliability analysis (HRA) and this research effort. 

If the procedures do all the thinking for the operators, then why should we be interested in how 

operators think?

Two important points provide an answer to this question. First, research with NPP operators 

has shown that even when they are following procedures in response to an event, operators are 

still engaging in diagnostic or sensemaking activity. Operators “think ahead of the procedures.” 

Operators use the procedures both as a check to ensure that they meet all critical safety 

functions and regulatory requirements (Roth, 1997), and as a guide to help them develop their 

understanding of the event. They anticipate how the event is going to unfold based on their 

current knowledge, they develop dynamic situation assessments that provide explanations for 

the symptoms they are observing and identify expectations for additional symptoms they should 

see if their explanation is correct. If their expectations are violated, they seek additional or 

alternative factors that can provide an explanation for the plant behavior (Roth, 1997; Vicente,

2007; Vicente, Mumaw, & Roth, 2004; Vicente, Roth, & Mumaw, 2001). This all happens in 

addition to and parallel to following procedures. Second, there may be instances where the 

procedures do not match the situation well, and operators will have to engage in sensemaking 

and diagnosis without the help of procedures.

Operating experience has demonstrated this to be the case. There have been a number of 

incidents at various NPPs that challenged operators because their procedures did not work for 

the situation. For example, in an event at the Crystal River 3 NPP (Meyer, 1992), a pressurizer 

spray valve stuck open during a power increase following a shutdown, causing a slow 

depressurization. The indication for this valve in the control room showed incorrectly that it was 

closed. Operators perceived plant behavior as inexplicable, as all they could see was evidence 

of a depressurization with no indications of problems with the pressurizer or any other reason to 

explain it. This was primarily a problem with the indication, but the fact that their operating 

procedures did not help them determine the cause of the depressurization or prompt them to 

verify that all relevant spray valves were closed compounded the difficulty of the event. Without 

procedures to guide them, operators did not understand the nature of the situation and made 

several inappropriate knowledge-based decisions, including increasing power in an attempt to 

increase pressure and later, after the reactor had automatically tripped, temporarily bypassing 

engineered safety features. Eventually, a senior shift superintendent made a knowledge-based 

decision to close the pressurizer spray line isolation valve based on his understanding of the 

possible causes of reactor coolant system (RCS) pressure leaks, which terminated the leak. As 

this example shows, procedures do not always provide appropriate guidance for the situation, 

and operators consequently have to use their knowledge and experience to respond to the 

event.

Sensemaking does not stop merely because operators are working from procedures. 

Therefore, for the Understanding and Sensemaking function, procedures can be viewed as a 

source of information, or data, and as an influence on the selection or construction of a mental 

model. Procedures can greatly reduce the cognitive burden on operators to figure things out 

when they are well suited to the situation at hand. Procedures may also act as cues or triggers 
50

that allow operators faster and easier access to the knowledge they already have that is most 

pertinent to the situation. Sensemaking activities continue regardless.

Given the discussion above, we made the following assumptions about Understanding and 

Sensemaking in NPP control rooms for internal, at-power events: (1) Operators have mental 

models of the scenarios, (2) operators have procedures to guide and verify their understanding 

of the scenarios; and (3) operators’ experience and procedures are adequate for them to make 

a full assessment of plant situations. These assumptions set forth the scope of the literature 

review for the function. 

A High-Level Overview of the Cognitive Mechanisms of Understanding and Sensemaking

When reviewing the various models for how humans understand their environment, it is possible 

to generalize at a very high level the basic processes involved in making sense of a situation 

(see Figure 4-1). The human sensory systems perceive information from the external world, 

subject to the various influences that can affect perception as discussed in Chapter 3. The 

process of perception applies initial, non-context specific meaning, and this new information 

(i.e., “data” or “percept/perception”) is integrated with prior information stored in long-term 

memory to interpret and make sense of the information; the integration occurs in working 

memory. These basic processes are dynamic and iterative. At each of these stages, certain 

PIFs can affect the underlying cognitive processes. 

Figure 4-1. High-level overview of sensemaking.
51

The value of such a high-level overview is two-fold. First, it indicates the process required for 

sensemaking. Sensemaking requires that:

• the information stored in working memory be processed for understanding; 

• an appropriate, effective frame is retrieved; 

• the information and frame is correctly and reliably integrated; and

• the integration process is iterated and dynamically updated until the situation makes sense. 

Second, the overview indicates points within the sensemaking process where failures can 

occur. 

• The information coming into the sensemaking process may be inaccurate due to:

o problems with the quality, accuracy, or availability of the information itself, external to the 

human perceptual process;

o failures or errors in perception; or

o decision errors about to which information to attend.

• There may be a problem with the existing knowledge that the new information is being 

integrated with, as when a person uses an incorrect mental model with which to interpret the 

information. 

• The integration of new information with prior information itself can produce incorrect 

understanding. Failures can also occur if the integration process is not adequately iterated 

to achieve the correct understanding, or the process is not updated with dynamic evolving 

information.

Moving deeper from this high-level overview, research in macrocognition has developed several 

models of sensemaking that describe the dynamic processes in greater detail. These models 

are founded on common cognitive mechanisms yet they often focus on different aspects of 

Understanding and Sensemaking, therefore they are typically complementary to each other 

although they may appear different. To make it easier to understand the commonalities and 

differences among models, we first summarize the basic concepts and terminologies that are 

frequently used in the models. 

• Data/information. What one perceives from the external world. Yet, given that 

Understanding and Sensemaking iterate with perception, the data and information entering 

the understanding process may have already represented the integration of both the 

perceived external world and the individual’s initial understanding of what were perceived. 

• Long-term memory (LTM). Knowledge and experience stored in the brain that can be 

retrieved for use.

• Working memory (WM). A process or a mechanism that maintains pieces of information 

online for immediate use and binds pieces of information together.

• Mental model. This term has been used in the literature with different meanings.

Sometimes a mental model is referred to as a static representation of something stored and 

pulled out of long-term memory (e.g., a mental model of how a plant works). The term is also 

used to mean something that is ‘constructed’ dynamically to explain the current situation 

(e.g., there are currently two malfunctions in the plant – one explains why the steam 

generator (SG) level is increasing and one explains why I have radiation in an auxiliary

building). In this report, we refer to a mental model as the static representation.
52

• Frame/framing. The concept of “frame” was introduced by Klein, et al. (2007). A frame 

encompasses the concepts of a mental representation, a mental model, a story, a map, a 

schema, a script, or a plan, and serves as a structure for explaining the data and guiding the 

search for more data. Yet, the concept also has two meanings in the literature. Some use 

the term “frame” as something static that is pulled out of long-term memory and 

‘instantiated’. Others used the term “frame” to refer something more active that is 

‘constructed’ to account for the existing evidence—to make sense of the evidence. In this 

report, we use the term “frame” for the dynamically constructed representation of a person’s 

current understanding of a situation. However, the initial frame in the sensemaking process 

could be the same as the static, instantiated representation of data stored in the long-term 

memory. Subsequently, the term “framing” is used to refer the process of constructing a 

frame. Klein et al. (2007) refer to sensemaking as a process of framing and reframing.

• Situation awareness. Endsley developed the situation awareness theory to explain human 

cognition, where situation awareness is the mental representation derived through a 

conscious deliberative process to form an understanding of what is going on. It is also often 

a highly automatic process of situation recognition, using a schema of prototypical situations 

that is dynamic and ongoing. On the other hand, Endsley believed that sensemaking is 

characterized as primarily of the conscious deliberative type, and therefore situation 

awareness encompasses a broader understanding of what is going on than sensemaking 

does. 

• Situation model. O’Hara et al., 2008 used the term “situation model” to refer to the active 

knowledge representation that is constructed dynamically and used to explain symptoms 

and derive predictions and projections in a particular situation (as opposed to a mental 

model which was reserved for long-term memory representations). This term is similar to 

the “frame” concept proposed by Klein et al. (2007).

4.1.3 An Overview of the Dynamic Data/Frame Theory of Sensemaking

Gary Klein and his associates have put forth a model of how people in real-world settings make 

sense of their environment that they term “a data/frame theory of sensemaking” (Klein, Moon, & 

Hoffman, 2006; Klein, et al., 2007). Weick (1995) introduced the concept of sensemaking as a 

primary cognitive function that people must carry out in a variety of natural settings. 

Sensemaking allows people to question what is known, evaluate what is conjectured, 

hypothesize and diagnose, and integrate facts with theory (Klein, et al., 2007). Klein et al. have 

developed a model that is an extension of Neisser’s (1967) perceptual cycle theory, in which a 

person’s sampling of the environment updates their cognitive map of the environment, which in 

turn directs further exploration. Klein et al. assert that sensemaking is a cyclical process that

starts when a person or group of people recognize that their current understanding of things is 

inadequate. This typically happens when people are surprised or when something unexpected 

occurs. Sensemaking involves not only understanding the meaning of pieces of information, but 

more importantly, putting together pieces of information to form a complete understanding of the 

situation. Sensemaking can therefore be defined as a deliberate effort to understand events or 

situations (Klein, et al., 2007). Figure 4-2 provides a high-level overview of the data/frame 

theory of sensemaking.

The data/frame theory of sensemaking posits that information coming into the sensemaking 

process is data (depicted in Figure 4-2 as small green circles of differing shapes and sizes).

This data is integrated with an existing frame, which is the current understanding that links data 

with other elements to explain and describe the relationship of the data with other entities.

Figure 4-2 depicts frames as the blue discs into which the green bits of data fit. A frame 

encompasses the concepts of a mental representation, a mental model, a story, a map, a 
53

schema, a script, or a plan, and serves as a structure for explaining the data and guiding the 

search for more data (Klein, et al., 2007). The data identify or construct the frame, and the 

frame determines the data to which the person attends. Neither the data nor the frame comes 

first; rather, it is a constant process of moving back and forth from data to frame. This dynamic 

aspect is the central theme of the model.

Klein et al. (2007) explain further, “Sensemaking is a process of framing and reframing, of fitting 

data into a frame that helps us filter and interpret the data while testing and improving the frame 

and cyclically moving forward to further adapt the frame. The purpose of a frame is to define the 

elements of the situation, describe the significance of these elements, describe their relationship 

to each other, filter out irrelevant messages, and highlight relevant messages. Frames can 

organize relationships that are spatial (maps), causal (stories and scenarios), temporal (stories 

and scenarios), or functional (scripts)” (p. 119). Section 4.2.1 discusses this model of 

sensemaking in more detail.

Figure 4-2. The sensemaking process (Klein, et al., 2006).

4.1.4 Working Memory and the Neurological Basis of Sensemaking

Integrating pieces of information in the brain requires working memory, which maintains pieces 

of information online and associates them. Working memory processes pieces of present 

information and combines them with the knowledge retrieved from long-term memory. It is 

limited in the amount of information it can handle at once, because of this people break up 

information into manageable chunks. Working memory has been referred to as short-term 

memory in the past because of limitations to how long information can stay active. Recent

research has elaborated upon older short-term memory models. The Baddeley model of 

working memory (2000; 1974) is the model that has received support from psychological and 

neurophysiological experiments. Working memory has several subsystems or processing 

buffers that process different types of information: the visuospatial sketchpad handles visual and 

spatial information, and the phonological loop processes verbal and auditory information.

Working memory interacts with long-term memory through the episodic buffer. The central 

executive controls and directs attention and the flow of information between these systems. 

In reviewing the working memory literature over the last 20 years, there seem to be two camps 

of research when it comes to the limits of working memory. One group of research shows 

working memory with a limit of about seven to nine chunks of information, whereas other 

research shows that people have a limit of about four chunks of information that they can keep 
54

in working memory. We synthesized this research, and determined that working memory for 

actively processing information has a different limit than working memory for maintaining 

information in active consciousness. Working memory has a “maintenance buffer” that keeps 

active pieces of information that are not being immediately used. The maintenance buffer can 

hold between about seven to sixteen chunks of information. Information that is actively being 

used in more complex or effortful thinking is manipulated in a “processing buffer.” This 

processing buffer has a limit of four chunks of information, plus or minus one (Xing, 2007).

Taking a step back to see how information flows in and out of working memory allows one to 

see where working memory fits in the larger sensemaking process. 

Figure 4-3 provides a high-level illustration of the psychological processes behind sensemaking, 

based on the neurological organization of the human brain. Again, a discussion of neural 

structures is too fine a level of detail for HRA purposes, but reviewing the functions that the 

neural structures perform is instructive to inform readers of the basic processes. Information 

enters working memory from the sensory systems and/or from long-term memory through the 

episodic buffer. This information can go into either the maintenance or active processing buffer.

The processing buffer also receives input from other neural systems that are related to belief 

and intention. Information then can be transferred to long-term memory, or moved forward into 

the areas of the brain that handle decisionmaking. Decisionmaking also influences the 

information that is kept and processed in working memory.

Figure 4-3. Cognitive processes behind sensemaking (Xing et al., 2011).

4.2 Major Cognitive Models of Sensemaking

There are a number of cognitive models related to Understanding and Sensemaking. We 

discuss four of them here. The criteria for selection include: 

1. The model is detailed enough to explain how Understanding and Sensemaking works and 

fails.

2. The model focuses on sensemaking of the environment or situation, not on detailed 

step-by-step cognitive activities such as the ACT-R model (Adaptive Control of Thought—

Rational; Anderson, 1993).

3. The model has gained recognition in the cognitive psychology community and has been 

applied to various applications.

Each of the four models discussed in this chapter explains the processes of Understanding and 

Sensemaking. However, they are each different in their perspective, focus, and intended 
55

applications. For example, Klein’s model focuses on the dynamic processes of sensemaking,

while Endsley’s situation awareness model was intended to make the products of the internal 

processes of sensemaking observable and measurable. Together, these models present a 

more comprehensive description of sensemaking mechanisms; each model offers some unique 

perspective on human errors associated with sensemaking. Presented along with the 

description of each model is a discussion of its strengths and limitations.

The four models describe the psychological aspects of Understanding and Sensemaking

without addressing the underlying neurocognitive basis for the psychological processes. One

critical mechanism for Understanding and Sensemaking is working memory. The sensemaking 

processes described in the four models require working memory to bind or integrate related 

information together to form the elements of mental representation of the situation and to retain 

the presentation. Associated with working memory is attention. Attention is needed for 

execution of information binding and selection. Both working memory and attention have 

limitations in their resources, capacities, and timing. These limitations are root causes to many 

human errors. Therefore, following the description of the four psychological models, we also 

generalize the functionalities of working memory and attention in Sensemaking as well as their 

limitations.

4.2.1 Detailed Discussion of the Data/Frame Theory of Sensemaking

As overviewed previously in Section 4.1.3, the data/frame theory of sensemaking describes the 

process of how people dynamically compare and integrate incoming data with a frame to 

understand the situation. Frames are the key structural component to the data/frame theory of 

sensemaking. A frame is a mental representation, or the current understanding that links data 

with other elements to explain and describe the relationship of the data with other entities.

Other research discusses concepts such as mental representations, situation assessment, 

mental pictures, mental models, mental maps, stories, schemas, scripts, if-then rules, 

expectations; the concept of frame encompasses all of these and any other related constructs 

(Klein, et al., 2007). A frame serves as a structure that explains the data in the environment and 

guides the search for more data (Klein, et al., 2007). Specifically, the person interprets all 

perceived information (data) in light of his or her frame, and the frame guides the person to seek 

out additional data and shapes how the person determines what information is relevant.

For example, an NPP operator has a mental model about the design of the plant, including the 

layout of the control and indication panels, relationships between the various systems and how 

they interact, the configuration of the various systems, and the nominal behavior of the plant 

when it is at power and when it is starting up or shutting down. This mental model of the system 

is a frame. When the operator starts a shift, she or he participates in a shift briefing in which 

she or he is informed about any ongoing work, any components or systems that are out for 

maintenance, etc. The operator then updates his or her frame of the system accordingly. 

Operators also have frames for the types of events that they are trained to handle. NPP 

operators must be able to pass training on a number of design-basis events, such as steam 

generator tube ruptures (SGTR), or a loss of coolant accident (LOCA). These events have 

certain associated symptoms, such as increasing steam generator level and secondary system 

radiation alarms for an SGTR event. Through training, operators create mental models (i.e., 

frames) for these events that pair symptoms with explanations. During an actual event at a 

plant, operators will compare the symptoms they are observing in the plant to the frames stored 

in their memory to help them diagnose the event. Operators may also integrate multiple frames, 

such as considering both the frame for the event in progress with the frame for the current 

system configuration.
56

The purposes of sensemaking (problem detection, forming explanations, projecting future 

states, finding relationships, etc.) imply complexity and a variety of human thinking as one 

strives to understand the environment. People are active interpreters and constructors of 

information, and people actively judge what data is relevant and deserves attention.

Sensemaking involves more than just deriving inferences from perceived cues (Klein, et al., 

2007). Sensemaking involves adductive reasoning, more so than deductive logic (Klein, et al., 

2007; Lundberg, 2000). This means that people search for explanations that best account for 

the symptoms or data they are seeing, rather than looking for data to confirm the conclusion 

they already have, as they would with deductive logic. Once people have an understanding, 

they will often attempt to confirm it with the data, but until they have a diagnosis, they are 

searching for explanations for the data they have (Klein, et al., 2007; Lundberg, 2000).

Klein et al. present seven basic types of sensemaking that they have been able to distinguish in 

research to date, or seven types of activities that can occur in the sensemaking process, as 

depicted in Figure 4-4. These are not “stages” that occur in a linear fashion, but rather are 

types of sensemaking that can occur in the process, depending on the data and frame. It is 

important to note that there is no start or end in Figure 4-4; the process can start in any of the 

ovals in the diagram, and sensemaking can proceed through some or all of these activities in 

one situation, often more than once.

As in Figure 4-2, Figure 4-4 depicts data as the small, variously shaped green circles, and 

frames as the larger blue circles into which the data pieces fit. In each of the types of 

sensemaking activities, Klein et al. (2007) lists the ways that the frame drives data collection on 

the left of the circle, and lists the ways that the data affects the frame on the right of circle. 

Thus, within each type of sensemaking activity, the data and frame define and update one 

another, as depicted by the reciprocal arrows in the top oval (although not displayed, Klein et al. 

assert that the same reciprocal relationship between data and frame exists in each of the 

sub-types of sensemaking shown in the smaller ovals). 

The basic data/frame match is an attempt to connect incoming data with a frame. In Figure 4-4, 

this is the large oval at the top of the diagram, in which the data fit neatly in the frame. The 

frame a person uses depends on the available information, the person’s goals, available gallery 

of frames, and other factors such as workload, fatigue, and motivation (Klein, et al., 2007). 

Klein et al. posit that the initial matching of data with frame may be automatic and 

pre-conscious, but overall the sensemaking process is volitional and conscious. Pattern 

matching can be unconscious or preconscious, whereas comparing different frames tends to be 

conscious and deliberate (Klein, et al., 2007).

As people continue to operate in the environment, they will elaborate and extend the frame they 

are using with the new information they learn about the situation. They will not seek to replace 

the frame if no surprises or unexpected circumstances arise (Klein, et al., 2007). In this type of 

sensemaking, neither the data nor the frame is rejected. As the person learns more information, 

his or her understanding gradually deepens and broadens. Figure 4-4 illustrates this with 

smaller, more finely detailed bits of data in the frame that represent the more nuanced 

understanding.
57

Figure 4-4. A more detailed view of the sensemaking process (Klein, et al., 2007).

However, when something unexpected happens or people are surprised, they begin the process 

of questioning the frame. Here, people consider data that are inconsistent with the frame in use

(shown in Figure 4-4, “questioning the frame”, with a piece of data that does not fit in the frame).

It may be the case that the frame is incorrect, the situation has changed, or the inconsistent 

data are inaccurate, but at this point all that is known is that there is a mismatch. The violation 

of the expectations provided by the frame leads people to question the accuracy of their current 

understanding or situation assessment (i.e., the frame) (Klein, et al., 2007).

Sometimes, when questioning the frame, people preserve the frame by discounting or 

dismissing the data that do not match the frame (depicted in Figure 4-4, “preserving the frame”,

by the white line that cuts off the part of the data that does not fit in the frame as a metaphor for 

ignoring that part of the data). If the data are indeed inaccurate or unreliable, this is appropriate. 

This is also a potential mechanism for causing error, particularly if the data are indications that 

the current explanation (i.e., frame) is faulty. Dismissing, ignoring, or explaining away data can 

lead to problems such as routinizing deviance, accepting inappropriate amounts of risk, or 

entrenching “knowledge shields” that people use to defend frames in the face of contradictory 

evidence. These problems can cause fixation errors, wherein people maintain their current 

frame despite evidence that indicates the frame is incorrect (Klein, et al., 2007). This is a very 

important class of errors in the control room as showed by human performance simulations of 

control room operation. 

Sometimes it is necessary to deliberately compare multiple frames to figure out what is 

happening in the situation. This may be the case when the data have more than one plausible 
58

explanation. Figure 4-4 depicts “comparing the frames” with two similar frames, with the only 

difference as the size of the slot for one of the pieces of data. Experts can sometimes evaluate 

up to three alternative frames simultaneously, as when doctors consider multiple diagnoses for 

a set of presenting symptoms, and deliberately select frames that allow them to sharpen the 

distinctions between the diagnoses that are most relevant. They will then search for data that 

allows them to determine which frame is correct. This approach can be referred to as using a 

“logical competitor set,” in that the person will develop one or two logical competitors to the 

current frame and use the competitors as a means for identifying critical data that will allow the 

correct frame to be selected (Klein, et al., 2007).

Reframing occurs when people replace the operating frame with one that better explains the 

data (shown in Figure 4-4, “re-framing”, with a frame in which the data now fit). This new frame 

then guides the search for data and cues. During reframing, people consider data elements 

they may have previously discarded so that formerly discrepant information now fits the frame. 

In this way, reframing sensemaking involves hindsight understanding (Klein, et al., 2007; Weick, 

1995).

If people encounter data that make no sense whatsoever, or when they identify a frame as 

inadequate, people may seek a frame. Figure 4-4 illustrates “seeking a frame” as pieces of data 

without a frame. People may be able to reframe, as mentioned previously. If a suitable frame is 

not immediately available, people will have to search for a frame that works, or construct a new 

frame. This may involve looking for analogies, searching for more data to find anchors that can 

be used to construct a new frame, and assembling data elements as they build an explanation 

(Klein, et al., 2007).

All seven of these types of sensemaking activities can occur as people attempt to understand 

what is going on, and as people compare data with frames, they may move back and forth 

through the types of sensemaking. Errors made in any one point in the sensemaking process 

(such as inappropriately preserving a frame) can be recovered if the discrepancy between data 

and current explanation becomes so large that people have to acknowledge an incorrect frame 

and search for an accurate frame to replace it with.

In general, Klein’s sensemaking model integrates and incorporates decades of research on 

human cognition. It provides explanation for how humans understand their world, and as such, 

it can serve as an important part of the cognitive psychological basis for HRA.

4.2.2 The Product of Sensemaking: Situation Awareness

Klein’s sensemaking model provides a theoretical description of the sensemaking processes. 

On the other hand, Endsley’s Situation Awareness (SA) model focuses on making those inner 

processes of sensemaking observable and measurable. The central theme of Endsley’s SA 

model is to classify the momentary product of sensemaking, referred to as situation awareness, 

into three stages or levels: perception, comprehension, and projection. Associated with this 

model, Endsley also developed experimental techniques to measure one’s SA during task 

performance. Because of the availability of this technique, SA has been the subject of 

numerous studies in applied, real-world situations for years. Therefore, SA research is highly 

relevant to identifying the cognitive mechanisms for error and the contributing factors that can 

lead to error.

Situation awareness is a term that has become popular in real-world settings and refers broadly 

to people “knowing what is going on.” It is used to indicate people’s dynamic understanding of 

events and situations, and the construct has been applied in many domains, including aviation, 

air traffic control, power plant operations, military operations, and more. One benefit of SA

research is that it allows measurement of an internal process and provides insight into how 
59

those internal processes interact with the environment. Additionally, the SA literature provides 

information about the factors that influence performance.

There are three models of SA that are instructive for examining how sensemaking plays out in 

real-world settings: Endsley’s model of SA, the functional model of orienting activity based on 

activity theory, and the perceptual cycle theory of SA. Each model is discussed below to 

identify the key insights into performance that the model provides.

4.2.2.1 Endsley’s Model of Situation Awareness

Endsley’s model of SA (Endsley, 1995) has received by far the most attention and research.

Shown in Figure 4-5, Endsley’s model is an information-processing model that describes SA as 

a product of mental processing (termed situation assessment) that consists of three levels. The

first stage, Level 1, consists of perception of the status, attributes, dynamics, and other relevant 

aspects of objects and information in the task environment, which Endsley refers to collectively 

as “elements” (Endsley, 1995; Salmon et al., 2008). The information is merely perceived at this 

point; higher-level processing or comprehension has not yet occurred. Level 1 of Endsley’s SA 

model corresponds to the Detecting and Noticing macrocognitive function used in this report; 

many of the mechanisms identified for Detecting and Noticing in Appendix A.1 are based on or 

drawn from Level 1 of Endsley’s model.

Figure 4-5. Endsley’s model of situation awareness.

Level 2 and Level 3 SA correspond to the Understanding and Sensemaking macrocognitive 

function. Level 2 SA involves the comprehension of Level 1 information to understand the 

relation of the data to the situation (Endsley, 1995; Salmon, et al., 2008). This involves 

combining, interpreting, storing, integrating multiple pieces of information, and determining 

relevance of the data to the situation (Endsley, 2000). In Level 2 SA, the operator forms a 
60

holistic mental model or picture of what is going on, and attempts to understand objects and 

aspects of the situation in light of the larger mental picture (Endsley, 1995).

Level 3 SA involves projecting the current understanding of the situation into the future to 

mentally forecast the future state given currently available information (Endsley, 1995, 2000; 

Salmon, et al., 2008). This ability to project and anticipate is crucial to subsequent 

Decisionmaking. Level 3 SA is indicative of experience and marks operators who have the 

highest level of understanding of the situation (Endsley, 2000).

Each of these levels is a product: Level 1 SA is a product of perceptual processes, and Level 2 

and Level 3 SA are products of mapping data from Level 1 SA to mental models that facilitate 

comprehension of current status and projection of future state (Endsley, 1995; Salmon, et al., 

2008). Mental models are formed based on knowledge, training, and experience, and are used 

to direct attention to critical elements in the environment as well as integrating elements 

together into a coherent picture (Endsley, 1995; Salmon, et al., 2008).

Endsley refers to this process as SA, but it bears remarkable similarity to Klein’s (Klein, et al., 

2006) sensemaking model. It may be plausible that SA as Endsley describes it is the product of 

the sensemaking process as Klein et al. describe it.

The application of the SA Model in HRA

Endsley’s model is generic and applicable to multiple domains, and the simplicity of the three 

levels lends itself to easy measurement and extension to training guidelines and system design 

(Salmon, et al., 2008). The model has also been extended to team SA (Prince & Salas, 2000).

One of the main strengths of Endsley’s model is its immense popularity. It has been applied 

widely in a large number of domains, including but not limited to aircraft pilots, air traffic control, 

military operations, NASA missions, power plant operations, rail system operations, equipment 

maintenance, and medicine. This has produced a large body of research, vastly more than 

research on Klein et al.’s sensemaking model, much of which has elucidated factors that can 

affect SA, from system design, training, expertise, workload, complexity, stress, automaticity, 

goals, and individual differences, among others. This is very valuable for the present effort to 

provide a psychological basis for HRA, as these factors map well to the performance influencing 

factors examined in HRA and provide information regarding the nature of the effect of these

factors on SA.

There are similarities and differences between the SA theory and Klein’s framing theory for 

sensemaking. The framing theory focused on how people work to make sense of the 

information and situations they find themselves in, largely at the organizational level with

respect to explaining organizational accidents or unusual events. In Endsley’s view, 

sensemaking is basically “the process of forming level 2 SA from level 1 data through effortful 

processes of gathering and synthesizing information, using story building and mental models to 

find some representation that accounts for and explains the disparate data.” (Endsley, 2004, 

p. 324). Framing theory focuses on forming reasons for past events and diagnosing the 

causative factors for observed faults; while situation awareness incorporates such assessments 

as a part of comprehension, it also focuses on understanding how such factors influence other 

aspects of the situation and projections of the future. Together, both theories provide a 

comprehensive picture of Understanding and Sensemaking and therefore serve as the cognitive 

basis for the function. 

4.2.2.2 Perceptual Cycle Theory of Situation Awareness

Smith and Hancock (1995) take a holistic view of SA, defining it as “adaptive, externally directed 

consciousness” that is a “generative process of knowledge creation and informed action taking.” 
61

Consciousness is the part of an operator’s knowledge-building behavior that the operator 

intentionally manipulates. Situation awareness generates behavior that is directed toward 

achieving a goal in a specific task environment (1995).

Smith and Hancock’s perspective on SA is based on Neisser’s (1967) perceptual cycle model. 

As such, this model has the closest resemblance to Klein’s sensemaking model, which is also 

based on Neisser’s perceptual cycle theory. The perceptual cycle theory posits that an agent 

and the environment continually interact to produce performance: the environment informs the 

agent, modifying knowledge, which directs the agent’s activity in the environment. This in turn 

samples and potentially alters the environment, which informs and updates the agent’s 

knowledge, and the cycle continues (Neisser, 1967; Smith & Hancock, 1995).

Smith and Hancock argue that the process of obtaining and maintaining SA centers on mental 

models that contain information about certain situations. These internal mental models facilitate 

the anticipation of events, direct an individual’s attention to cues, and direct their course of 

action to ensure that the situation continues to conform to expectations as it evolves. 

Unexpected events prompt additional searches for more information, which then updates the 

operator’s mental model (Salmon, et al., 2008; Smith & Hancock, 1995). Figure 4-6 shows the 

perceptual cycle model of SA.

The perceptual cycle approach to SA is expressly not merely a snapshot of the person’s mental 

model. Rather, SA guides the process of updating knowledge (Smith & Hancock, 1995). In air 

traffic control, for example, controllers build mental pictures of the aircraft in their airspace. SA 

is not the picture, but instead the controller’s SA builds the picture (Salmon, et al., 2008; Smith 

& Hancock, 1995). In this perspective, SA is both process and product. The model provides 

explanation for how the product—operator’s knowledge—is produced by the perpetual cycle of 

knowledge directing action, sampling the environment, and updating knowledge (Salmon, et al., 

2008; Smith & Hancock, 1995).

Adams, Tenney, and Pew (1995) elaborated on the concept of SA as both process and product, 

“As product, it is the state of the active schema—the conceptual frame or context that governs 

the selection and interpretation of events. As process, it is the state of the perceptual cycle at 

any given moment. As process and product, it is the cyclical resetting of each by the other” (p. 

89).
62

Figure 4-6. The Perceptual Cycle model of situation awareness (Salmon, et al., 2008; 

Smith & Hancock, 1995).

Strengths and Limitations of the Perceptual Cycle Model of Situation Awareness

A key strength of the perceptual cycle theory of SA is that it is a complete model, providing 

explanation for both the process and the product of the process. It is based on sound 

psychological theory (Neisser, 1967), and it showcases the dynamic, non-linear nature of SA 

(Salmon, et al., 2008). It explains how SA is dynamically obtained, maintained, and updated, 

and it explains anticipation, or as Endsley describes it, Level 3 SA (Adams, et al., 1995; Salmon, 

et al., 2008).

Despite the strength of this model, it has two major drawbacks. First, Endsley’s model has 

essentially eclipsed the perceptual cycle model—there has been very little research conducted 

with this model, and as a result, there is nearly no information available to expand upon the 

model or identify factors that affect it. This may be due in part to the second major drawback: 

the nature of this model, its focus on dynamic process and product, makes measuring the 

construct as defined by the model very difficult (Salmon, et al., 2008). Without a way to 

measure SA based on the model, conducting research is not feasible.

The key insight from this model for the present purpose of supporting HRA, then, is that the 

internal mental models are the most important factor for success or failure of SA. The mental 

model establishes expectations and directs actions in the environment, including which 

information to sample. This information then integrates with or updates the mental model. If the 

mental model is not complete or is inaccurate, the person could be looking for information that is 

objectively unimportant. The model does not describe a way for a person to correct the mental 

model; in this aspect, Klein’s Sensemaking model is more complete.
63

4.2.2.3 The Functional Model of Orienting Activity (Activity Theory)

The Functional Model of Orienting Activity is a model related to the concept of SA based on 

Activity Theory (AT), a psychological paradigm developed for the study of work behavior (e.g., 

Leont'ev, 1977; Platanov, 1982; Vygotsky, 1960, 1978). It assumes a unique human 

psychology defined by goal-directed behavior (Bedny, Seglin, & Meister, 2000). Activity Theory 

is an entire psychological paradigm, and as such is too broad to discuss in entirety in this review 

of sensemaking-related literature. For the purposes of this project, it is more appropriate to 

provide an overview of the relevant assumptions underpinning the theory and discuss in more 

detail the application of AT to the concept of SA.

As stated previously, AT assumes that human cognition is goal-directed. Workers go about 

achieving their goals through a process of psychological reflection—they build an internal model 

of the situation that is an image, a reflection of reality as they have understood it (Bedny, 

Karwowski, & Jeng, 2004; Bedny & Meister, 1999; Bedny, et al., 2000). Psychological reflection 

is a dynamic activity—reflection allows the person to capture changes in the situation and look 

at it from different perspectives (Bedny, et al., 2004). This allows the mental representation built 

from the reflection to incorporate multi-faceted sets of features, the nature of the relationships 

between aspects of the situation, and changes in these over time (Bedny, et al., 2004).

Goals are held as mental images of the ideal or desired state, called an image-goal; the 

projected outcome of activity is also represented internally as an image (Bedny & Meister, 

1999). AT allows for unconscious motives, but states clearly that goals are always conscious 

(Bedny, et al., 2000). The basic structural components of activity are the image-goal, motives 

that direct individuals toward the goal, and a system of actions that allow the person to achieve 

the goal (Bedny & Meister, 1999). Individuals may subjectively form internal goals for the 

situation, and objectively given goals are still subject to interpretation and acceptance or 

rejection by the person. Differences between the desired goal and current state are evaluated 

based on personal significance, which influences how motivated a person is to achieve the goal 

(Bedny & Meister, 1999).

According to AT, activity consists of three components or stages: orientational, executive, and 

evaluative. In the orientational stage, people develop a subjective mental model of reality and a 

dynamic picture of the world that provides a coherent explanation of reality (Bedny & Meister, 

1999). This stage can include both mental and external exploration of the world. The 

orientational stage of AT appears to be a composite of the macrocognitive functions of 

Detecting and Noticing and Understanding and Sensemaking.

The executive stage of AT includes decisionmaking and performance of actions (Bedny & 

Meister, 1999), which maps to the macrocognitive functions of Decisionmaking and Action. In 

the evaluative stage, the person assesses the result of the action, which leads to corrective 

actions. In the macrocognitive model used in the present project, this maps back to Detecting 

and Noticing and Understanding and Sensemaking. The remainder of this section will be 

devoted to the orientational stage, which is a basic component of operator performance, as it 

involves situation assessment and comprehension, understanding of the current situation, and 

anticipation of future status (Bedny & Meister, 1999). In this kind of activity, the concept of SA is 

critical, and the orientational stage has the most relevance for sensemaking. Figure 4-7 shows 

how individuals accomplish the task of comprehending the meaning of the situation (called 

gnostic activity) through dynamic reflection of the situation (Bedny & Meister, 1999).
64

Figure 4-7. The Functional Model of Orienting Activity (Bedny & Karwowski, 2004; 

Bedny, et al., 2004; Bedny & Meister, 1999).

As shown in Figure 4-7, multiple factors affect the meaning of perceived information (Block 1).

To properly interpret the information, the operator must have professional experience (Block 7) 

and knowledge (Block 8). The goal of the activity, in the form of an image (Block 2) and the 

components of motivation associated with it (Block 4) also affect the interpretation of the 

information. There are two aspects of motivation: sense and motivation. Sense refers to the 

subjective significance of goal attainment, and motivation determines the direction and amount 

of energy devoted to achieving the goal. Sense and motivation affect the methods of 

interpretation through the orienting and explorative actions (Block 5) to obtain further 

information, which feeds back into the image-goal of the desired state (Bedny & Meister, 1999).

Block 5 is important in that it shows how the sensemaking process dynamically modifies 

interaction with the world and how interaction with the world dynamically modifies the 

understanding of the event (Salmon, et al., 2008).

The other important functional mechanism for dynamic reflection and understanding of the 

situation is Block 3, subjectively relevant task conditions. This function block includes 

conceptual and image components of critical features of the environment, based on what is 

significant to the operator (Salmon, et al., 2008), and provides a dynamic representation of 

reality that is tightly connected with the concept of SA. Mentally manipulating features of the 

object or situation and comparing them with the goal accomplish this function. This function 

interacts with the goal through motivation (Block 4) and orienting actions (Block 5), and can 

correct the goal, if necessary (Bedny & Meister, 1999).

The manipulation of the operative image can be to a large extent unconscious. But SA, as 

described in Endsley’s model, is part of this function and is very conscious. The operator is also 

conscious of the aspects of the operative image that overlap with SA (Bedny & Meister, 1999). 
65

Therefore, the “subjectively relevant task information is involved in the dynamic reflection of the 

situation, and the constant transformation of information on conscious and unconscious levels 

according to the goals that arise before an operator” (Bedny & Meister, 1999).

Together, the conceptual model (knowledge), image-goal, and subjectively relevant task 

conditions form the “mental model of reality” that allows operators to generate descriptions of 

system purpose and design, explain system functioning and observed system state, and 

prediction of future system status (Bedny & Meister, 1999).

Strengths and Limitations of the Activity Theory Model

Some of the function blocks in the model bear a strong resemblance to performance influencing 

factors. Operator knowledge and experience have a long history of treatment as performance 

influencing factors (PIFs) in various HRA methods. The Functional Model of Orienting Activity 

also explains the importance of personal PIFs such as individual motivation and personal goals, 

which influence the energy people devote to the task of understanding information and selecting 

relevant information that is deemed to be critical to the situation for further integration into the 

mental model and image-goal. A number of HRA methods acknowledge the importance of 

personal PIFs, but few actually include them in their human performance models. In addition, 

like Klein’s model, the AT model also recognizes cognition as a series of process cycles rather 

than a linear process. 

Activity Theory does not speak much to the types of errors that can occur within this process, 

but it does mention that faulty orientation is possible. What is subjectively significant to the 

individual may not be objectively important to the situation, and if the person focuses on 

objectively unimportant elements in the dynamic reflection of the event, this can lead to faulty 

orientation in the situation and distortion of or an inaccurate internal model of reality (Bedny & 

Meister, 1999). The authors also state that knowledge and experience are necessary to 

properly interpret information in the situation, which would indicate that if the person does not 

have the necessary experience, this can lead to problems in formulating the image-goal and 

interpreting the information correctly.

While AT does provide a more dynamic view of human cognition than Endsley’s three-stage SA

model, it is not without a number of limitations. Activity Theory has not been entirely embraced 

by Western psychologists, and there is consequently a lack of empirical research to support the 

model (Salmon, et al., 2008). The model has received much less attention than Endsley’s 

model, and the emphasis on process and product of SA makes the model difficult to use to 

measure SA. Furthermore, there are logical connections that should exist between function 

blocks that are not in the model, which calls into question its validity (Salmon, et al., 2008). For 

example, there is no connection between knowledge or experience (Blocks 8 and 7, 

respectively) and motivation (Block 4). It makes sense to think that one’s experience and 

knowledge would influence the information that a person finds personally significant. For 

example, an operator who has been chastised in training for failing to adequately monitor steam 

generator levels and promptly identify a steam generator tube rupture (SGTR) may focus on 

steam generator indications to the detriment of other systems or be more likely to interpret 

steam generator (SG) level indications as signifying a tube rupture when the situation is actually 

a different event.

4.2.3 The Proximate Causes for the Understanding and Sensemaking Function

Klein’s model delineates the major steps underlying the Understanding and Sensemaking

function. With the assumptions that NPP operators have developed mental models for 

scenarios through their training and have procedures guiding and verifying their understanding 

of scenarios, the key steps to ensure the success of function are acquiring correct and complete 
66

data, selecting the right mental model/frame, and integrating the data with the frame to generate 

situation assessment. Therefore, although failure of Understanding and Sensemaking can 

occur in a number of ways, we can categorize them into the following proximate causes:

• Incorrect Data. The data the person is comparing with a frame is incomplete, incorrect, or 

otherwise insufficient to understand the situation. This may be due to one of the following 

reasons:

o the information itself is faulty, external to the person;

o there are errors in the perceptual process (in which case the failure is in the Detecting 

and Noticing function); or

o the person attends to inappropriate information, or focuses on inappropriate aspects of 

the information (e.g., operators are focused on coolant level and overlook rate of 

change).

• Incorrect Frame. The frame used to understand the situation is incorrect, incomplete, or 

otherwise insufficient to properly interpret the data. In this proximate cause, the mental 

model in use is not adequate to provide a correct understanding of the situation. This can 

be due to a number of factors, such as the person does not have the necessary experience 

to develop appropriate frames, or because the situation seems similar to a different frame. 

Whatever the reason, the cause of failure of Understanding and Sensemaking is that the 

person is using an inappropriate frame for the situation.

• Incorrect Integration of Data, Frames, or Data with a Frame. This refers to errors in 

integration. The person does not properly integrate pieces of information together, does not 

correctly match data with a frame, or does not appropriately integrate multiple frames, such 

as when a person does not properly merge a frame for a system (i.e., a system model) with 

the frame for the ongoing event (i.e., a situation model). In addition, because sensemaking 

is a continuous, dynamic process, the person must integrate new data into the frame 

periodically as the situation evolves. In this proximate cause, the data is correct, the frame 

is correct, but the integration, matching, or updating process goes awry.

4.2.4 Cognitive Mechanisms for Proximate Causes

One of the important differences between the Detecting and Noticing and Understanding and 

Sensemaking functions is the nature of the research that supports and explains the 

phenomenon. The Detecting and Noticing function lends itself to microcognitive models that 

focus on one small slice of the perceptual process. However, when it comes to how people 

understand and make sense of that perceived information, micro models no longer suffice. The 

manner in which people go about understanding their environment ranges from simple 

recognition to much more complex and effortful processing. To represent adequately this range 

of complexity, the models of cognition are consequently at a higher, more complex level. Most 

of the research reviewed in this effort focused on more macro models, such as SA and

sensemaking.

As a result, there are not groups or categories of literature outside of the models of 

sensemaking and SA that provide information on the cognitive mechanisms that can lead to the 

proximate causes of failure for the Understanding and Sensemaking function. We identified the 

cognitive mechanisms by the models themselves. We cluster the remainder of the literature 

reviewed in relation to these models by the factor the research examined, which maps well to 

the concept of performance influencing factors. The remainder of this chapter will be devoted to 

summarizing the cognitive mechanisms that can lead to the proximate causes and to a 

discussion of the most relevant PIFs to the Understanding and Sensemaking function (Section 

4.2.5).
67

4.2.4.1 Working Memory and Attention

As discussed earlier, memory plays a significant role in the sensemaking process, as the activity 

of merging perceived information or data with a frame or mental model occurs within working 

memory. Working memory and attention are the cognitive mechanisms that enable 

sensemaking to occur. As such, the human sensemaking process is strongly subject to the 

limitations of working memory. Before discussing the specific mechanisms identified in the 

cognitive framework for the proximate causes, it is important to overview some important 

findings regarding memory and SA or sensemaking. Relevant findings in the literature include:

• Working memory span, or the number of chunks one can hold active in memory at once, is 

correlated with SA performance. People with larger working memory spans tend to develop 

better SA (Gugerty & Tirre, 2000). The more information a person can work with, the more 

likely they are to develop a full picture of what is happening.

• Maintaining SA involves the verbal, spatial, and central executive components of working 

memory, selectively depending on the situation and nature of the task. Tracking a moving 

object relies on the spatial subsystem, and working memory works to actively support SA 

through processing and storage in the various working memory subsystems (Johannsdottir, 

2005).

• Long-term working memory (LT-WM) refers to the activated area of long-term memory that 

interacts with working memory as people cognitively process a problem or situation. It 

serves to extend working memory in a manner that involves skilled use of storage in and 

retrieval from long-term memory (Cook, 2001; Sohn & Doane, 2004). LT-WM has also been 

called skilled-memory, in that it refers to memory that is a function of knowledge and 

experience, and it is a significant mnemonic skill or structure for sensemaking performance 

(Cook, 2001), as increased efficiency in retrieving information (or frames or mental models) 

stored in long-term memory is important for sensemaking and SA. LT-WM has been shown 

to be a skill that operators can use to overcome the limitations of working memory 

(Jodlowski, 2008).

o LT-WM capabilities are strongly associated with SA performance in experts: experts with 

better LT-WM capacity perform better than experts with lower LT-WM capacity. 

However, for novices, working memory span is more important for SA performance than 

LT-WM (Sohn & Doane, 2004).

o Skilled memory or LT-WM may be more applicable to operation of systems in highly 

familiar conditions and less applicable to managing unexpected catastrophic failures. 

Expertise is built up by experience and repeated practice of cognitive and 

perceptual-motor skills. This means that LT-WM is highly beneficial for normal or 

familiar operations. Yet when operators are faced with unexpected, extreme, unfamiliar, 

surprising, or otherwise novel events, skilled-memory or LT-WM is less useful, and 

working memory span is more important to performance (Cook, 2001).

• Situation awareness failures can occur due to insufficient working memory span, or the 

decay of information in working memory over time. If people do not work to keep 

information active (e.g., by repeating or revisiting it), it can rapidly fade from memory 

(Baddeley, 1992; Baddeley & Hitch, 1974; Endsley, Bolté, & Jones, 2003). Endsley states 

that memory is crucial for SA, as aspects of the situation must be kept and used in memory 

to produce and maintain SA. Significant errors can result from systems that force operators 

to rely heavily on their memory (Endsley, 2000; Endsley, et al., 2003; Endsley & Garland, 

2000).
68

Attention is a phenomenon that is strongly related to memory. It is theorized to be the primary 

duty of the central executive, a component of working memory, to direct and control attention 

and flow of information in and out of working memory (Baddeley, 1992; Baddeley & Hitch, 

1974). Consequently, attention is also a primary factor for sensemaking or SA, as attention to 

the right or wrong information can make the difference between understanding the situation and 

not understanding. Relevant findings related to attention include:

• Taxing the central executive with tasks that require attention often impairs working memory 

performance on problems. Soliman (2010) found that executive function is strongly 

associated with SA. For people with already low SA, taxing the central executive produced 

significant performance decrements on executive tasks and on SA, whereas for people with 

high SA, taxing the central executive did not affect performance on the distractor tasks or 

overall SA.

• Experienced operators have mental models that inform them of the information they need to 

attend to and which information to disregard, and experts’ search patterns through even 

complex displays for information can be quite efficient (Durso & Gronlund, 1999; Endsley, 

2000).

• Operators tend to make rapid repeated visits to faulty systems with their gaze, rather than 

looking at the information for a longer duration. This tends to be due to the top-down 

knowledge and the structured environment, which allows operators to know what 

information they need to look at and when (Durso & Gronlund, 1999).

• Stressors can narrow the attentional field, or the area of the visual field that operators can 

attend to (Baddeley & Hitch, 1974). This means that potentially information in the periphery 

may be neglected, leading to poorer SA (Durso & Gronlund, 1999; Endsley, et al., 2003).

The above section overviewed the important general literature findings for the mechanisms of 

working memory and attention; the following section will discuss the mechanisms that were

developed specifically for the cognitive framework structure.

4.2.4.2 Links between cognitive mechanisms and proximate causes

As stated earlier, the three proximate causes for failure of the Understanding and Sensemaking

function are:

• Incorrect data

• Incorrect frame

• Incorrect integration of data, frames, or data with a frame

The cognitive mechanisms identified from the literature review provide an explanation of how

and why those causes may occur. 

Table 4-1 presents information excerpted from Appendix A.2 to show the cognitive mechanisms 

that are relevant for each proximate cause. Note that a few of the cognitive mechanisms apply 

to more than one proximate cause, such as “data not properly recognized, classified, or 

distinguished.”
69

Table 4-1. Cognitive mechanisms for the proximate causes.

Proximate Cause Cognitive Mechanism

Incorrect data 

(incorrect, incomplete, or inadequate information 

used to understand the situation)

Information available in the environment (including 

procedures) is not complete, correct, or otherwise 

sufficient to create understanding of the situation

Attention to wrong or inappropriate information 

Improper data or aspects of the data selected for 

comparison with or identification of a frame 

Incorrect or inappropriate or inadequate frame 

used to search for, identify, or attend to information 

Data not properly recognized, classified, or 

distinguished

Incorrect integration of data, frames, or data with 

a frame

Data not properly recognized, classified, or 

distinguished

Improper integration of information

Improper aspects of the frame selected for 

comparison with the data

Improper data or aspects of the data selected for 

comparison with or identification of a frame

Incorrect or failure to match data or information to 

a frame or mental model

Mental manipulation of the information (including

projection of future status) is inadequate, 

inaccurate, or otherwise inappropriate

Working memory limitations impair processing of 

information

Improper control of attention

Incorrect frame

(incorrect, incomplete, or improper frame or 

mental model used to understand the situation)

Incorrect or inadequate frame or mental model 

used to interpret or integrate information

Frame or mental model inappropriately preserved 

or confirmed when it should be rejected or 

reframed

Frame or mental model inappropriately rejected or 

reframed when it should be preserved or 

confirmed

Incorrect or inappropriate frame used to search for, 

identify, or attend to information

No frame or mental model exists to interpret the 

information or situation

4.2.5 The Effects of PIFs on Understanding and Sensemaking

PIFs influence sensemaking through various means, such as acting on working memory and 

attention. In building the cognitive framework, we identified experimental findings and inferred 

links between PIFs and proximate causes via the cognitive mechanisms identified in this 

literature review.
70

This section discusses specific literature findings related to PIFs that have been shown to affect

Understanding and Sensemaking. Consistent with the models of sensemaking and SA, most of 

the literature about the effects of PIFs falls into the following areas:

• Expertise

• Workload and task complexity

• Quality and availability of information

There is also information about other factors that affect performance such as stress, fatigue, 

and individual differences. We discuss each of these groups as they relate to PIFs of interest to 

HRA: training and experience, task complexity and workload, and human-system interface (HSI)

and procedures. We present some relevant findings in each of those areas to demonstrate the 

effects of those PIFs.

4.2.5.1 Training and Experience—Expertise and Goals

Much of the literature reviewed discussed expertise and goals as important influences on 

sensemaking. However, HRA traditionally concerns itself with training and experience, because 

those factors are easier to objectively measure by HRA analysts in the field. For example, it is 

simpler for an HRA analyst (who is typically not a psychologist) to determine that an operator 

has successfully passed all qualification training and requalification training, and has 12 years of 

experience as a reactor operator than to subjectively evaluate whether the operator is an expert 

in reactivity control. An HRA analyst can more easily identify whether an operator has received 

training on a particular event than she/he can determine whether that operator has retained the 

knowledge the training was intended to impart. Training and experience are means for building 

expertise and teaching operators on which goals are relevant and how they should prioritize 

goals. Training, experience, and expertise are clearly linked; subsequently, when a PIF of 

expertise is identified by the literature, the PIFs of knowledge/experience/expertise and training 

are selected in the Appendix A.2 table.

Expertise

One of the primary factors identified repeatedly in psychological and human factors research 

that affects how well people understand their environment is expertise. Numerous researchers 

have documented that people with more experience tend to outperform novices (Anderson, 

2000; Endsley, 2000, 2006; Endsley & Garland, 2000; Klein, et al., 2007). A large body of 

research has been conducted to determine why this is the case. It is not always possible in 

real-world settings to have an expert on hand to assist in determining what is happening, so 

finding a way to improve novice performance is considered to be of great importance. In the 

case of plant upset, control room operators have access to expertise in the form of shift 

technical advisors (STAs), more senior operations personnel, subject matter experts (SMEs), 

and in the case of major events, an entire technical support center that is designed to provide 

operators with the expertise and skill they need to control the plant.

What is it about expertise that facilitates such improved performance over novices? On this, 

both basic laboratory and applied psychological research has provided a large amount of 

important information. For ease of understanding, the findings related to expertise are grouped 

in loose categories of Expertise and Frames, Expertise and Interpretation of Information, 

Expertise and Memory/Attention, Expertise and Problem Solving/Use of Information, and Other 

Findings.

Expertise and Frames

• Experts and novices use the same reasoning strategies, logic, and abductive reasoning, but 

experts have a richer repertoire of frames. They have more knowledge and a broader, more 
71

detailed set of frames (or models/patterns/information) from which to work. Novices, on the 

other hand, have a limited knowledge base from which to work (Anderson, 2000; Klein, et 

al., 2007).

• Experts’ mental models are richer in terms of having more detail, more variety, more subtle 

and finer degrees of differentiation, and more comprehensive coverage of the domain. 

Experts are more sensitive to the context of the situation (Anderson, 2000; Klein, et al., 

2007).

• Experts are more likely to be able to work with multiple frames at once. When faced with 

data that has more than one explanation, experts can deliberately elaborate two to three 

frames simultaneously, looking for information that will rule out one of the frames (Feltovich, 

Johnson, Moller, & Swanson, 1984; Klein, et al., 2007).

Expertise and Interpretation of Information

• Information in the environment is not presented in neat packages of data to people. Instead, 

expertise is needed to sift through information to select and define what information is 

important (Klein, et al., 2007).

• Novices are more likely than experts to interpret information that is actually irrelevant as 

important (Klein, et al., 2007).

• Experts are more prone to question data than novices because they are more familiar with 

cases of faulty information. This may also mean that experts are more confident in their 

frames and skeptical of contrary evidence (making them more susceptible to fixation errors) 

than novices, who are less confident in frames and more sensitive to contrary data (Klein, et 

al., 2006).

Expertise and Memory/Attention

• Experts have a greater ability to store problem information in long-term memory, store it in a 

well-structured and organized manner, and to be able to quickly and easily retrieve it. 

Experts are better able to remember more patterns (or frames) and larger, more complex 

patterns than novices (Anderson, 2000).

• As people develop expertise in an area, they create more complex chunks of information. 

People can only process a limited amount of information in working memory at once. One 

strategy to deal with this limitation is to break information into chunks. This is why telephone 

numbers are broken up into three chunks—area code, prefix, and four-digit number. People 

recall the three chunks, not ten digits. Experts’ chunks are more complex and structured 

than novices, so experts can work with more information successfully than novices 

(Anderson, 2000). Experts are better than novices at organizing information into meaningful 

units (Doane, Sohn, & Jodlowski, 2004).

• Experts have stored the solutions to many problems in long-term memory, which gives them 

an advantage over novices who have to tackle each new problem from scratch. This allows 

experts to focus on more sophisticated and unique aspects of the problem (Anderson, 

2000).

• Working memory capabilities are more predictive of novice SA, but long-term working 

memory is more important for expert SA. For experts with a low working memory span, 

long-term working memory skills are crucial for successful SA (Sohn, 2000).

• Experts develop automaticity to their cognitive processes as well as to the physical actions 

for a task. This allows them to devote their cognitive resources and attention to areas that 

are less routine (Endsley, 2006; Shebilske, Goettl, & Garland, 2000).
72

Expertise and Problem Solving/Use of Information

• Experts can take advantage of their knowledge base to use redundancies and constraints in 

a situation to generate expectancies. They can fill gaps in their understanding with 

assumptions and inferences based on knowledge of similar situations (Klein, et al., 2007).

• Experts are more likely to have a functional understanding of the situation or problem and 

invoke principles that they can use to solve the problem, whereas novices tend to have a 

poorer understanding of how to go about solving the problem, even if they know what 

factors are relevant. They just do not know how to use that information (Chi, Feltovich, & 

Glaser, 1981; Klein, et al., 2007). Experts are better able to organize their problem solving 

efforts in a way that is optimally suited to solving the problem (Anderson, 2000).

• Experts show a greater capability for anticipation—to identify what is coming next and what 

must be done in preparation (Klein, et al., 2007; Klein, Snowden, & Pin, 2011). This is 

plausibly due to their frames or mental models having more information about the 

progression of a situation across time, so they know what comes next.

• Experts perceive problems in ways that enable more effective problem solving. They have a 

richer set of perceptual features for encoding problems (Anderson, 2000).

• Experts are more accurate than novices in determining whether changes in the situation are 

in accordance with expectations of their mental model (Doane, et al., 2004).

• Experts are more context-dependent than novices, meaning that they are more sensitive to 

surroundings, circumstance, and the specific aspects of the situation than novices. This 

makes experts better at classifying situations than novices (Federico, 1995)

Other Findings Related to Expertise

Expertise also has negative aspects. First, a great deal of practice is required to develop 

expertise in any field (Anderson, 2000), and the acquired expertise can become degraded 

without reinforced learning over the time. Next, expertise can be quite narrow. Expertise 

developed in one area may fail to transfer to other domains, even areas that are similar. 

Goals

Goals are important to sensemaking for a number of reasons. Sensemaking is a goal-driven 

process, with the goal being to understand what is going on and respond appropriately (Klein, et 

al., 2006; Klein, et al., 2007), perhaps to bring an unexpected event under control. In 

goal-driven processing, a person focuses attention in accordance with the active goals. The 

operator actively seeks information related to accomplishing the goal, and the goal contributes 

to interpretation of the data (Endsley, 2000).

Specific findings related to goal selection include:

• For the following reasons, selection of correct or appropriate goals is critical for properly 

understanding the situation. If a person is pursuing an inappropriate or less important goal, 

critical information may be missed (Endsley, 2000):

o The active goals direct the selection of the mental model used for understanding the 

situation (Endsley, 2000).

o The goal and associated mental model are used to direct attention, identify and select 

important information from the environment, and direct scan patterns (Endsley, 2000).

o People use goals and their associated mental models to interpret and integrate 

information into a full understanding or comprehension of the situation. “The goal 

determines the ‘so what’ of the information” (Endsley, 2000, p. 20).
73

• Failure to maintain multiple goals has been shown to cause problems in maintaining SA 

(Endsley, 1995)

• Interpretation of information from the environment depends on the goal of the activity and 

the motivation behind it, including the significance the operator places on the goal (Bedny & 

Meister, 1999). If the goal (image-goal) is inappropriate for the situation, then the person 

may incorrectly interpret the information.

4.2.5.2 Task Complexity—Working Memory and Attention

Complexity refers to the quantity, variety, and intermingled relations of the elements of an object 

or situation (Xing, 2004). Park and Jung (2007) developed a task complexity measure to 

quantify the complexity of tasks stipulated in emergency operating procedures in NPPs. The 

measure consists of five complexity factors: (1) amount of information to be managed by 

operators, (2) logical entanglement due to the logical sequence of the required actions, (3) 

amount of actions to be accomplished by operators, (4) amount of system knowledge in 

recognizing the problem space, and (5) amount of cognitive resources in establishing an 

appropriate decision criterion. Task complexity contributes to human performance degradation 

because the increased load demands more cognitive resources such as working memory and 

attention. With higher task complexity, such as the amount of information that a person has to 

work with, the person must increase the cognitive resources to maintain and integrate that 

information.

We treat memory and attention as cognitive mechanisms in the cognitive framework, both of 

which were discussed in detail in Section 4.2.4.1. Below is a brief summary of certain findings 

regarding the effects of task complexity on memory and attention, which in turn affect 

Understanding and Sensemaking.

• Situation awareness failures can occur due to insufficient working memory span, which is 

needed to process complex tasks. Significant errors can result from systems that force 

operators to rely heavily on their memory (Endsley, 2000; Endsley et al., 2003; Endsley & 

Garland, 2000).

• Information in working memory decays over time. If the task requires individuals to retain 

the information for a long period of time before the information is used, it can rapidly fade 

from memory unless individuals work to keep information active (i.e., by repeating or 

revisiting it) (Baddeley, 1992; Baddeley & Hitch, 1974; Endsley, et al., 2003). Maintaining a 

high and accurate level of SA necessitates expending cognitive resources—it requires 

cognitive workload, which may compete with other parallel cognitive tasks. Heavy 

concurrent task demands may in turn divert resources from the maintenance of SA.

• Even experienced operators can make errors in the process of prioritizing attention and 

neglect to attend to certain information, particularly when they have to juggle numerous 

competing tasks and pieces of information. Distraction due to other tasks or overly salient 

information is a major contributor to SA errors (Adams, et al., 1995; Endsley, 2000; Jones & 

Endsley, 1996).
74

4.2.5.3 Workload—Working Memory and Attention

Workload6 is another strong influence on sensemaking performance. It is generally referred to 

as the amount of cognitive or physical tasks an operator must handle in a situation. By this 

definition, workload is related to task complexity and task load for the time available to perform 

the task. The more tasks a person has to perform, the more demands on the person’s 

resources to perform the tasks within the given time. Cognitive or mental workload may be 

conceptualized as a function of the supply and demand of cognitive processing resources, with 

two main determinants: the task demands (e.g., task difficulty, priority, and situational factors); 

and the internal supply of attention or cognitive resources available to support processing (e.g.,

Detecting and Noticing, Understanding and Sensemaking, planning, and Decisionmaking) 

(Tsang & Vidulich, 2006). Therefore, with higher cognitive workloads, demands on attention, 

working memory, and long-term working memory are increased, and it becomes more likely that 

sensemaking or SA will be impaired. Experience may mediate cognitive workload: experienced 

operators often have learned strategies for managing high workload. For example, teamwork is 

a way to split high workload among the team members; displays can reduce the load of 

remembering information; and procedures can reduce the mental workload in assessing 

situations by assisting operators with diagnosis. Below are listed several important findings 

related to workload:

• With increasing physical workload, there was a general trend of decreasing SA for 

perceptual knowledge (Level 1 SA), comprehension (Level 2 SA), and overall SA. However, 

physical workload did not appear to affect Level 3 SA or cognitive task (a military helicopter 

loading simulation) performance. Therefore, physical workload appears to affect SA, but 

this does not necessarily translate to task performance decrements (Perry, Sheik-Nainar, 

Segall, Ma, & Kaber, 2008).

• Both high and low workload are associated with degraded SA, while sufficient SA is needed 

for reliable task performance regardless of the workload. This relationship between 

workload and SA is also mediated by level of expertise. An experienced operator can 

maintain SA with lower investment of cognitive resources, often due to the schema or 

mental model used to assist in understanding the situation. However, it is a dangerous trap 

to assume that a more experienced operator necessarily has a better understanding of the 

situation; heavy reliance on resource-free mental models may mean that the operator 

incorrectly interprets or fails to notice an unexpected or surprising event (Wickens, 2001).

• Skill level and expertise moderate the supply of attentional and cognitive resources. A 

certain level of task demands may impose different levels of workload on an operator, 

depending on ability and skill level. A high skill level is functionally equivalent to having a 

larger cognitive processing resource supply (Tsang & Vidulich, 2006). Similarly, external 

resources, such as cognitive aids, (e.g., notes, procedures, decision support systems), 

support centers, and teamwork can also moderate the supply of attentional and cognitive 

resources.

• It takes a careful balancing of workload and SA to ensure that one can maintain SA without 

excessive workload. While it takes a certain amount of workload to maintain SA, too much 

workload overwhelms cognitive resources. Strategic management of this balancing act is 

postulated to be a function of executive control of attention (the central executive component 

 

6. While most research conducted on workload used the phrase “workload,” the PIF taxonomy used in the cognitive 

framework labels workload as “task load.” Groth’s (2009) taxonomy also calls out time load (the perception of 

time pressure) and other loads (to catch any other loadings on the person’s activities). In this report, the authors 

use “workload” and “task load” interchangeably.
75

of working memory), which is a distinct aspect of human cognition that consumes 

processing resources. Strategic management of workload and attention competes with the

other mental processes that generate cognitive workload on the processing resources. 

However, strategic management can optimize performance by smartly planning and 

allocating the limited resources to the processes that need the most attention (Tsang & 

Vidulich, 2006).

4.2.5.4 HSI and Procedures—Quality and Availability of Information

Information quality (e.g., accuracy, structure, and availability) in a situation is important for 

correct understanding of the situation—the better and more accurate the information available, 

the better people are able to make sense of it. For information to be considered to be of high 

quality for Understanding and Sensemaking, the information should be well-organized and 

structured in such a way that it is consistent or compatible with a person’s mental model or the 

frames that the person is using to comprehend the situation. Thus, information from the 

human-system interface (including procedures) is of crucial importance for properly 

understanding an event (Endsley, 1995, 2000; Endsley & Garland, 2000; Jones & Endsley, 

1996, 2000; Kim, Yoon, & Choi, 1999). Hence, HSI features such as multimodality, visibility of 

others actions, alarm filtering, overview displays, and memory aids can substantially affect 

understanding and sensemaking of the information.

• A person is more likely to identify information that is extremely inconsistent with a current 

schema (or schema-bizarre information) as contradictory. This often leads to recognizing 

that either the data or the schema is in error, whereas information that is seen as irrelevant 

to the schema (but is relevant to the objective situation) is simply not noticed, and an 

incorrect schema persists (Jones & Endsley, 2000).

• System automation can undermine SA by taking the operator out of the loop. If complex, 

automated systems do not provide operators with enough information to keep them in the 

loop, operators may struggle to understand what is going on when the automation fails or 

reaches situations that it is not equipped to handle (Endsley, et al., 2003).

• Highly complex systems make it difficult for people to develop sufficient internal 

representations of how the systems work. System complexity can slow down the ability of 

people to take in information (due to the overwhelming amount of information), and it 

undermines their ability to correctly interpret the information and project what is likely to 

happen next (Endsley, et al., 2003).

• The “Las Vegas Strip” phenomenon—when systems display flashing lights, moving icons, 

overuse bright colors, or simply present too much information, information overload can 

occur. Misplaced salience can also occur when information that is objectively irrelevant (or 

inaccurate) is physically more salient than important information, drawing operator attention 

away from needed information (Endsley, et al., 2003). This phenomenon is certainly 

relevant for Detecting and Noticing; it is also relevant for Understanding and Sensemaking

because it may lead to operators to use inappropriate data in the sensemaking process, or 

prevent them from finding the necessary information that is buried amongst irrelevant yet 

overly salient information.

4.2.5.5 Individual Factors—Stress, Fatigue, and Cognitive Style

Research has also shown that certain individual factors affect sensemaking performance. 

These factors are often referred to as “individual differences,” because they vary from person to 

person. Individual difference factors include such things as personality types, adaptability, 

ability to handle stress, and approaches to strategic thinking. During an HRA, an analyst 

typically will evaluate some individual factors, such as stress and fatigue. However, many of 
76

these types of variables are not commonly considered in HRA, particularly variables like 

personality. Primarily, performance based on personality is considered to be something that 

HRA analysts will not be able to predict. Nevertheless, when one considers performance of the 

Understanding and Sensemaking function, some individual difference findings may prove 

informative in identifying how sensemaking may fail. Several important findings in the literature 

related to individual factors are listed below.

• People tend to be less effective in gathering information under stress. They are more likely 

to pay less attention to peripheral information, become disorganized in their scanning 

patterns, succumb to attentional tunneling, and make decisions without considering all 

available information (Endsley, et al., 2003).

• Fatigue can also reduce working memory span and attentional executive control of working 

memory. If people have to focus on remaining vigilant, they may have less attention 

resources to use on important information (Endsley, et al., 2003).

• Analytic thinkers (people who view the world as composed of separate elements that can be 

understood individually and independently) tend to focus on objects and dispositions. 

Holistic thinkers, on the other hand, are more likely to include context, and focus on the 

relationships between different objects and context (Lin, 2009). Analytic and holistic 

thinkers use information differently during sensemaking, with the difference being strongest 

during initial sensemaking with limited information. Holistic thinkers are more influenced by 

the type of information presented during initial sensemaking, identify a problem in the 

situation sooner, and change their sensemaking more than analytic thinkers. With new 

contradictory information, though, the difference between analytic and holistic thinkers 

disappears. The information content ultimately influences sensemaking (Lin, 2009).

4.3 Summary

This chapter discussed Understanding and Sensemaking, the function by which people make 

sense of the information in their environment. The chapter presented an overview of four 

important models that explain how Understanding and Sensemaking works, including the 

data/frame theory of sensemaking (Klein, et al., 2007) and three models of SA (Bedny & 

Meister, 1999; Endsley, 1995; Smith & Hancock, 1995). The chapter then discussed the 

proximate causes of and cognitive mechanisms for failure of sensemaking, and identified 

relevant PIFs that influence the sensemaking process.

Some of the literature reviewed demonstrates a general effect on performance, such as 

workload, without linking the effect to specific errors. Other research provided information 

regarding cognitive mechanisms (e.g., working memory), also without linking the mechanisms to 

specific errors. Therefore, when organizing the literature that was reviewed, we developed a 

structure that linked PIFs to cognitive mechanisms to proximate causes of failure, links that 

previously were not identified in the literature in such a manner. This linked structure is the 

primary product of this literature review effort, and can be found in the detailed tables in 

Appendix A.2 and the associated tree diagrams in Appendix B.2. Notice that the literature 

review of this chapter was under the assumptions that operators have mental models/frames for 

scenarios they work on and there are procedures to guide and verify their understanding of 

scenarios. Therefore, the review did not explore in depth the mechanisms for how humans 

achieve understanding and sensemaking for unfamiliar scenarios or without procedure 

guidance. Also, the review did not address a large body of literature about how humans 

achieve understanding for data/information with high uncertainties and how humans use team 

or community expert judgment to assess situations when data is incomplete. These topics 

should be addressed when using the cognitive framework to analyze human events outside the 

scope of the assumptions for this function. 
77

5. DECISIONMAKING

5.1 Introduction

Decisionmaking is the judgment of what should be done and the decision to do it. Yates (2003)

defines decision as “a commitment to a course of action that is intended to yield results that are 

satisfying for specified individuals” (p. 24). Researchers have long been interested in 

understanding how decisions are made and being able to predict the decided upon solutions. 

Modeling decisionmaking has progressed from the study of how decisions ought to be made to 

the study of how they are actually made. The study of decisionmaking has also evolved to 

include a greater emphasis on the context in which the decision is made and consideration that 

decisions are often made in a dynamic and changing environment. The decisionmaking

process described in this chapter builds on the information and situation assessment completed 

through the Understanding and Sensemaking function (Chapter 0).

Decisionmaking within a nuclear power plant (NPP) is characterized as involving experts and 

being largely driven by procedures. In recognition of the complex environment inherent in NPP 

operations, Espinosa-Paredes et al. (2008) focus on the need to develop emergency operating 

procedures that aid the operator in navigating these complexities and arriving at the correct 

decision. Although procedures usually dictate the actions of the operators, Roth (1997) explains 

that operators still maintain a mental model of the situation and plan their course of action 

semi-independently of the procedures. That is, operators know the goals that they need to 

accomplish and how that should be done, and they use the procedures to confirm these beliefs. 

Furthermore, situations may arise that procedures do not cover. In these instances, operators 

must rely on their expert knowledge to solve the problem and implement the appropriate 

decision.

Human reliability analysis (HRA) needs to model the complexity within the NPP environment 

and the potential for decisionmaking errors despite operators being provided with procedural 

direction. One HRA method in particular, ATHEANA, provides some guidance in evaluating

errors within NPP operations, especially in the response planning phase (NUREG-1624; NRC, 

2000). For example, the method provides plausible explanations and error mechanisms to 

explain why a small or large change in a parameter may be missed or not attended to, such as 

the control room crew being fixated on another parameter or having apathy or a lack of urgency 

in responding to the parametric change. To have a better understanding of the type of 

decisionmaking errors made within an NPP, and to be better able to represent the error within 

HRA, it is important to understand the thought processes and phases of decisionmaking that 

lead to errors.

5.2 Historical Background of Decisionmaking

Decisionmaking has been studied for many years within the laboratory and in real world 

settings. Early attempts were based on normative models and had origins in economics, 

mathematics, and philosophy. These models described the human decisionmaker as directed 

by “rational” maximizing behavior—in other words, trying to choose the alternative that 

maximizes gains. These models are often referred to as normative models. Rational choice 

theory (RCT) was one of the earliest examples and was built on axioms assuming the 

decisionmaker would act in a consistent manner and would be guided by rational rules 

(Nicholson, 1995). Models within this paradigm considered either decisions made in an 

environment of uncertainty or the absence of uncertainty. For example, in the absence of 

uncertainty, someone looking to purchase a product knows exactly what he or she will get once 

he or she makes the decision to buy. However, many decisions are made with uncertain 

outcomes. For example, buying a lottery ticket allows you odds of winning money, but not 
78

certainty. One key model proposed to account for decisions under uncertainty was expected 

utility theory (EUT) (Von Neumann & Morgenstern, 1947).

The primary claim of normative models, that behavior is rational, fails to portray realistic 

decisionmaking. The set of rules defined by the models has been shown to be violated in 

numerous studies (Mellers, Schwartz, & Cooke, 1998). For example, the theories posit that the 

decisionmaker should act in a consistent fashion such that if one alternative is preferred, it 

should always be preferred; however, research has shown that this is not always the case. 

Most notably, the existence of cognitive heuristics and biases such as the framing effect 

(Kahneman & Tversky, 1984) has demonstrated the “irrational” behavior of people. Specifically, 

the framing effect states that the way in which a problem is worded will influence the decision 

made. This effect directly contradicts the hypothesis that if one alternative is preferred it will 

always be preferred as this preference can be swayed based on the description of the problem.

Descriptive models were developed in an attempt to better represent actual decisionmaking

behavior. Whereas normative models describe how a decision ought to be made, descriptive 

models seek to describe how a decision is made. Simon (1959) introduced a transition to more 

descriptive theories of decisionmaking by describing limitations on the human processing 

capacity leading to the idea of bounded (limited) rationality. This concept states that the 

decisionmaker operates via satisficing instead of maximizing when making decisions (Simon, 

1955). Therefore, the decisionmaker acts in ways in which the desired outcome is achieved or 

satisfied, but it may not be in the best optimal way (i.e., outcome is maximized). That is, the 

decisionmaker finds a solution that meets the desired criteria and is “good enough,” but does 

not spend the extra time and resources to thoroughly explore each solution alternative to find 

the best or optimal solution.

To explain some behaviors previously unaccounted for by normative models, Kahneman and 

Tversky (1979) developed prospect theory, which also falls into the descriptive camp of 

decisionmaking theories and models. Prospect theory not only accounts for risk seeking and 

risk aversion (i.e., the idea that some people engage in behaviors for the thrill of the risk [risk 

seeking] or adapt an overly conservative nature to avoid risk [risk aversion]), the theory also 

explains loss aversion (Tversky & Fox, 1995). Loss aversion is the phenomenon observed in 

people’s behavior of being overly sensitive and opposed to losses. That is, the person is more 

averse to the loss of some amount than they are to a gain of the same amount (i.e., more 

averse to losing $20 than to gaining $20).

Prospect theory and other descriptive models or theories are often studied in contrived 

situations involving simple decisions being made by novices. Within these contrived 

environments, the models have fared well in predicting behavior; however, the models have not 

been shown to generalize well to environments that are more complex. The descriptive models 

typically do not adequately explain real-world decisionmaking involving experts in a time-critical 

situation. The models have been unable to explain the process the decisionmaker goes through 

in evaluating a situation, reviewing options, and deciding on a course of action in real-world 

settings.

5.3 Naturalistic Decisionmaking

Naturalistic decisionmaking (NDM) sought to expand the setting in which decisionmaking was 

examined. NDM departs from the earlier theories of decisionmaking and considers the 

decisionmaker in a real-world setting in which decisions are typically embedded in a larger task.

Researchers in this area focus on studying “time pressure, uncertainty, ill-defined goals, high 

personal stakes, and other complexities that characterize decisionmaking in real-world settings” 

(Lipshitz, Klein, Orasanu, & Salas, 2001, p. 332). Several characteristics of NDM diverge from 

previous representations of decisionmaking. Lipshitz, Klein, Orasanu, and Salas (2001) explain 
79

four essential characteristics of NDM models. First, NDM focuses on the cognitive processes 

that the decisionmaker engages in instead of solely trying to predict the decision that will be 

made. Second, NDM describes the decisionmaker as judging the applicability of alternative 

solutions for reaching a goal one at a time instead of in a comparative manner (a versus b), and 

the search for alternatives is not exhaustive. Third, NDM focuses on experts instead of novice 

decisionmakers and places a great deal of importance on the context in which the decision must 

be made. Fourth, alternatives evaluated by the decisionmaker are only those that can be 

realistically implemented.

One model of particular note within NDM is the recognition-primed decision (RPD) model (Klein, 

1993, 1998). RPD was primarily developed in an attempt to explain decisionmaking of experts 

in stressful situations and under time pressures. A defining feature of RPD is that the 

decisionmaker does not compare alternative solutions to each other when searching for an 

optimal solution; instead, the decisionmaker will sequentially go through the list of alternatives 

until arriving at a sufficient solution. Thus, RPD is based on satisficing (finding a sufficient 

solution) instead of optimizing (finding an optimal solution).

RPD consists of three phases: situation recognition, serial option evaluation, and mental 

simulation. The process engaged in by the decisionmaker is represented in Figure 5-1 (Klein, 

1993). In the first stage (situation recognition), the decisionmaker assesses the situation and 

determines if the situation is typical or if he or she has encountered it before. A decisionmaker 

may rely on relevant cues from the situation, expectancies about how the situation may evolve, 

and/or stated goals for success in comparing the situation to previously encountered situations. 

A decisionmaker can respond to a typical situation with typical and known responses. A novel 

situation, on the other hand, will require new techniques or new applications of known 

techniques. In the second stage (serial option evaluation), the decisionmaker sequentially 

reviews alternative solutions to the problem. The solutions that are considered most typical 

would be considered first. The solutions are evaluated, one at a time, within the third phase as 

the decisionmaker mentally simulates the implementation of the solution and the outcome. 

Based on this simulation, the decisionmaker implements the solution as-is, changes it 

somewhat, or discards it and imagines another solution.

The phases presented by the RPD model can be generalized to represent the phases of 

decisionmaking within an uncertain, continuously evolving environment. The phases progress 

through first comparing the currently encountered situation to the mental model and developing 

goals to be achieved with the decision, next developing alternative solutions for reaching these 

goals, and finally implementing the solution. The amount of time spent within each phase will 

depend on the familiarity with the situation as well as the complexity of the situation. The 

amount of procedural guidance (and other types of decision aids) can also have an impact on 

the decisionmaking process. This realization is particularly true when considering 

decisionmaking in an NPP setting.
80

Figure 5-1. Integrated version of recognition-primed decision model (Klein, 1993).

5.4 Decisionmaking in a Nuclear Power Plant

Although each of the theories and models of decisionmaking discussed in the previous sections 

has provided insight into the decisionmaking processes, not all of them are useful for our 

purposes of describing the decisionmaking process undertaken by the operator within an NPP. 

For instance, normative models are very useful in developing decision aids because they focus 

on how decisions should be made (Edwards & Fasolo, 2001). However, these philosophies are 

not useful in helping determine where errors occur in the decisionmaking process employed by 

NPP operators. That is, the focus on determining where errors may occur in the 

decisionmaking process must focus on how decisions are actually made and not on how they 

should be made. For future applications, normative models may serve a role in preventing an 

error from occurring by aiding the NPP operator in making a decision (i.e., the normative model 

philosophy is used in constructing the decision aid to be used by the operator).

Descriptive models provide some insight when discussing errors made by NPP operators. 

However, these models typically focus on studying inexperienced individuals making decisions 

in laboratory settings where contextual factors play a limited role. These studies have been 

useful in uncovering decision heuristics that people use in making everyday decisions. These 

heuristics may be very useful in reducing the cognitive complexity of decisions, but may lead to 

biasing errors (Tversky & Kahneman, 1974). These biases may still be seen in NPP operations 

and are useful in this study; however, a broader view of decisionmaking within the NPP 

environment must be taken to fully understand the types of problems and errors that may be 
81

encountered. Researchers only achieved the ability to study people within their environments 

and account for the environmental and situational impacts on the decision by moving to NDM 

models. In fact, Zsambok (1997) defines NDM as, “the way people use their experience to 

make decisions in field settings.” (pg. 4).

NDM provides a good framework for modeling the decisions made by NPP operators. Greitzer, 

Podmore, Robinson, and Ey (2010) tailored the NDM model to NPP control room operation 

where decisionmaking is largely directed by procedures. Figure 5-2 shows a reproduction of 

their model.

Figure 5-2. Integrated NDM model (Greitzer, et al., 2010).

This integrated NDM model includes concepts from situation awareness (SA) (covered more 

extensively in Chapter 0 of this document), RPD, and recognition/metacognition (R/M) (Cohen, 

Freeman, & Thompson, 1997). The initial processing of the cues and formation of the story is 

done through processes of SA (Endsley, 1997) and early stages of RPD. Chapter 4 of this 

document covers this process, which sets the groundwork for response planning. It is during 

the process of Understanding and Sensemaking that the person forms the initial mental model 

characterizing the situation. Following these phases, the person generates alternative options 

and tests for their applicability to the situation through mental simulations. One feature of this 

model is its recognition of the continued role of mental models in forming the story of the 

situation and in selecting an applicable course of action.

R/M (Cohen, et al., 1997) explains how decisionmakers evaluate and improve their situational 

assessment. Given enough time exists for the evaluation, the decisionmaker will engage in a 

process of critiquing in which the assessment will be judged based on incompleteness 

(assessing if the situation is represented completely), unreliability (searching for assumptions 

made during the assessment that are unreliable or doubtful), and conflict (assessing if there are 

existing cues that contradict the assessment). The integrated NDM model proposed by Greitzer 
82

et al. (2010) includes the idea of critiquing by modeling additional loops of mental simulation 

during the pattern recognition process.

This model seems to work well in identifying the process an experienced operator goes through 

in making a decision, even in the presence of procedures. In the case of experienced operators 

when several procedures are available and numerous situations and recovery strategies are 

trained, the operator may take three approaches when planning a response (Cacciabue, 

Mancini, & Bersini, 1990):

1. In a very familiar setting in which the cues match almost perfectly the procedural guidance, 

the operator may follow the procedures with little diagnosis needed.

2. In a familiar setting that deviates just slightly from either procedural guidance or from 

previously encountered situations, the operator will have to adapt some and plan a response 

based on an analogous experience.

3. In a novel setting, the operator will have to construct a new response plan using his or her 

knowledge of the plant and system and previous experience.

Each of these options, but particularly the last two, may be seen through the lens of the 

integrated NDM model. The operator or crew will use cues presented in the situation to 

construct a story of what is happening and how the scenario is unfolding. This mental image 

will be used in developing a response plan and alternative actions; the response plan may be 

largely prompted by procedures or entirely conceived by the operators. The operator may 

evaluate the response plan or action script through mental simulation to evaluate its suitability 

and then put it into action.

One of the defining features of decisionmaking in an NPP is the dynamic nature of the event. 

Maintaining appropriate situation awareness of the event, updating the mental model of the 

situation, and planning the response accordingly are important steps (Murphy & Mitchell, 1986). 

And these steps continue to occur even in the presence of procedures. During the evolving and 

dynamic nature of an NPP event, operators were found to follow their procedures, but they also 

“actively construct a mental representation of plant state and use this mental representation to 

identify malfunctions, anticipate future problems, evaluate the appropriateness of procedure 

steps given the situation, and redirect the procedural path when judged necessary” (Roth, 1997, 

p. 176). So, even with procedures present, the operator’s ultimate decision is largely impacted 

by his or her own cognitive processes.

5.5 Proximate Causes of Failure of Decisionmaking

Errors in decisionmaking have been studied in each of the fields reviewed. Descriptive theories 

have typically focused on the reliance on fallible heuristics and breakdowns in decision 

processes. However, NDM “generally tries to understand error in a broader context, including 

insufficient experience.” That is, “NDM researchers... are less likely to attribute the error to 

faulty reasoning strategies, preferring to use the error as an indicator of poor training or 

dysfunctional organizational demands, or flawed design of human-computer interface” (Lipshitz, 

et al., 2001, p. 340). Research on errors in NDM has searched for the contextual factors that 

may contribute to the errors. This focus in NDM makes it a prime candidate for the present 

effort, as a primary goal for this study is identifying the PIFs that contribute to errors.

We examined the phases or steps of the decisionmaking process where failures were likely to 

occur. This was performed on the RPD model proposed by Klein (1993, 1998) and the 

integrated NDM model proposed by Greitzer, Podmore, Robinson and Ey (2010). We

conducted a comprehensive review of literature within decisionmaking (including the models 

discussed in the previous sections) to identify the cognitive mechanisms that lead to errors 
83

within these phases and processes. The cognitive mechanisms identified were grouped into 

three proximate causes according to the major phases in the decisionmaking models.

The proximate causes and relevant cognitive mechanisms identified are the follows: 

1. Incorrect Goals or Priorities Set. Goals are set as the objectives that the decision should 

achieve. Goals are the measure for viewing the decision as successful or not. Although 

goals are formed during any decisionmaking process, they are especially relevant during 

novel situations when there is no previous experience that the current situation and the 

outcome of the decided upon action can be compared against to measure success. If more 

than one goal is selected, priorities are assigned to the goals to determine the order in which 

they are to be addressed. This proximate cause includes errors that occur in either what 

goals are set or what priorities are assigned. Cognitive mechanisms include:

a. Incorrect goals selected. Errors may arise if the operators select the wrong goal to work 

toward. A variant of this cognitive mechanism is if the operator selects an implausible 

goal that cannot be achieved.

b. Incorrect prioritization of goals. Goals may be ordered incorrectly in the operators’ mind 

or given the wrong priority, such that less important goals are addressed first.

c. Incorrect judgment of goal success. The threshold used by the operator to judge goal 

success may be incorrectly set too low, or be incorrectly determined as met when it was 

not.

2. Incorrect Internal Pattern Matching. During the Understanding and Sensemaking stage, an 

operator forms a mental model of the current situation. During pattern matching, the 

operator matches the situation with stored plans, scripts, and stories to judge the typicality of 

the situation and devise a plan as needed. If the operator judges the situation as being 

typical, a previous response plan can be used again. If the situation is novel, the operator 

may find a similar situation that can be adapted to fit the current situation. This proximate 

cause includes errors that occur during the mental exercise of pattern matching. Cognitive

mechanisms include:

a. Not updating the mental model to reflect the changing state of the system. Events within 

an NPP may evolve quite quickly, and the operator must update his or her mental model 

to reflect this dynamic nature.

b. Fail to retrieve previous experiences. During pattern matching, the operator compares 

the current situation to previously encountered situations to devise an appropriate 

response plan. Errors may occur in this recollection process if the operator fails to 

evoke appropriate previous experiences.

c. Incorrect recall of previous experiences. Similar to the previous cognitive mechanism 

dealing with the recollection of previous experiences, in this case the error may occur 

due to an incorrect recollection of the previous experience. In other words, the operator 

may incorrectly remember how the previous experience was responded to.

d. Incorrectly comparing the mental model to previously encountered situations. The 

comparison with previously encountered situations may cause an error either because 

the comparison was incomplete or simply because a mistake occurred in the 

comparison.

e. Cognitive biases. Confirmation bias and availability bias may be particularly pertinent to 

causing errors in this phase of decisionmaking (Einhorn & Hogarth, 1978; Tversky & 

Kahneman, 1974). Confirmation bias states that people tend to seek out evidence that 

confirms their current position. Availability bias states that the ease with which an item 

can be brought out of memory will influence the value assigned to the memory. These 
84

biases may affect the recollection of previously encountered situations, the comparison 

of the mental model to the previously encountered situations, or the update of the mental 

model.

3. Incorrect Mental Simulation or Evaluation of Options. To evaluate the appropriateness of 

the different proposed actions, a mental simulation is done in which the operator runs 

through the application of the actions. The operator will probably not do an exhaustive test 

of all proposed solutions, but will choose the first acceptable option. This proximate cause

includes errors that occur during the mental simulation or evaluation of options. Cognitive 

mechanisms include:

a. Inaccurate portrayal of action. This cognitive mechanism includes incorrectly 

characterizing the action (i.e., forgetting a step of the action during the mental 

simulation) or incorrectly predicting how the action will be implemented.

b. Incorrect inclusion of alternatives. The operator may forget to include some alternatives 

that should be considered.

c. Inaccurate portrayal of the system response to the proposed action. This cognitive

mechanism manifests in the operator incorrectly predicting how the system will respond 

to the proposed action.

d. Misinterpretation of procedures. Response planning within the NPP is done by 

consulting procedures. An error may occur because either the incorrect procedure

selection or inaccurate interpretation or the procedures have complicated logic making 

them difficult to use and understand.

e. Cognitive biases. The cognitive biases of overconfidence and the anchoring effect may 

be especially prevalent for this cognitive mechanism. Overconfidence affects the 

operator’s confidence in the ability of an action to work. Especially if the operator has 

had previous success with an action, he or she may be overconfident in its ability to work 

in the present case. The anchoring effect states that people are biased toward the first 

option they see or the first judgment they make. Therefore, an operator may be biased 

toward choosing the first action that occurs to him or her, and apply an unsuitable action.

We further identified relevant PIFs for each of these cognitive mechanisms. The figures shown 

in Appendix B.3 depict the PIFs that the project team identified and how they connect to the 

proximate causes and cognitive mechanisms. The diagrams in Appendix B.3 serve as graphic 

summary of the information in the tables of Appendix A.3.

5.6 Summary

This chapter introduced the macrocognitive function of decisionmaking. As defined in this 

chapter, the process of decisionmaking proceeds after the establishment of the mental model 

discussed in the previous chapter on Understanding and Sensemaking. Yet, this chapter also 

shows the cyclical nature of the two processes Decisionmaking and Sensemaking. The chapter 

presented a historical account outlining the movement of the field and evolution of the research 

within decisionmaking. The categorization scheme of proximate causes closely resembles 

models in NDM. The specific cognitive mechanisms outlined are particularly relevant for 

decisionmaking within the main control room of an NPP during normal or abnormal operations.

Note that the literature review and the proximate causes identified did not address several 

aspects of decisionmaking (team decisionmaking, distributed decisionmaking, and dynamic 

decisionmaking) because most control-room decisions are routine and directed by procedures. 

However, in situations of severe NPP accidents, the decisionmaking process could be very 

different from the current understanding of normal and event control room operations. This 

literature review does not address specific decisionmaking considerations in those situations. 
85

6. ACTION

6.1 Introduction

This section addresses errors that can arise in executing physical control actions to achieve a 

particular goal. From a psychological perspective, such actions are goal-directed motor 

interactions with the system.

Action errors are often classified as observable errors in human behavior. For example, an 

early human reliability analysis (HRA) method, the Technique for Human Error Rate Prediction 

(THERP) (NUREG/CR-1278; Swain & Guttman, 1983), lists the following human errors:

• errors of omission

o omits entire task

o omits a step in a task

• errors of commission

o selection errors

o error of sequence

o time error

o qualitative error

In this context, errors can be the result of failures of any of the cognitive functions associated 

with detecting stimuli, understanding, planning, or action execution. This chapter deals with the 

cognitive mechanisms that underlie the failures of attempted execution of a decided upon action 

plan. These errors are commonly referred to as execution failures or errors of execution 

(Reason, 1990).

Note that errors of execution, where the individual(s) intended to take the correct action but 

failed in execution, are distinctly different from instances where the individual(s) had an incorrect 

intention due to a failure in Detecting and Noticing, Understanding and Sensemaking, or 

Decisionmaking. Errors of intention are commonly referred to as mistakes (Reason, 1990); 

mistakes due to failures in one of the other macrocognitive functions are covered in the 

respective macrocognitive function.

Nuclear power plant (NPP) operators take physical actions on the plant to achieve a particular 

goal. Physical control actions are performed both by operators within the control room as well 

as by field operators in the plant itself. In the information, decision, action (IDA) model (Smidts, 

Shen, & Mosleh, 1997), “action” is defined as “implementing the decided response.” This 

definition specified whether the response is on the level of executing an entire procedure, 

executing a procedure step, or executing a single manual action (such as opening a valve). A 

large portion of the cognitive activities involved in executing a procedure (or procedure steps) 

include detecting appropriate cues, understanding the situation given those cues and the mental 

model of the process, and deciding upon the correct response (i.e., the macrocognitive 

functions reviewed above). For the purposes of this chapter, action is defined as implementing 

an action on the level of a single manual action (such as operating a valve) or a predetermined 

sequence of manual actions. The action(s) must involve the manipulation of the human-system 

interfaces of the plant and would consequently alter plant status. Therefore, physical activities 

in communication (e.g., giving commands) and collecting information about the plant status are 

not within the scope of this chapter. Errors in those activities are dealt with in other 

macrocognitive functions.
86

NPP operator actions can take different forms:

• Simple actions – One or a few action steps without much mental effort (e.g., turning a 

switch to a particular position; turning a pump on or off).

• Sequence of actions – A sequence of multiple steps of single, discrete actions to 

achieve a particular goal. An example of a sequence of actions is realigning a set of 

valves to change a flow path.

• Control actions – Sustained control of the action human-system interface (HSI) is 

needed to complete an action. This requires operators to continuously monitor the 

feedback from the interface and possibly to adjust action steps. For example, operators 

make continuous adjustments to maintain a parameter within a specified set of limits. 

Another example is when operators take manual control of auxiliary feedwater to 

maintain steam generator level between a specified upper and lower bound. In those 

cases the operators are engaged in continuous manual control that requires close 

monitoring process parameters. Continuous manual control in NPPs can be particularly 

challenging because of the complex process dynamics that are involved.

• Long-lasting actions – Actions that take many hours or days to achieve an intended goal, 

such as the actions in NPP shutdown operations or some ex-control actions. Performing 

steps or sequences of steps of the action require monitoring and evaluating plant status 

to ensure that the conditions or requirements for the steps are met.

• Non-procedural actions – Actions that operators do not have procedures or haven’t been 

trained to perform. While this is rare in internal, at-power events, they are anticipated in 

many ex-control room actions and severe accidents beyond emergency-operating 

procedures. 

We focus the literature review on the cognitive mechanisms of the first three types of actions 

because they are pertinent to NPP internal, at-power events. Although the mechanisms are 

generic to humans and are, therefore, applicable to all kinds of human actions, the long-lasting 

and non-procedural actions involve additional cognitive mechanisms that are not covered in this 

chapter. 

6.2 Cognitive Mechanisms of Action Failure

This section provides an overview of cognitive mechanisms of action execution and the 

associated limitations and vulnerabilities. Appendix A.5 presents in detail the specific 

mechanisms for Action failure as well as the links between the proximate causes, underlying

cognitive mechanisms, and relevant performance influencing factors (PIFs).

6.2.1 Overview of Psychological Causes of Action Errors

Reason (1990) presented a detailed psychological analysis of the psychological causes of 

errors of execution. He divided errors of execution into two major forms: slips and lapses. 

According to Reason (1990), “Slips and lapses are errors which result from some failure in the 

execution and/or storage of an action sequence, regardless of whether or not the plan which 

guided them was adequate to achieve its objective.”

Slips are errors where actions executed are “not as planned.” They typically occur in largely 

automatic tasks performed by a skilled individual in familiar conditions, especially when attention 

is diverted (e.g., because of distraction or preoccupation).

A “capture error” is a common type of slip where a more frequently performed behavior 

“takes-over” when a similar but less familiar action is intended. Capture errors are most likely to 
87

occur when the intended action (or action sequence) involves a slight departure from the more 

routinely performed action, and the action sequence is relatively automated and, therefore, not 

monitored closely by attention. A paradigmatic example is the case where a person intends to 

go to the store and the route begins the same way as her routine drive to work, but deviates at a 

later point. In those cases, automatic processes may inappropriately override intentional, 

controlled processes, and the person may find herself driving all the way to the work site rather 

than to the store.

A closely related psychological cause of errors of execution is negative transfer. This occurs 

when people are highly trained on a task that requires a particular response action and then are 

required to perform a different action response under very similar conditions. A paradigmatic 

example is learning to drive on the right side of the road and then having to drive in a different 

country where you are required to drive on the left side of the road.

Situational factors that can contribute to slips are instances that violate stimulus—response 

compatibility principles. Wickens and Hollands (2000) describe a number of important 

dimensions of stimulus response compatibility including:

• Location compatibility. This involves the mapping between physical location of a control and 

the location of the corresponding display of the value being controlled. Humans have an 

intrinsic tendency to move toward the source of stimulation. Controls located next to the 

relevant display will be less error prone. If the controls and displays (e.g., corresponding 

dials or meters) are organized in a non-intuitive manner that violates location compatibility 

(for example, if two controls are spatially crossed with their corresponding displays), errors 

are more likely to occur.

• Movement compatibility. This involves the mapping between the direction of movement of 

the control and the corresponding value being controlled. When an operator moves a 

position switch, rotary, or sliding control, movement compatibility defines the set of 

expectancies that an operator will have about how the display will respond to the control 

action. For example, an operator would expect that moving a switch up will cause the 

corresponding display value to go up. Violations of movement compatibility are more likely 

to result in error.

• Population stereotypes. This involves whether mappings are consistent with expectations 

based on experience and conventions. Mappings that are inconsistent with population 

stereotypes (e.g., if red is used for “go” and green for “stop”) will be more error prone.

• Modality compatibility. This involves mapping the modality of the stimulus and modality of 

the response required. If the stimulus is in the auditory modality, performance is better 

when a vocal response is required than a manual response. In contrast, when the stimulus 

is visual, then the response is faster and more accurate if the required response involves 

manual pointing than a vocal response.

Another important psychological cause of execution errors is memory lapses. Lapses involve 

failures of memory, where an individual may forget to perform an intended action. This is often 

due to a failure in prospective memory—memory for intended actions in the future. Examples of 

lapses include instances where operators may forget to perform a specific step, may lose their 

place in a procedure, or may forget to perform an entire sequence of steps. Lapses most often 

arise when interrupted in the process of performing a task.

A common, well-studied type of memory lapse is a “post-completion” error, where an individual 

terminates a task prematurely and forgets to perform the final step(s) that occurs after the main 

goal of a task has been completed. A classic example of this is forgetting to take out the 

original from a copier, once the copies have been made. Post completion errors are a particular 
88

problem in maintenance tasks, where personnel may omit one or more steps involved in 

reassembling a piece of equipment and placing it back in service, once the main objective of the 

maintenance task has been accomplished. Post-completion errors have been well studied by 

psychologists and a number of theoretical models have been developed to explain the 

psychological mechanisms that produce them (e.g., Altmann & Trafton, 2002; Byrne & Bovair, 

1997).

Mode errors are a form of error that shares elements of slips as well as lapses. They refer to 

situations where an operator loses track of what mode a system is in, where a given action will 

have different consequences in different modes of the system. An example is pressing the 

accelerator of a car to move forward when the transmission is in the “reverse” mode. Mode 

errors are a well-recognized problem of major concern in automated cockpits, which have 

various modes of autopilot control (Sarter & Woods, 1995). They can arise in NPP control 

rooms as well. For example, instances where actions will be blocked from having their intended 

effect in certain system modes due to complex interlocks. If the displays do not effectively 

communicate the mode of the system and the fact that there are interlocks preventing the action 

from having its intended consequence, then the operator may not realize that the intent of his or 

her action has not been accomplished.

A final psychological cause of execution errors relates to problems in manual control of 

continuous processes. The term “manual control deficiencies” has been used to refer to 

psychological causes of problems in manual control of a continuous process. In NPP, operators 

are often required to control dynamic systems to make them conform within certain time-space 

trajectories (e.g., increasing or decreasing steam generator level to a particular target value at a 

certain rate).

Morray (1997) provides an overview of the human performance issues associated with manual 

process control. Wickens and Hollands (2000) provide a concise summary of the psychological 

literature on manual control tasks and the factors that contribute to manual control deficiencies. 

They point out that there are three situational factors that can contribute to the challenges 

associated with a manual control task. These are: 

• The dynamics of the system itself—how it responds in time to the forces applied (i.e., 

whether there are response lags or shrinks and swells that complicate the ability to control 

the parameter) 

• The process change that is required (i.e., the desired trajectory of the system) 

• The display of information concerning the desired and actual state of the system (i.e., 

whether the information required to control the process is available to the operator and how 

the information is displayed).

Wickens and Hollands (2000) summarize the human information processing limitations that 

influence the likelihood of manual control deficiencies. These include processing time, 

information transmission rate, predictive capabilities, processing resources, and compatibility. 

Problems in manual control can lead to system instability, with excessive oscillations 

(overshooting and undershooting target values and trajectories) with the result that critical 

parameter limits may be inadvertently exceeded (e.g., steam generator (SG) level may exceed 

reactor trip set points).
89

6.2.2 Overview of Action Execution in the Brain

Neurophysiological Substrate for Action

To produce planned, goal-directed movements, a cognitive system must be capable of running 

mental processes, which virtually simulate action sequences aimed at achieving a goal. The 

mental process either attempts to find a feasible course of action compatible with a number of 

constraints (internal, environmental, task specific, etc.) or selects it from a repertoire of 

previously learned actions, according to the parameters of the task. Multiple brain cortical 

regions are involved. These regions are connected in a network. Figure 6-1 outlines the major 

pathways in the network: the hierarchy pathway, automaticity, and feedback.

Figure 6-1. Pathways for motor execution.

Hierarchy Pathway. The hierarchy pathway involves movement programming, storing, and 

sequencing, and movement execution. Programming an action into a single movement or a 

series of movement commands is done mainly in the forward portion of the frontal lobe. This 

part of the cortex receives information about the movement goals and decides which set of 

motor organs (e.g., body, hands, or head) to be involved and how to achieve the required 

movements. Often an action consists of a sequence of movements in a given order and timing. 

The cortical network first programs the movement commands and then stores the commands in 

the network via working memory (in supplementary movement field of the frontal lobe), and 

executes the commands in the programmed order and timing. Movement commands are 

processed in the motor cortex, which sends signals to motor neurons to activate specific motor 

organs for movement execution.

Automaticity Pathway. Action automaticity is the ability to implement actions without occupying 

the brain with the low-level details required, allowing it to become an automatic response 

pattern. It is usually the result of learning, repetition, and practice. The sequence of actions 

appropriate to solve a problem often must be discovered by trial and error and recalled in the 

future when faced with the same problem. Many routine tasks are performed almost 

automatically, but such actions may become invalid if the environment changes, at which point 

individuals need to switch behavior by overcoming automatic actions that are otherwise 

triggered automatically. Such behavioral switching can occur either retroactively based on error

feedback or proactively by detecting a contextual cue.

Sensory Feedback. Human goal-directed behavior depends on multiple neural systems that 

monitor and correct for different types of errors. To ensure that all of these movements are 

precise and coordinated, the nervous system must constantly receive sensory information from 
90

the outside world and use this information to adjust and correct the movements. According to 

closed-loop accounts of motor control, movement errors are detected by comparing sensory 

feedback to an acquired reference state. Differences between the reference state and the 

movement-produced feedback results in an error signal that serves as a basis for a correction.

Capacity Limits of the Neural Substrates that Perform Action Execution 

The following are some capacity limits in the brain network for action implementation reported in 

the literature:

• Action Programming. The cortical areas for programming action (frontal motor field and 

lateral intraparietal cortex) can only program one action at a time (Andersen & Buneo, 

2002). Therefore, if a task requires simultaneous action goals, the action programming can 

interfere with each other, resulting in loss of one action, incomplete action programming 

(e.g., missing a movement step or making the wrong movement order), or transposing 

movement steps in two action sequences.

• Cost of Switching. Complex jobs require frequent shifts between cognitive tasks—a change 

of task by requiring subjects to switch frequently among a small set of actions. A 

performance switch cost is observed such that switching to a new task results in a slower 

and more error-prone execution of the actions. The actions are substantially slower and, 

usually, more error-prone immediately after a task switch. Resolution and protection from 

interference by previous actions explain part of the switching cost. The aspects of the task

set, including task variations, task-set overlap, and task-set structure, and modalities of the 

actions are related to action error rates caused by task switching.

• Coarse Motor Ending. Movement commands for an action generated in the action 

programming areas coarsely encode the direction and amplitude of the movement. Precise 

programming of the movement commands is achieved by continuous sensorimotor 

integration.

• Delayed Actions and Action Sequencing. If an action is programmed but is not executed 

immediately, the commands for such delayed actions or action sequences are maintained 

through working memory. Therefore, delayed actions and action sequences are subject to 

all of the capacity limits of working memory, such as the number of items that can be stored, 

the duration of the action commands can be maintained without attention reinforcement, and 

the vulnerability to disruptions. Errors of omission typically occur as the result of excessive 

demands on working memory.

• Automaticity. Although developing automaticity greatly reduces the demands for the brain 

action network to program detailed movements, the automaticity is limited to the scope of 

the learning and training environment. Such actions become invalid when the environment 

is changed.

Modulators in the Neural Substrate for Action Execution

Reaching or exceeding the capacity limits of cognitive processes can result in errors. However, 

in working with complex control systems, operational personnel often have to perform tasks that 

demand cognitive resources exceeding their capacity limits, yet they still need to perform the 

tasks reliably. Fortunately, the human brain has many mechanisms (referred to as modulators) 

that allow human information processing to cope with the limits. Moreover, humans develop 

and intentionally use various strategies to reliably perform complex tasks. Hence, human errors 

occur when the task demands reach or exceed some cognitive limits and the corresponding 

modulators or coping strategies are not effective.
91

The brain has many modulating processes to cope with the limits in the hierarchy action 

pathway. The following are some modulators reported in the literature:

Executive Control. Unlike a purely reactive system where the motor output is exclusively 

controlled by the actual sensory input, a cognitive system must be capable of running mental 

processes that virtually simulate action sequences aimed at achieving a goal. The lateral 

prefrontal cortex is critically involved in broad aspects of executive behavioral control. Neurons 

in this area take part in selecting attention for action and in selecting an intended action.

Furthermore, the lateral prefrontal cortex is involved in the implementation of behavioral rules 

and in setting multiple behavioral goals. This area is responsible for strategic planning of 

macrostructures of event-action sequences.

Switching Control. There are control processes that reconfigure mental resources for task 

switching. This “switch cost” is reduced, but not eliminated by an opportunity for preparation. 

Advance preparation reduces task-switching cost through a preparatory control mechanism. 

Advance task preparation can reduce the task error rate to rates seen in non-switch trials.

Sensory Inputs in Action Programming. Errors of commission often occur because of failures to 

detect stimulus deviance. Precise and continuous sensory inputs make adjustments to motor 

functions to enhance action correctness and accuracy. Also, movement programming has been 

shown to be optimized when the participant is permitted to see his or her hand resting on the 

starting base prior to movement initiation.

Error-Monitoring and Correction. Goal-directed actions depend on multiple neural systems that 

monitor and correct for different types of errors, especially errors in delayed or sequences of 

actions. The frontal error system assesses high-level errors (i.e., goal attainment), whereas the 

posterior error system is responsible for evaluating low-level errors (i.e., trajectory deviations 

during motor control). The posterior medial frontal cortex (pMFC) is assumed to monitor 

performance problems and to interact with other brain areas that implement the necessary 

adaptations such as attentional focusing. Upon the occurrence of errors, the pMFC selectively 

interacts with perceptual and motor regions and thereby drives attentional focusing toward 

task-relevant information and suppresses the perceptual activities encoding action-irrelevant 

stimulus features.

Automaticity Control. The frontal cortical areas play executive roles in automaticity switching. 

The anterior cingulate cortex acts retroactively and the pre-supplementary motor area acts 

proactively to enable switching. The lateral prefrontal cortex reconfigures cognitive processes 

constituting the switched behavior. The subthalamic nucleus and the striatum in the basal 

ganglia mediate these cortical signals to achieve behavioral switching. The network enables

switching by first suppressing an automatic unwanted action and then boosting a controlled 

desired action.

The Effects of PIFs on Modulators – How the Brain Fails to Reliably Execute 

Goal-Directed Actions

Although the capacity limits in the hierarchy pathway of action implementation are generally 

stabilized in the normal adult brain, the effectiveness and efficiency of the modulator are subject 

to an individual’s intention, experience and strategies of activating the modulators, and 

individual factors such as fatigue, stress, and drug/alcohol abuse. This is particularly true for 

the control mechanisms. The modulators are also affected by environmental constraints and an 

individuals’ mental condition. Cognitive and neurophysiologic studies have demonstrated many 

instances where the effectiveness of modulators was impaired by various factors (i.e., the PIFs). 

Below are some examples:

• Sleep deprivation impairs error detection and error correction (Hsieh, Tsai, & Tsai, 2009).
92

• Anxiety impairs error monitoring and correction (Aarts & Pourtois, 2010).

• Higher stress levels are associated with errors of commission (Helton, Head, & Kemp, 

2011).

• Using memory aids can reduce the likelihood of errors of omission. According to Reason 

(2002), conspicuity, contiguity, content, context, and countability of the memory aids are 

important in reducing omission errors.

6.2.3 Summary of Cognitive Mechanisms Underlying Action Failures

Many cognitive mechanisms for action implementation have been revealed in 

neurophysiological and psychological studies. We summarize the mechanisms into three 

processes: control selection, cognitive aspects of action control, and motor or physiological 

aspects of manual execution. The table in Appendix A.5 presents detailed descriptions of the 

mechanisms grouped into the three processes. Here we briefly describe each process and the 

related mechanisms.

Control Selection. Many errors in execution are associated with unintentional failure of initiating 

the action or initiating the wrong action. The underlying neurophysiological mechanisms of 

control selection include the effectiveness of monitoring and evaluating parameters required to 

initiate the action and the use of feedback to detect and correct the erroneous outcomes of 

action steps. 

• principles of response selection and compatibility (Proctor & Van Zandt, 2008; Wickens & 

Hollands, 2000)

• population stereotypes in control coding (Swain & Guttman, 1983)

• error monitoring and correction (Wickens, Lee, Liu, & Becker, 2004).

Cognitive Aspects of Manual Execution. The execution of an action plan can fail due to many 

cognitive factors. The underlying mechanisms are described in the cognitive limits of “delayed 

actions and action sequencing.” In brief, maintaining an action plan requires working memory 

and attention, which are vulnerable to their resource limits and interference. 

• interference (Kiesel et al., 2010; Reason, 2008)

• memory limitations (Baddeley, 1992; Dodhia & Dismukes, 2009; Reason, 2008)

• attention (Proctor & Van Zandt, 2008; Wickens & Hollands, 2000)

• stimulus response compatibility (Proctor & Van Zandt, 2008; Wickens & Hollands, 2000)

• error monitoring and correction (Wickens, et al., 2004).

Motor/Physiological Aspects of Manual Execution. An action plan can fail to be executed 

correctly due to many physiological factors and factors related to motor functions. The 

underlying neurophysiological mechanisms are described in the cognitive limit of “coarse motor 

encoding.” That is, motor commands for an action generated by the action control mechanism 

only coarsely encode the direction and amplitude of the movement; precise motor execution is 

achieved by continuous sensorimotor integration that requires sensory feedback. In addition, 

the motor execution system can become automatic through training and learning; the 

autonomous movements occur without action control. Yet, such automaticity can introduce 

execution errors, as described in the cognitive limit of “Automaticity.” The psychological factors 

affecting the motor and physiological aspects of manual execution are listed below:

• automaticity control

• negative transfer and habit intrusion (Reason, 2008)
93

• population stereotypes in operation of controls (Swain & Guttman, 1983)

• motor learning (Schmidt & Bjork, 1992)

• stimulus response compatibility (Proctor & Van Zandt, 2008; Wickens & Hollands, 2000)

• error monitoring and correction (Hsieh, et al., 2009; Wickens, et al., 2004)

• manual control deficiencies (Wickens, et al., 2004).

6.3 Identification of Proximate Causes

Given that the two major phases of action execution are initiating the intended action and 

executing the action, we identified the following two proximate causes for Action failure: 

• Failed to take planned action. These are errors of omission. Only those omissions 

associated with action execution are considered. For example, failing to take an action 

because of a failure in decisionmaking would be an omission that is dealt with in Chapter 5 

of this review.

• Executed the planned action incorrectly. This proximate cause includes errors in the 

outcomes of the action, such as selecting the wrong control or turning a switch to the wrong 

position. Actions that are carried out too slowly, too quickly, too soon, or too late are also 

captured by this proximate cause. It is important to note that manual execution of most 

actions in the control room and in the plant do not require precise (approximately seconds or 

milliseconds) timing.

Most existing HRA methods have accumulated state-of-practice information that classified 

action errors at an objective, observable level, such as the manual control failure modes from 

THERP (NUREG/CR-1278; Swain & Guttman, 1983) and the action error classification scheme 

developed as part of the ongoing U.S. Nuclear Regulatory Commission (NRC) HRA simulator 

data collection effort (Roth, Chang, & Richards, 2011). The proximate causes identified in this 

report should correspond and cover those error types in HRA methods. We reviewed those 

error taxonomies to better comprehend and articulate the proximate causes.

Action Failure Modes in THERP

The following list of failure modes is taken from the manual control section (Chapter 13) in 

THERP (NUREG/CR-1278; Swain & Guttman, 1983). As discussed in Section 6.1, it is often 

unclear which macrocognitive function failed to lead to an action error. For example, an 

operator could press the wrong button because he misdiagnosed a problem or because he got it 

confused with another button. This list of failure modes is appropriate for capturing the type of 

failures that would occur in the action execution:

• selection of wrong control

• incorrect operation of the control

• inadvertent operation of control by unintentional contact.

Action Error Classification in the HRA Simulator Data Collection scheme

The NRC developed an error classification scheme to categorize operator errors from simulator 

exercises. The scheme is referred to as Scenario Authorization, Classification, Analysis, and 

Debriefing Applications (SACADA). The scheme classifies action errors into the following types: 

• Failed to take required action (did not attempt action)

• Incorrect timing
94

a. Delayed in initiating action

b. Performed action too slowly (behind the event)

c. Initiated action prematurely

d. Other

• Executed desired action incorrectly

a. Omitted one or more steps

b. Incorrect order of steps

c. Incorrect position (e.g., turn switch to wrong position)

d. Action prevented because of interlock

e. Manual control problem (e.g., overshot, undershot)

f. Other

• Executed undesired action (from perspective of what plant needs or requires)

a. Blocked a needed function from initiation (e.g., an engineered safety system)

b. Stopped or turned off a needed function (e.g., an engineered safety system)

c. Unnecessary initiation of a function (e.g., manual trip)

d. Other

All the above error types can be classified into one of the two proximate causes.

6.4 Summary

This chapter reviewed the cognitive mechanisms underlying action execution and identified two 

proximate causes for failure of the Action function. The chapter also briefly described how 

some PIFs may affect the cognitive mechanisms. We further mapped the mechanisms to one 

of the two proximate causes (note that in some cases a single cognitive mechanism could serve 

as an explanation for both of the proximate causes). We also inferred links between PIFs and 

the cognitive mechanisms. Appendix A.5 describes the details of these mechanisms and how 

they relate to the proximate causes and PIFs.

Notice that the scope of literature review in this chapter is limited to cognitive mechanisms and 

PIFs related to simple, sequences, or control actions in NPP internal, at-power events in which 

operators are well-trained to perform the actions with detailed procedures. We did not review 

the literature on additional mechanisms and PIFs relevant to long-lasting or non-procedural 

actions, or those that operators were not trained for. Such actions may require cognitive 

mechanisms in addition to the ones reviewed in this chapter. 
95

7. TEAMWORK

7.1 Introduction

The vast majority of the technology used to control critical and complex systems, such as 

nuclear power plants (NPPs), requires teams or groups because the cognitive (and physical) 

requirements for operating these systems far exceed the abilities of a single person. As such, it

is important to understand the cognitive basis of teamwork in human reliability analysis (HRA). 

Teamwork is the macrocognitive function that focuses on how people interact with each other to 

coordinate as the individual or crew works on a task. In this chapter we discuss how teamwork 

functions in NPPs and the unique features of team macrocognition as an emergent process. 

We then review major models and theories of teamwork, followed by a description of the key 

components of teamwork in NPPs, and identification of the cognitive mechanisms and 

proximate causes of failure of the Teamwork function. We focus the review on research findings 

that explain why Teamwork can fail (and succeed), and research that models aspects of 

performance that are unique to teams as they work to accomplish a common task or goal. 

7.1.1 Teamwork in NPP Control Rooms

Salas, Dickinson, Converse, and Tannenbaum (1992), defined a team as “a distinguishable set 

of two or more people who interact dynamically, interdependently, and adaptively toward a 

common and valued goal/object/mission, who have each been assigned specific roles or 

functions to perform, and who have a limited life span of membership” (p. 4). Salas, Cooke, and 

Rosen (2008) elaborated on this definition by adding, “Teams are social entities composed of 

members with high-task interdependency and shared and valued common goals (Dyer, 1984).

They are usually organized hierarchically and sometimes dispersed geographically; they must 

integrate, synthesize, and share information; and they need to coordinate and cooperate as task 

demands shift throughout a performance episode to accomplish their mission” (p. 541).

A crew of operators in an NPP is a specialized case of a team, having unique roles and 

responsibilities for individual members, and a clearly defined and expected approach to the 

conduct of operations. In the United States, commercial NPPs are required to have a minimum 

of three crewmembers on shift in the control room any time the plant is in operation: a senior 

reactor operator (SRO) and two licensed operators, one of which is assigned for the shift as the 

reactor operator (RO) and the other as the balance-of-plant operator (BOP). 7

 The standard 

operating model for NPP crews has a hierarchical command structure (i.e., the SRO is in 

charge), and a clear division of primary responsibilities. The SRO is responsible for directing 

the operation of the nuclear reactor, and primarily directs the RO and BOP to manipulate the 

controls of an NPP. The RO manages the controls for the reactor core, and the BOP manages 

the balance of plant controls. The SRO is also responsible for analyzing complex plant 

conditions and evolutions to provide proper operations focus and ensure error-free 

performance, investigating off-normal system/plant status to diagnose and correct problems, 

and ensuring the completion of assigned activities.

The successful operation of an NPP requires effective Teamwork. For instance, crew members 

play an important ‘back-up’ role for each other—they both formally (formal peer checks) and 

informally assist in catching and recovering from errors made by others. Team situation 

awareness is fundamental to these roles. NPP crew members also actively contribute to 

decisionmaking. For example, in a properly functioning crew there are ‘hold-points’ before

 

7. The three-operator crew model is used only as a simplified example for purposes of discussion. Most U.S. plants 

employ more than the minimum three-person crew for an individual control room as defined by 10 CFR 50.54(m). 

Typically, in addition to the three operators, there is a Shift Supervisor (SS) and a Shift Technical Advisor (STA).
96

major decision points (e.g., when transitioning to a new procedure; or about to take major 

action) where the supervisor announces his/her current understanding of the situation and what 

he/she has decided and solicits agreement/objection. Empirical investigations have also 

illustrated the importance of Teamwork in NPP operations. A study by Roth, Mumaw, and

Lewis (1994), followed up with another study by Lang, Roth, Bladh, and Hines (2002), showed 

that the NPP crew’s cognitive activities, which are the basis for team performance, are strongly 

related to successful NPP safety performance, particularly for emergency situations. 

7.1.2 The Emergent Aspect of Teams

In a team setting, individual macrocognitive functions like Detecting and Noticing and 

Understanding and Sensemaking are often occurring in parallel by all team members. For 

example, in cases such as an alarm annunciation in an NPP control room, it is highly likely that 

all three crewmembers (e.g., the SRO, RO, and BOP) will engage, independently and in 

parallel, in the Detecting and Noticing and Understanding and Sensemaking functions.8 The 

combining of individual macrocognitive processes that are occurring in parallel is the emergent 

aspect of teams. For teams to function in a coordinated fashion, the parallel processes must be 

merged and some emergent “group process” must ultimately occur that determines how the 

group proceeds. The merging of these parallel processes provides additional chances for 

human error and opportunities for recovery as team members interact and coordinate activities. 

A number of researchers have recognized this emergent aspect of team cognitive processing. 

Roth, Multer, and Raslear (2006) noted that, “An area of growing consensus in the literature on 

teamwork is the importance of shared contextual knowledge in supporting coordination and 

facilitating work. Effective performance depends on shared information about both the situation 

and the other team members. This includes mutual knowledge and beliefs about the current 

situation, each other’s goals, and current and future activities and intentions. Various labels 

have been used to denote this shared contextual knowledge, including shared mental models 

(Cannon-Bowers, Salas, & Converse, 1993); team cognition (Espinosa, Lerch, & Kraut, 2004), 

common ground (Clark & Brennan, 1991; Klein, Armstrong, Woods, Gokulachandra, & Klein, 

2000; Klein, Feltovich, & Woods, 2005); shared situation awareness (Endsley, Bolté, & Jones, 

2003), and shared work space awareness (Gutwin & Greenberg, 2004)” (p. 968).

This chapter’s general focus is on the aspects of team performance that emerge because

individuals are working together toward a common goal. The way this emergent aspect is 

distinguished from the other macrocognitive functions is by describing how and when the 

emergent team aspects of the Teamwork function become manifest within the continuously 

iterating evolutions of the other macrocognitive processes.

7.2 Models and Theories on Teamwork

7.2.1 Research on NPP Crews

Our review identified three different groups of researchers studying NPP crew behaviors, 

performance, and/or coordination. Roth (1997) and Vicente, Roth, and Mumaw (2001)

performed field studies of NPP crews, and showed that operators often work outside of their 

narrowly prescribed roles as a way to try to improve their holistic understanding of the situation 

and improve their overall performance. According to this research, there may be cases where a 

 

8. An exception to this parallel processing may be when a particular teammate is explicitly directed to engage in a 

different cognitive activity or task, given some predefined or organizationally mandated roles, responsibilities, or 

authorities (e.g., conduct of operations). 
97

crewmember made a decision or took an action that was not strictly identified as being within his 

or her purview, but did so because he or she was attempting to: 

• Enhance information extraction by increasing the salience of important indicators and 

reducing the background “noise.” 

• Create new information.

• Offload some of the cognitive processing onto the interface (e.g., creating external aids and 

reminders for monitoring).

Research by Massaiu, Hildebrandt, and Bone (2011) on NPP crews in a simulator at the Halden 

Reactor Project produced complimentary findings to Roth (1997) and Vicente, Roth, and 

Mumaw (2001). From their research, they developed the Guidance-Expertise Model (GEM) of

NPP control room crews in emergency response situations. GEM specifies that crews use two 

cognitive control modes during emergencies: 

• Narrowing (where the crew is focused on the steps in the emergency procedure) 

• Holistic (self-initiated cognitive activities like additional monitoring and developing team 

situation awareness (SA)).

GEM also posits that the control modes are affected by external PIFs, such as the quality of the 

emergency procedures, and internal PIFs, such as the quality of the crew’s teamwork. 

Interestingly, the authors also reveal that the outcome behaviors in the GEM model are not 

errors. Rather, they are generic types of crew activity that typically impact the performance of 

tasks.

Carvalho, Vidal, and de Carvalho (2007) and Carvalho, dos Santos, and Vidal (2006) also 

performed field studies of NPP crews, and showed that communication, shared cognition, and 

plant culture can affect crew coordination, and ultimately plant safety. Their analyses showed 

that communication was key in developing a shared understanding of the situation among the 

crewmembers (i.e., shared cognition), such that the correct decisions and courses of action 

could be taken. Perhaps more importantly, they showed that this emergent process is messy, in 

that:

• The communication process can iterate numerous times.

• It is not always possible to predict how many iterations it will take for a shared 

understanding to be created.

• Numerous cultural factors, including the leadership style of the SRO and the plant’s safety 

culture can affect communication and the process of forming a shared understanding, which 

other research (i.e., Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000) has shown 

is key to effective Teamwork.

The findings of these three groups of researchers make it clear that people are remarkably 

adaptable and do not always strictly follow the rules. This phenomenon persists despite 

intensive training, and despite the fact that the conduct of operations has been continually 

updated to address issues related to Teamwork based on years of operational experience. 

Reason (2008) and Hollnagel (2009) have both pointed out that this phenomenon can be both a 

detriment and a benefit to safety, depending on the circumstances. For example, in the 

behaviors described above by Roth (1997) and Vicente, Roth, and Mumaw (2001), it is possible 

that the crews’ coping strategies enhance performance as intended, or they may unintentionally 

cause an error. Crews may make errors of omission or commission when employing strategies 

to reduce background “noise”, creating new information, or offloading cognitive information to an 

external aid. In the context of GEM, crews may inappropriately engage in the “narrowing”
98

cognitive control mode because of poor quality emergency procedures, which may then lead to 

an error of omission, such as failing to detect new information being presented on an auxiliary 

display. Crews may also inappropriately engage in the “holistic” cognitive control mode 

because of an unfamiliar upset condition, and consequently make an error of omission, such as 

failing to execute a key step in the emergency procedure. The research by Carvalho and his 

colleagues suggest that a poor safety culture and time pressure can lead to truncated 

communication among the NPP crew, resulting in an incorrect shared understanding of the 

situation that leads to errors in decisionmaking and/or action.

7.2.2 Team Errors: Definition and Taxonomy by Sasou and Reason (1999)

Traditionally, literature related to error analysis or error management has focused on individuals 

making errors. Yet in large complex systems most people work in teams or groups. Sasou and 

Reason (1999) have attempted to address this gap by developing a taxonomy to characterize 

different types of team errors. The error taxonomy has two main parts. The first part defines 

four error types: independent individual errors, dependent individual errors, independent shared 

errors, and dependent shared errors. Individual errors are made by a single team member 

without involvement from any other member of the team. Shared errors are errors that are 

shared by some or all of the team members, regardless of whether or not they were in direct 

communication. Individual and shared errors are further subdivided into two categories: 

independent and dependent. Independent errors occur when all available information is correct, 

whereas dependent errors occur when some part of the information is inappropriate, absent, or 

incorrect. 

The second part of the taxonomy defines the error recovery process. The three error recovery 

processes are: failure to detect, failure to indicate, and failure to correct. The first step in 

recovering errors is to detect their occurrence. If the team does not notice the error, they will 

have no chance to correct them. Actions based on those errors will be executed. Once 

detected, the recovery of an error will depend upon whether team members bring it to the 

attention of the remainder of the team. This is the second barrier to team error making. An 

error that is detected but not indicated will not necessarily be recovered and the actions based 

on those errors are likely to be executed. The last barrier is the actual correction of errors. 

Even if the remainder of the team notices and indicates the errors, the people who made the 

errors may not change their minds. If they do not correct the errors, the actions based on those 

errors will go unchecked.

Sasou and Reason tested the utility of their definitions and error taxonomy by reviewing events 

that occurred in the nuclear power industry, aviation industry, and shipping industry. Their 

review also included an analysis that determined the relationship between the team errors they 

defined and PIFs. The relationship between their error taxonomy and PIFs (referred to as 

performance shaping factors (PSFs)) is shown in Figure 7-1.

The key information to take away from this research is that they identified many PIFs that can 

influence failure of the Teamwork macrocognitive function. The SRO mismanaging the time to 

engage in a course of action, leading to the crew experiencing time pressure is an example of 

deficiencies in resource/task management. The PIF excessive professional courtesy can 

contribute to the appearance of unanimous agreement in the social phenomenon of groupthink, 

and other social pressures like “not rocking the boat” to get along. These social pressures can

lead to the failure to indicate the presence of an error and/or failure to correct an error.
99

Figure 7-1. Sasou & Reason error taxonomy with mapping to PSFs.

7.2.3 Crew Resource Management

Crew Resource Management (CRM) is a training program created for the aviation industry that 

attempts to improve crew coordination and flight deck management (Helmreich, 1999). It 

focuses on team and managerial aspects of flight operations, which are complimentary 

components to the technical, “stick and rudder” aspects of flight. As Helmreich and Foushee 

(1993) put it, “CRM includes optimizing not only the person-machine interface and the 

acquisition of timely, appropriate information, but also interpersonal activities including 

leadership, effective team formation and maintenance, problem-solving, decisionmaking, and 

maintaining situation awareness” (p. 4).

CRM is an input-process-output model, whereby inputs are antecedent characteristics of 

individuals, groups, and the organizational/operational environment that are posited to influence 

the performance of the team. The 1993 chapter by Helmreich and Foushee describes the 

model for CRM in detail, so this information is not repeated here. The key information to take 

away from the research on CRM is that many of the factors they identified as affecting aviation 

crew performance are relevant to NPP crew performance, and translate into performance 

influencing factors (PIFs) that can explain why the Teamwork function may fail. CRM identifies 

group composition, organizational culture, and regulatory requirements as “crew performance 

input factor” PIFs that can affect crew performance. As an example of group composition, if the 

crew has not logged many hours together and/or is very inexperienced, they may not 

communicate effectively, which can lead to a breakdown in Teamwork.
100

Similarly, many of the “crew and mission performance functions” in the CRM model describe the 

nature and quality of the emergent psychological mechanisms of team performance, including 

communication skills, leadership, planning, prioritization, and coordination of tasking. For 

example, if the SRO’s leadership style is highly autocratic, he may disregard contradictory 

information that the BOP or the human-system interface (HSI) is presenting, leading to an 

incorrect decision. Figure 7-2 shows the some of PIFs that have been identified by CRM.

Figure 7-2. The CRM input-process-output model of crew performance.

7.2.4 Team Sensemaking

Team sensemaking is defined as the process by which a team manages and coordinates its 

efforts to explain the current situation and to anticipate future situations, typically under 

uncertain or ambiguous conditions. Team sensemaking is very similar to individual 

sensemaking, but Klein, Wiggins, and Dominguez (2010) argue that in many ways, it is more 

critical than individual sensemaking, because team sensemaking poses additional emergent

coordination requirements and has additional ways for sensemaking to break down. They 

argue team sensemaking is more difficult to accomplish, and it may be a larger contributor to 

accidents than failures at the individual level.

Like individual sensemaking, Klein, Wiggins, and Dominguez (2010) describe team 

sensemaking as a process by which teams reconcile data with their existing frames (i.e., mental 

models of the situation). The steps for team sensemaking are:

• Identifying a frame to give meaning to the data deemed important enough to be captured.

• Questioning the appropriateness of the frame identified, given the data captured.

• Preserving and elaborating on the original frame, or re-framing by comparing frames or 

creating a new frame.

An emergent feature of team sensemaking is that the team must decide how to work together 

as they try to make sense of their situation. Individual team members are engaged in their own 

internal sensemaking process, but ultimately the team must decide on how the different 

individual sensemaking processes will be reconciled. Furthermore, the emergent team process 

of reconciling individual sensemaking applies to all steps of team sensemaking—identifying, 

questioning, and then preserving/elaborating or re-framing.
101

The general strategies in which teams work together are hierarchical, collaborative, and 

opportunistic. For example, if the team is using a hierarchical strategy when identifying a frame, 

then the team leader will decide the official conclusion. The collaborative strategy involves 

using consensus to make a team decision. The opportunistic strategy gives everyone on the 

team the same rank and responsibility for detecting problems and prescribing corrective actions. 

This reconciliation/resolution process is a unique feature of teams, because individuals will be 

going through their individual sensemaking processes in parallel, and these parallel processes 

must eventually be merged and/or narrowed down to one output. 

The researchers elaborated on this emergent concept by developing a set of “behavioral 

markers” for team sensemaking, which are specific examples of the general strategies teams 

can employ for each step of the sensemaking process, and can serve as the basis for inferring 

PIFs that could affect team sensemaking. Table 7-1 lists the behavioral markers.

Table 7-1. Team sensemaking behavioral markers (Klein, Wiggins, and Dominguez, 

2010).

Team Sensemaking Step Behavioral markers

Identifying a frame Team formulates criteria or rules used to identify the frame

A team member announces the frame 

Team collaborates to identify the frame

Questioning a frame Appoint a team member to play devil’s advocate and raise doubts 

about the suitability of the frame

Team creates rules or tripwires to alert them that the frame may 

be unsuitable

Team members voice and discuss what might go wrong using the 

current frame

Preserving and elaborating a 

frame 

Team discusses and rejects anomalous data as transient signals 

or otherwise insignificant

The team’s data synthesizers direct the activities of the data 

collectors to seek new data to verify the frame

The team’s data synthesizers and data collectors collaborate to 

discover new relationships that preserve or extend the frame

Reframing: comparing frames Team compares frames and votes for one

Team forges consensus on which frame is most appropriate

Leader announces, which frame is most appropriate

Reframing: creating a new frame Individual suggests a frame and it is adopted, modified, or rejected 

as the team compares frames

Team speculates on data and suggests causal beliefs; leader or a 

team member combines viewpoints into a frame

Team collaborates to synthesize competing frames

7.2.5 Macrocognition in Teams

Letsky and colleagues (Fiore, Smith-Jentsch, Salas, Warner, & Letsky, 2010; Letsky, 2007; 

Letsky, Warner, Fiore, & Smith, 2008) have been the main developers of the concept of team 

macrocognition (Figure 7-3). This model of team collaboration (i.e., Teamwork) was developed 

based on military research, and it identifies macrocognitive processes that are specific to the 

types of teams that are often seen in military work: asynchronous, distributed, multi-cultural, and 

hierarchical. The model includes four major team collaboration stages (knowledge construction, 

collaborative team problem solving, team consensus, and outcome evaluation and revision), 

and details the macrocognitive and metacognitive processes that relate to those stages.
102

Figure 7-3. Macrocognitive model of team collaboration (Letsky, et al., 2007).

Although this model is certainly relevant to the present research effort, it was not designed with 

NPP operations in mind. Nuclear power operations are highly proceduralized, which provide 

predefined goals and decisions dictated by the procedures. There is, for example, little 

recognition-primed decisionmaking when operations are governed by procedures. However, 

there are aspects of this model that can apply to the NPP domain. For instance, it is important 

to note that this model has strong theoretical ties to the psychological literature, and like the 

work of Klein, Wiggins, and Dominguez (2010) on team sensemaking, the model touches on the 

idea that there is an emergent aspect to Teamwork.

Fiore et al. (2010) documented that team cognition “draws on four general categories of 

research: externalized cognition, team cognition, group communication and problem solving, 

and collaborative learning and adaptation” (p. 205). Team externalized cognition is the study of 

when, how, and why a team uses tools and decision aids such as maps, procedures, or 

computers to help visualize and conceptualize complex problems, and contributes to the 

understanding of team macrocognition with respect to how teams use technology to assist in 

their coordination. The research on team cognition or shared cognition augments the model of 

team macrocognition by providing insights into how teams communicate information and 

generate a common understanding of the situation, task, and/or problem. The research on 

group communication and problem solving is the study of how team dynamics and group 

processes can affect the assumptions the team makes about the situation, task, and/or 

“problem space” (similar to the findings in groupthink). The research on collaborative learning 

and adaptation provides insights into how teams work together to create new knowledge, which 

is one feature that differentiates Teamwork from an individual working in isolation.
103

Specifically, within the team macrocognition model, the knowledge building process is 

considered an emergent process in that each individual on the team is involved in their own 

independent process of collecting Data and converting that data into Information, and ultimately 

Knowledge (referred to as the D-I-K process). As Figure 7-4 shows, parallel individual 

processes called “individual knowledge building processes” merge together and become the 

“team knowledge building processes.”

Figure 7-4. Knowledge building process within team macrocognition (Fiore, Rosen, et 

al., 2010).

It is also interesting to note that the D-I-K process bears a rough resemblance to the 

macrocognitive framework presented in this document, in that both describe cognitive 

processes where by external stimuli are received as inputs that are processed to facilitate a 

behavioral response. The key difference is that D-I-K describes the functional outcomes of the 

process, as data becomes knowledge, whereas the macrocognitive framework presented in this 

document describes the process in terms of psychological constructs.

7.3 Cognitive Mechanisms and Proximate Causes of Failure of

Teamwork

Multiple studies have attempted to identify the important characteristics of an effective team. For 

example, O’Hara and Roth (2005) investigated the importance of team performance in NPPs

and the role of technology in supporting and/or disrupting teamwork. Salas et al. (2005)

identified team characteristics like ‘mutual performance monitoring’, ‘backup behavior’ and 

‘adaptability.’ Furthermore, Paris, Salas and Cannon-Bowers (2000) summarized the skills 

required for effective teamwork from multiple domains into 10 teamworking behaviors:

adaptability, shared situational awareness, mutual performance monitoring, motivating team 

members/team leadership, mission analysis, communication, decisionmaking, assertiveness, 

interpersonal relations, and conflict resolution. O’Conner et al. (2008) classified teamwork 

functions for NPP crews into five categories: building situation awareness, team-focused 

decisionmaking, communication, collaboration, and coordination. The first two categories are 
104

goals of teamwork, whereas the latter three categories are basic teamwork functions. 

Characterizing teamwork as consisting of communication, collaboration, and coordination 

provides a useful framework for modeling the key components of teamwork in NPP operations. 

• Communication is concerned with the exchange of any information between different team 

members. Communication includes assertiveness (i.e., communicating ideas and 

observations in a manner which is persuasive to other team members) and exchanging 

information clearly and accurately between team members (O’Conner et al., 2008). The 

failure to exchange information and coordinate actions is one factor that differentiates 

between good and bad team performance (Driskell and Salas, 1992). 

• Coordination applies to team members organizing their joint activities to achieve a goal. In 

particular, NPP operations team members must support the other members of the team as 

required and monitor their own and others’ workload. O’Conner et al. (2008) use 

coordination to refer to temporal relationships among activities. Coordination includes 

reacting flexibly to changing requirements of a task or situation, giving help to other team 

members in situations in which it was thought they need assistance, and prioritizing and 

coordinating tasks and resources. 

• Collaboration refers to the manner in which members of a team are working together.

O’Conner et al. (2008) characterize the elements of collaboration as leadership (directing 

and coordinating the activities of, and motivating other team members, assessing team 

performance, and establishing a positive atmosphere), cooperation (two or more team 

members working together on a task which requires meaningful task interdependence 

without any leadership), and followship (cooperating in the accomplishment of a task as 

directed by a more senior team member).

Theoretically, the proximate causes for failure of Teamwork can be attributed to the three basic 

teamwork processes identified by O’Conner et al. (2008): communication, collaboration, and 

coordination. However, failures that are unique to the macrocognitive function of Teamwork in 

control room settings and independent of each other can be more simply grouped into two 

categories: communication and leadership (which is one of the elements in collaboration). 

Failures in coordination (adaptability, supporting behavior, and team workload management) 

can be attributed to either failures of communication or leadership. In addition, the cooperation 

and followship elements of collaboration are less relevant for NPP control room operations 

during at-power events given the highly structured and proceduralized environment. Therefore, 

we focus on two key proximate causes of failure of Teamwork in the NPP control room: 

communication and leadership. 

7.3.1 Communication

Communication is ubiquitous in Teamwork because every aspect of coordination and team 

performance depends on communication, either in real-time or during prior planning stages. In 

identifying the key communication-related cognitive mechanisms underlying the Teamwork 

function, we considered the fact that there are, in the simplest case, two different people 

involved in the communication process: the source of the communication (sender), and the 

target of the communication (receiver). Both the source and target of the communication can 

commit either errors of omission or errors of commission. Finally, in the simplest case, the 

communication process (or cycle in “three-way communication”) needs to occur only one time. 

When these concepts are combined, the following basic cognitive mechanisms can be 

identified:

• Source error of omission (sender does not communicate information).

• Source error of commission (sender communicates wrong information).
105

• Target error of omission (receiver does not detect or notice communication, or receiver does 

not comprehend information).

• Target error of commission (receiver is listening to the wrong source, or receiver is listening 

for wrong information).

• Incorrect timing of communication (e.g., delayed, premature, communicated too slowly).

Lee, Ha, and Seong (2011) performed an analysis of communication errors using a derivation of 

the Cognitive Reliability and Error Analysis Method (CREAM) (Hollnagel, 1998) and developed 

a very similar set of communication error types. Their nine communication error types, shown in

Table 7-2, are organized by error mode, and focuses on the source/sender and message 

content errors, but not on target/receiver errors.

Table 7-2. Communication error types (Lee, Ha, and Seong, 2011).

Error Mode Error Type

Timing (too early, too late, omission) Message is sent at the wrong time

Message is not sent at all

Acoustic feature (uncommon) Message is sent with an uncommon acoustic 

feature

Channel (wrong direction, wrong route) Message is sent to the wrong place or person

Message is sent through an inadequate route

Contents (wrong terminology, unexpected 

contents for receiver, unrelated contents)

Message production is inadequate

Message content is inappropriate for the receiver

Message content is wrong

Sequence (jump forward, repetition, 

reversal)

Message content is inconsistent with other 

information

There are numerous PIFs that can influence team communication. Groth & Mosleh’s (2012) 

taxonomy of PIFs, discussed in Chapter 2, included team-based factors, such as team 

coordination, team cohesion, and role awareness. Each of these PIFs can influence the 

cognitive mechanisms related to communication in a team setting. For example, a lack of team 

coordination may adversely affect the timing of communication. Similarly, a lack of role 

awareness may result in either a source or target error of omission, because the crew member 

is not aware of his or her responsibilities for communicating information or receiving information 

from other crew members. Additional types of PIFs that can affect team communication include 

social or environmental factors (e.g., time pressure, team dynamics, excessive authority 

gradient), and individual differences (e.g., knowledge/experience, risk perception, excessive 

professional courtesy). Appendix A.5 provides more detail on how each of the communication

cognitive mechanisms ties to various PIFs and is explained by various theories and models in 

psychology. For example, a source error of omission could be due to team cohesion (Helmreich 

& Foushee, 1993), social pressure (Zajonc, 1965), or groupthink (Janis, 1972, 1982), just to 

name a few PIFs and their psychological bases.

Complexities of Communication 

In addition, communication in NPPs rarely involves only the simplest case of one sender and 

one receiver. A NPP control room has at least three crewmembers. Moreover, if 

communication is occurring outside of the control room, there may be more than three 

individuals involved, and it is important to note that the growth of opportunities for 
106

communication errors is geometric as additional individuals become involved in the 

communication process. 

However, in the more complicated instances of communication involving more than two 

individuals, the cognitive mechanisms do not fundamentally change. The extent and severity of 

the consequences of communication errors may become increasingly more complicated as the 

number of individuals involved increases. Some example consequences include an individual 

failing to have an updated understanding of the situation (when everyone else has the correct 

updated understanding of the situation), multiple individuals having the same incorrect 

understanding of the current situation, or multiple individuals having different incorrect 

understandings of the current situation.

7.3.2 Leadership

In the context of this chapter, leadership includes the management of the emergent team 

process, which is likely to manifest itself multiple times during an operating shift. That is, 

individual macrocognitive functions like Detecting and Noticing and Understanding and 

Sensemaking are regularly occurring in parallel by all team members when a crew is on duty, 

and leadership is required when it is necessary to combine these independent processing 

efforts in order to facilitate Teamwork.

At a minimum, leadership involves making decisions, supervising direct reports, communicating, 

and managing the emergent Teamwork processes. Given this minimal definition, the cognitive 

mechanisms for leadership mirror these aspects. The cognitive mechanisms for leadership 

include:

• Decisionmaking failures.

• Failure to verify that the RO, BOP, and/or other operator have correctly performed their 

responsibilities.

• Failure to consider information communicated by an individual.

• Failure to iterate the communication process sufficiently.

Leadership decisionmaking works in the same way as other individual decisionmaking 

processes. As such, PIFs related to decisionmaking are discussed in Chapter 5 as part of the 

Decisionmaking macrocognitive function. In the context of Teamwork, leadership 

decisionmaking can affect how parallel processes are combined by the team to develop a 

shared understanding of the situation, make a decision, and proceed to take actions based on 

that decision. 

For supervision failures, the relevant PIFs include time pressure, organizational influences 

(team composition, role definitions, operating philosophies, etc), and individual differences in 

leadership style. For example, time pressure may cause the SRO to rush through his or her 

responsibilities and he or she decides to skip the task of performing a second check (Helmreich 

& Foushee, 1993). Another example is the SRO’s leadership style could be more “hands off,” 

and so instead of micromanaging, the SRO decides not to perform a second check of a routine 

action (Massaiu, et al., 2011).

An example of failure to consider information communicated by an individual is when the SRO 

ignores or discounts information the RO or BOP is communicating because of poor team 

cohesion (Helmreich & Foushee, 1993). Leadership style can also serve as a PIF for this 

cognitive mechanism. For example, an overly autocratic SRO who is unwilling to consider the 

RO’s and BOP’s perspectives may create a large power distance gap between the crew 

members, such that they are unwilling to challenge the SRO with contrary information 
107

(Helmreich & Foushee, 1993). On the other hand, an important aspect of the SRO leadership is 

to discount unsound hypotheses from other operators, and terminate unfruitful discussions. 

The combining of parallel processing efforts is usually done through communication, which can 

be a highly iterative process (Carvalho, et al., 2007). Communication among team members is 

rarely a one-time event, and team leaders can have a strong influence on how much the 

communication process iterates to develop a shared understanding of the situation, make a 

decision, or take action. The amount of iteration needed can depend on the complexity of the 

event and the degree of divergence between crew members’ parallel processing efforts. In

instances where the crew is engaged in understanding an event that they have not trained on 

recently, communication is likely to be highly iterative. An example of leadership failure to iterate 

the communication process would be if the SRO cuts off debate with the RO and BOP prior to 

having a sufficient understanding of the actual situation. 

7.4 Summary

This chapter summarized the psychological research literature relevant to teamwork in NPP 

operations. We focused on two key proximate causes that contribute to failure of the Teamwork 

function: communication failures and errors in leadership. Each proximate cause encompasses 

a number of cognitive mechanisms that explain how the Teamwork function can fail. It is clear 

from this review that there are many different researchers who have studied team performance

(c.f., Roth, 1997; Carvalho, Vidal, and de Carvalho, 2007; Sasou and Reason, 1999; Salas, 

Cooke, and Rosen, 2008; Klein, Wiggins, and Dominguez, 2010; Letsky et al., 2007). Each of 

these researchers approached the study of team performance from a different perspective, and 

provided insights into a variety of PIFs that can influence the failure of Teamwork in NPP 

operations. Appendix A.5 contained additional information about linkages between these PIFs 

and cognitive mechanisms.

The scope of this review focused on communication and leadership as proximate causes 

because those aspects of Teamwork were deemed relevant to crew interactions in NPP control 

rooms during at-power events. However, we noted that teamwork can also involve elements of 

coordination and collaboration. There may be unique aspects of coordination or collaboration 

that are not covered by the proximate causes of communication and leadership and, therefore,

should be considered in the future when modeling activities outside the control room. Another 

limitation in this chapter is that it focused on an individual’s contribution to teamwork, without 

specifically addressing many team-level characteristics. Thus, some mechanisms identified in 

this chapter are already covered under the mechanisms for other macrocognitive functions (e.g., 

under the ‘failure of leadership’ mode, the mechanism ‘failure to consider information 

communicated by an individual’ is equivalent to ‘information not attended to’ in the Detecting 

and Noticing function). In general, the team functions highlight some additional error 

mechanisms for a specific individual (the leader) and classes of recoverable communication 

errors. As a result, organizational factors that influence team performance are not explicitly 

included as PIFs. Examples include openness and democracy of team, procedure compliance 

policy, and prescribed communication protocols. There may be an opportunity to expand the 

scope of this review in the future to include more complete coverage of team-level and 

organization-level characteristics that influence macrocognition.

109

8. REFERENCES

Aarts, K., & Pourtois, G. (2010). Anxiety not only increases, but also alters early errormonitoring functions. Cognitive, Affective & Behavioral Neuroscience, 10(4), 479-492. 

doi:10.3758/cabn.10.4.479

Adams, M. J., Tenney, Y. J., & Pew, R. W. (1995). Situation awareness and the cognitive 

management of complex systems. Human Factors, 37(1), 85-104. 

doi:10.1518/001872095779049462

Altmann, E. M., & Trafton, J. G. (2002). Memory for goals: An activation-based model. Cognitive 

Science: A Multidisciplinary Journal, 26(1), 39-83. doi:10.1016/s0364-0213(01)00058-1

Amendola, A., Bersini, U., Cacciabue, P. C., & Mancini, G. (1987). Modelling operators in 

accident conditions: Advances and perspectives on a cognitive model. International Journal 

of Man-Machine Studies, 27, 599-612.

Andersen, R. A., & Buneo, C. A. (2002). Intentional maps in posterior parietal cortex. Annual 

Review of Neuroscience, 25(1), 189-220. doi:10.1146/annurev.neuro.25.112701.142922

Anderson, J. R. (1993). Rules of the Mind. Hillsdale, NJ: Erlbaum.

Anderson, J. R. (2000). Cognitive Psychology and Its Implications (5th ed.). New York, NY: 

Worth Publishers.

Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control 

processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and 

motivation: II. Oxford England: Academic Press.

Baddeley, A. D. (1992). Working memory. Science, 255(5044), 556.

Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in 

Cognitive Sciences, 4(11), 417-423. doi:10.1016/s1364-6613(00)01538-2

Baddeley, A. D., & Hitch, G. J. L. (1974). Working memory. In G. A. Bower (Ed.), The 

Psychology of Learning and Motivation: Advances in research and theory (Vol. 8, pp. 47-

89). New York: Academic Press.

Baddeley, A. D., & Larsen, J. D. (2007). The phonological loop: Some answers and some 

questions. The Quarterly Journal of Experimental Psychology, 60(4), 512-518. 

doi:10.1080/17470210601147663

Beck, M. R., Levin, D. T., & Angelone, B. (2007). Change blindness blindness: Beliefs about the 

roles of intention and scene complexity in change detection. Consciousness and Cognition: 

An International Journal, 16(1), 31-51. doi:10.1016/j.concog.2006.01.003

Bedny, G. Z., & Karwowski, W. (2004). A functional model of the human orienting activity. 

Theoretical Issues in Ergonomics Science, 5(4), 255-274. 

doi:10.1080/1463922031000086681

Bedny, G. Z., Karwowski, W., & Jeng, O.-J. (2004). The situational reflection of reality in activity 

theory and the concept of situation awareness in cognitive psychology. Theoretical Issues in 

Ergonomics Science, 5(4), 275-296. doi:10.1080/1463922031000075070

Bedny, G. Z., & Meister, D. (1999). Theory of activity and situation awareness. International 

Journal of Cognitive Ergonomics, 3(1), 63-72.

Bedny, G. Z., Seglin, M. H., & Meister, D. (2000). Activity theory: History, research and 

application. Theoretical Issues in Ergonomics Science, 1(2), 168 - 206.
110

Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. 

Psychological Review, 94(2), 115-147. doi:10.1037/0033-295x.94.2.115

Boring, R. L. (2011). Information foraging in nuclear power plant control rooms. Paper presented 

at the European Safety and Reliability (ESREL) Conference: Advances in Safety, Reliability, 

and Risk Management, Troyes, France.

Brewer, G. A., Knight, J. B., Marsh, R. L., & Unsworth, N. (2010). Individual differences in eventbased prospective memory: Evidence for multiple processes supporting cue detection. 

Memory & Cognition, 38(3), 304-311. doi:10.3758/mc.38.3.304

Broadbent, D. E. (1958). Perception and communication. Elmsford, NY US: Pergamon Press.

Bubić, A. (2008). Attentional bias in change detection. Review of Psychology, 15(1-2), 57-65.

Burns, K. (2000). Mental models and normal errors in naturalistic decision making. Paper 

presented at the The 5th Conference on Naturalistic Decision Making, Tammsvik, Sweden.

Bustamante, E. A. (2008). Implementing likelihood alarm technology in integrated aviation 

displays for enhancing decision-making: A two-stage signal detection modeling approach. 

International Journal of Applied Aviation Studies, 8(2), 241-261.

Byrne, M. D., & Bovair, S. (1997). A working memory model of a common procedural error. 

Cognitive Science: A Multidisciplinary Journal, 21(1), 31-61. doi:10.1016/s0364-

0213(99)80018-4

Cacciabue, P. C., & Hollnagel, E. (1995). Simulation of cognition: Applications. In J.-M. Hoc, P. 

C. Cacciabue & E. Hollnagel (Eds.), Expertise and technology: Cognition & human–

computer cooperation. (pp. 55-73). Hillsdale, NJ England: Lawrence Erlbaum Associates, 

Inc.

Cacciabue, P. C., Mancini, G., & Bersini, U. (1990). A model of operator behaviour for manmachine system simulation. Automatica, 26(6), 1025-1034.

Caggiano, D. M., & Parasuraman, R. (2004). The role of memory representation in the vigilance 

decrement. Psychonomic Bulletin & Review, 11(5), 932-937.

Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team 

decision making. In N. J. Castellan, Jr. (Ed.), Individual and group decision making: Current 

issues. (pp. 221-246). Hillsdale, NJ England: Lawrence Erlbaum Associates, Inc.

Carvalho, P. V. R., dos Santos, I. L., & Vidal, M. C. R. (2006). Safety implications of cultural and 

cognitive issues in nuclear power plant operation. Applied Ergonomics, 37(2), 211-223. 

doi:10.1016/j.apergo.2005.03.004

Carvalho, P. V. R., Vidal, M. C. R., & de Carvalho, E. F. (2007). Nuclear power plant 

communications in normative and actual practice: A field study of control room operators' 

communications. Human Factors and Ergonomics in Manufacturing, 17(1), 43-78. 

doi:10.1002/hfm.20062

Cellier, J. M., Eyrolle, H., & Marine, C. (1997). Expertise in dynamic environments. Ergonomics, 

40(1), 28-50. doi:10.1080/001401397188350

Chang, Y. H. J., & Mosleh, A. (2007a). Cognitive modeling and dynamic probabilistic simulation 

of operating crew response to complex system accidents. Part 1: Overview of the IDAC 

Model. Reliability Engineering & System Safety, 92(8), 997-1013. 

doi:10.1016/j.ress.2006.05.014
111

Chang, Y. H. J., & Mosleh, A. (2007b). Cognitive modeling and dynamic probabilistic simulation 

of operating crew response to complex system accidents. Part 2: IDAC performance 

influencing factors model. Reliability Engineering & System Safety, 92(8), 1014-1040. 

doi:10.1016/j.ress.2006.05.010

Chang, Y. H. J., & Mosleh, A. (2007c). Cognitive modeling and dynamic probabilistic simulation 

of operating crew response to complex system accidents. Part 3: IDAC operator response 

model. Reliability Engineering & System Safety, 92(8), 1041-1060. 

doi:10.1016/j.ress.2006.05.013

Chang, Y. H. J., & Mosleh, A. (2007d). Cognitive modeling and dynamic probabilistic simulation 

of operating crew response to complex system accidents. Part 4: IDAC causal model of 

operator problem-solving response. Reliability Engineering & System Safety, 92(8), 1061-

1075. doi:10.1016/j.ress.2006.05.011

Chang, Y. H. J., & Mosleh, A. (2007e). Cognitive modeling and dynamic probabilistic simulation 

of operating crew response to complex system accidents. Part 5: Dynamic probabilistic 

simulation of the IDAC model. Reliability Engineering & System Safety, 92(8), 1076-1101. 

doi:10.1016/j.ress.2006.05.012

Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and Representation of 

Physics Problems by Experts and Novices. Cognitive Science, 5(2), 121-152.

Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. 

Levine & S. D. Teasley (Eds.), Perspectives on socially shared cognition. (pp. 127-149). 

Washington, DC US: American Psychological Association.

Clemen, R. T. (2001). Naturalistic Decision Making and Decision Analysis. Journal of Behavioral 

Decision Making, 14, 353-384.

Cohen, M. S., Freeman, J. T., & Thompson, B. B. (1997). Training the naturalistic decision 

maker. In C. E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 257-268). 

Mahwah, NJ: Erlbaum.

Cook, M. (2001). Memory and complex skills. In J. Noyes & M. Bransby (Eds.), People in 

control: Human factors in control room design. (pp. 17-33). Edison, NJ US: Institution of 

Electrical Engineers.

Cooke, N. J., Gorman, J. C., & Rowe, L. J. (2009). An ecological perspective on team cognition. 

In E. Salas, G. F. Goodwin & C. S. Burke (Eds.), Team effectiveness in complex 

organizations: Cross-disciplinary perspectives and approaches. (pp. 157-182). New York, 

NY US: Routledge/Taylor & Francis Group.

Cooke, N. J., Gorman, J. C., & Winner, J. L. (2007). Team cognition. In F. T. Durso, R. S. 

Nickerson, S. T. Dumais, S. Lewandowsky & T. J. Perfect (Eds.), Handbook of applied 

cognition (2nd ed.). (pp. 239-268). Hoboken, NJ US: John Wiley & Sons Inc.

Coyne, K. A. (2009). A predictive model of nuclear power plant crew decision-making and 

performance in a dynamic simulation environment. Ph.D. Dissertation, University of 

Maryland, College Park, Maryland.

Davies, D. R., & Parasuraman, R. (1982). The Psychology of Vigilance. London: Academic 

Press.

Degani, A., Shafto, M., & Kirlik, A. (1999). Modes in Human-Machine Systems: Constructs, 

Representation, and Classification. International Journal of Aviation Psychology, 9(2), 125.
112

Doane, S. M., Sohn, Y. W., & Jodlowski, M. T. (2004). Pilot Ability to Anticipate the 

Consequences of Flight Actions as a Function of Expertise. Human Factors, 46(1), 92-103. 

doi:10.1518/hfes.46.1.92.30386

Dodhia, R. M., & Dismukes, R. K. (2009). Interruptions create prospective memory tasks. 

Applied Cognitive Psychology, 23(1), 73-89. doi:10.1002/acp.1441

Donald, C. (2001). Vigilance. In J. Noyes & M. Bransby (Eds.), People in control: Human factors 

in control room design. (pp. 35-50). Edison, NJ US: Institution of Electrical Engineers.

Donald, F. M. (2008). The classification of vigilance tasks in the real world. Ergonomics, 51(11), 

1643-1655. doi:10.1080/00140130802327219

Driskell, J. E., & Salas, E. (1992). Collective behavior and team performance.Human Factors: 

The Journal of the Human Factors and Ergonomics Society,34(3), 277-288.

Durlach, P. J., Kring, J. P., & Bowens, L. D. (2008). Detection of icon appearance and 

disappearance on a digital situation awareness display. Military Psychology, 20(2), 81-94. 

doi:10.1080/08995600701869502

Durso, F. T., & Gronlund, S. D. (1999). Situation awareness. In F. T. Durso (Ed.), Handbook of 

applied cognition. (pp. 283-314). New York, NY US: John Wiley & Sons Ltd.

Durso, F. T., Rawson, K. A., & Girotto, S. (2007). Comprehension and situation awareness. In 

F. T. Durso, R. S. Nickerson, S. T. Dumais, S. Lewandowsky & T. J. Perfect (Eds.), 

Handbook of applied cognition (2nd ed.). (pp. 164-193). Hoboken, NJ US: John Wiley & 

Sons Inc.

Dyer, J. L. (1984). Team research and team training: A state of the art review. In F. A. Muckler 

(Ed.), Human Factors Review (pp. 285-323). Santa Monica, CA: Human Factors Society.

Edwards, W., & Fasolo, B. (2001). Decision Technology. Annual Review of Psychology, 52, 

581-606.

Einhorn, H. J., & Hogarth, R. M. (1978). Confidence in judgment: Persistence of the illusion of 

validity. Psychological Review, 85(5), 395-416.

Einhorn, H. J., & Hogarth, R. M. (1981). Behavioral decision theory: Processes of judgment and 

choice. Annual Review of Psychology, 32, 53-88. 

doi:10.1146/annurev.ps.32.020181.000413

Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human 

Factors, 37(1), 32-64. doi:10.1518/001872095779049543

Endsley, M. R. (1997). The role of situation awareness in naturalistic decision making. In C. E. 

Zsambok & G. Klein (Eds.), Naturalistic decision making (pp. 269-284). Mahwah, NJ: 

Erlbaum.

Endsley, M. R. (2000). Theoretical underpinnings of situation awareness: A critical review. In M. 

R. Endsley & D. J. Garland (Eds.), Situation Awareness Analysis and Measurement. 

Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Endsley, M. R. (2004). Situation awareness: Progress and directions. A cognitive approach to 

situation awareness: Theory, measurement and application, 317-341.

Endsley, M. R. (2006). Expertise and Situation Awareness. In K. A. Ericsson, N. Charness, P. J. 

Feltovich & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert 

performance. (pp. 633-651). New York, NY US: Cambridge University Press.
113

Endsley, M. R., Bolté, B., & Jones, D. G. (2003). Designing for Situation Awareness: An 

Approach to User-Centered Design. London: Taylor & Francis.

Endsley, M. R., & Garland, D. J. (Eds.). (2000). Situation Awareness Analysis and 

Measurement. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Eriksen, C. W., & St. James, J. D. (1986). Visual attention within and around the field of focal 

attention: A zoom lens model. Perception & Psychophysics, 40(4), 225-240.

Espinosa, J. A., Lerch, F. J., & Kraut, R. E. (2004). Explicit versus implicit coordination 

mechanisms and task dependencies: One size does not fit all. In E. Salas & S. M. Fiore 

(Eds.), Team cognition: Understanding the factors that drive process and performance. (pp. 

107-129). Washington, DC US: American Psychological Association.

Espinosa-Paredes, G., Nunez-Carrera, A., Laureano-Cruces, A. L., Vazquez-Rodriguez, A., & 

Espinosa-Martinez, E.-G. (2008). Emergency Management for a Nuclear Power Plant Using 

Fuzzy Cognitive Maps. Annals of Nuclear Energy, 35, 2387-2396.

Federico, P.-A. (1995). Expert and novice recognition of similar situations. Human Factors, 

37(1), 105-122. doi:10.1518/001872095779049471

Feltovich, P. J., Johnson, P. E., Moller, J. H., & Swanson, D. B. (1984). LCS: The role and 

development of medical knowledge in diagnostic expertise. In W. J. Clancey & E. H. 

Shortliffe (Eds.), Readings in Medical Artificial Intelligence: The First Decade (pp. 275-319). 

Reading, MA: Addison-Wesley.

Fiore, S. M., Rosen, M. A., Smith-Jentsch, K. A., Salas, E., Letsky, M., & Warner, N. (2010). 

Toward an understanding of macrocognition in teams: Predicting processes in complex 

collaborative contexts. Human Factors, 52(2), 203-224. doi:10.1177/0018720810369807

Fiore, S. M., Smith-Jentsch, K. A., Salas, E., Warner, N., & Letsky, M. (2010). Towards an 

understanding of macrocognition in teams: Developing and defining complex collaborative 

processes and products. Theoretical Issues in Ergonomics Science, 11(4), 250-271. 

doi:10.1080/14639221003729128

Forester, J. A. (in press). NRC/EPRI Draft Report on a Hybrid Approach to Create a Consensus 

Approach for HRA: Sandia National Laboratories.

Gonzalez, C., Lerch, J. F., & Lebiere, C. (2003). Instance-based Learning in Dynamic Decision 

Making. Cognitive Science, 2003(27), 591-635.

Greitzer, F. L., Podmore, R., Robinson, M., & Ey, P. (2010). Naturalistic decision making for 

power system operators. International Journal of Human-Computer Interaction, 26(2-3), 278-

291. doi:10.1080/10447310903499070

Groth, K. M. (2009). A Data-Informed Model of Performance Shaping Factors for Use in Human 

Reliability Anlaysis. Doctor of Philosophy Dissertation, University of Maryland, College Park, 

MD.

Groth, K. M., & Mosleh, A. (2012). A data-informed PIF hierarchy for model-based Human 

Reliability Analysis. Reliability Engineering & System Safety. doi:10.1016/j.ress.2012.08.006

Gugerty, L. J., & Tirre, W. C. (2000). Individual differences in situation awareness. In M. R. 

Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement. (pp. 249-

276). Mahwah, NJ US: Lawrence Erlbaum Associates Publishers.

Gutwin, C., & Greenberg, S. (2004). The importance of awareness for team cognition in 

distributed collaboration. In E. Salas & S. M. Fiore (Eds.), Team cognition: Understanding 
114

the factors that drive process and performance. (pp. 177-201). Washington, DC US: 

American Psychological Association.

Helmreich, R. L. (1999). Ten years of change: Crew Resource Management 1989-1999. Paper 

presented at the Fourth ICAO Global Safety and Human Factors Symposium, Santiago, 

Chile.

Helmreich, R. L., & Foushee, H. C. (1993). Why crew resource management? Empirical and 

theoretical bases of human factors training in aviation. In E. L. Wiener, B. G. Kanki & R. L. 

Helmreich (Eds.), Cockpit resource management. (pp. 3-45). San Diego, CA US: Academic 

Press.

Helton, W. S., Head, J., & Kemp, S. (2011). Natural disaster induced cognitive disruption: 

Impacts on action slips. Consciousness and Cognition, 20(4), 1732-1737. 

doi:10.1016/j.concog.2011.02.011

Hendrickson, S. M. L., Forester, J. A., Dang, V. N., Mosleh, A., Lois, E. L., & Xing, J. (2012). 

HRA Method Analysis Criteria. Paper presented at the Joint 11th Probabilistic Safety 

Assessment and Management (PSAM) Conference and the 12th European Safety and 

Reliability (ESREL) Conference, Helsinki, Finland. 

Hendy, K. C., Beevis, D., Lichacz, F., & Edwards, J. L. (2001). Analyzing the Cognitive System 

From a Perceptual Control Theory Point of View. Human Systems IAC GATEWAY, 12(1), 

18-19.

Hendy, K. C., Farrell, P. S. E., & East, K. P. (2001). An information-processing model of 

operator stress and performance. In P. A. Hancock & P. A. Desmond (Eds.), Stress, 

workload, and fatigue. (pp. 34-80). Mahwah, NJ US: Lawrence Erlbaum Associates 

Publishers.

Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups 

as information processors. Psychological Bulletin, 121(1), 43-64. doi:10.1037/0033-

2909.121.1.43

Hitchcock, E. M., Warm, J. S., Matthews, G., Dember, W. N., Shear, P. K., Tripp, L. D., . . . 

Parasuraman, R. (2003). Automation cueing modulates cerebral blood flow and vigilance in 

a simulated air traffic control task. Theoretical Issues in Ergonomics Science, 4(1-2), 89-112. 

doi:10.1080/14639220210159726

Hoc, J.-M., & Amalberti, R. (2005). Modeling naturalistic decision-making cognitive activities in 

dynamic situations: The role of a coding scheme. In H. Montgomery, R. Lipshitz & B. 

Brehmer (Eds.), How Professionals Make Decisions (pp. 319-334). Mahwah, NJ: Lawrence 

Erlbaum Assoc.

Hollnagel, E. (1998). Cognitive reliability and error analysis method. Amsterdam: Elsevier 

Science Ltd.

Hollnagel, E. (2009). The ETTO Principle: Efficiency-Thoroughness Trade-Off. Aldershot, 

England: Ashgate.

Hsieh, S., Tsai, C.-Y., & Tsai, L.-L. (2009). Error correction maintains post-error adjustments 

after one night of total sleep deprivation. Journal of Sleep Research, 18(2), 159-166. 

doi:10.1111/j.1365-2869.2008.00730.x

Janis, I. L. (1972). Victims of groupthink: A psychological study of foreign-policy decisions and 

fiascoes. Oxford England: Houghton Mifflin.
115

Janis, I. L. (1982). Groupthink: Psychological Studies of Policy Decisions and Fiascoes. 

Wilmington, MA: Houghton Mifflin.

Jenkins, D. P., Stanton, N. A., Salmon, P. M., Walker, G. H., & Rafferty, L. (2010). Using the 

Decision-Ladder to Add a Formative Element to Naturalistic Decision-Making Research. 

International Journal of Human-Computer Interaction, 26(2-3), 132-146.

Jodlowski, M. T. (2008). Extending long term working memory theory to dynamic domains: The 

nature of retrieval structures in situation awareness. 69, ProQuest Information & Learning, 

US. Available from EBSCOhost psyh database.

Johannsdottir, K. R. (2005). Situation awareness and working memory: An integration of an 

applied concept with a fundamental cognitive process. 65, ProQuest Information & Learning, 

US. Available from EBSCOhost psyh database.

Jones, D. G., & Endsley, M. R. (1996). Sources of situation awareness errors in aviation. 

Aviation, Space, and Environmental Medicine, 67(6), 507-512.

Jones, D. G., & Endsley, M. R. (2000). Overcoming representational errors in complex 

environments. Human Factors, 42(3), 367-378. doi:10.1518/001872000779698187

Jones, P. E., & Roelofsma, P. H. M. P. (2000). The potential for social contextual and group 

biases in team decision-making: biases, conditions and psychological mechanisms. 

Ergonomics, 43(8), 1129-1152. doi:10.1080/00140130050084914

Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. 

Econometrica, 47(2), 263-291. http://www.econometricsociety.org

Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 

39(4), 341-350. doi:10.1037/0003-066x.39.4.341

Kasbi, C., & de Montmollin, M. (1991). Activity without decision and responsibility: The case of 

nuclear power plants. In J. Rasmussen, B. Brehmer & J. Leplat (Eds.), Distributed Decision 

Making: Cognitive Models for Cooperative Work (pp. 275-283). New York: John Wiley & 

Sons.

Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. 

(2010). Control and interference in task switching—A review. Psychological Bulletin, 136(5), 

849-874. doi:10.1037/a0019842

Kim, H., Yoon, W. C., & Choi, S. (1999). Aiding fault diagnosis under symptom masking in 

dynamic systems. Ergonomics, 42(11), 1472-1481. doi:10.1080/001401399184820

Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G. 

A. Klein, J. Orasanu, R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: 

Models and methods. (pp. 138-147). Westport, CT US: Ablex Publishing.

Klein, G. A. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press.

Klein, G. A. (2008). Naturalistic Decision Making. Human Factors: The Journal of the Human 

Factors and Ergonomics Society, 50(3), 456-460. doi:10.1518/001872008x288385

Klein, G. A., Armstrong, A., Woods, D. D., Gokulachandra, M., & Klein, H. A. (2000). Cognitive 

wavelength: The role of common ground in distributed replanning. (Technical Report AFRLHE-WP-TR-2001–0029). Wright Patterson Air Force Research Laboratory.

Klein, G. A., Feltovich, P. J., & Woods, D. D. (2005). Common ground and coordination in joint 

activity. In W. Rouse & K. Boff (Eds.), Organizational Simulation (pp. 139-178). New York, 

NY: Wiley.
116

Klein, G. A., Moon, B. M., & Hoffman, R. R. (2006). Making sense of sensemaking. Part 2: A 

macrocognitive model. IEEE Intelligent Systems, 21(5), 88-92. doi:10.1109/mis.2006.100

Klein, G. A., Phillips, J. K., Rall, E. L., & Peluso, D. A. (2007). A data-frame theory of 

sensemaking. In R. R. Hoffman (Ed.), Expertise out of context: Proceedings of the Sixth 

International Conference on Naturalistic Decision Making. (pp. 113-155). Mahwah, NJ US: 

Lawrence Erlbaum Associates Publishers.

Klein, G. A., Ross, K. G., Moon, B. M., Klein, D. E., Hoffman, R. R., & Hollnagel, E. (2003). 

Macrocognition. Intelligent Systems, IEEE, 18(3), 81-85.

Klein, G. A., Snowden, D., & Pin, C. L. (2011). Anticipatory thinking. Informed by knowledge: 

Expert performance in complex situations., 235-245.

Klein, G. A., Wiggins, S., & Dominguez, C. O. (2010). Team sensemaking. Theoretical Issues in 

Ergonomics Science, 11(4), 304-320. doi:10.1080/14639221003729177

Köhler, W. (1947). Gestalt psychology; an introduction to new concepts in modern psychology 

(Rev. ed.). Oxford England: Liveright.

Kurby, C. A., & Zacks, J. M. (2007). Segmentation in the perception and memory of events. 

Trends in Cognitive Sciences, 12(2), 72-79. doi:10.1016/j.tics.2007.11.004

Lang, A. W., Roth, E. M., Bladh, K., & Hine, R. (2002, 2002/01/01/). Using a benchmarkreferenced approach for validating a power plant control room: Results of the baseline 

study. Paper presented at the Human Factors and Ergonomics Society 46th Annual 

Meeting, Santa Monica, CA.

Latané, B., Williams, K., & Harkins, S. (1979). Many hands make light the work: The causes and 

consequences of social loafing. Journal of Personality and Social Psychology, 37(6), 822-

832. doi:10.1037/0022-3514.37.6.822

Lavine, R. A., Sibert, J. L., Gokturk, M., & Dickens, B. (2002). Eye-tracking measures and 

human performance in a vigilance task. Aviation, Space, and Environmental Medicine, 

73(4), 367-372.

Lee, S. M., Ha, J. S., & Seong, P. H. (2011). CREAM-based communication error analysis 

method (CEAM) for nuclear power plant operators‚Äô communication. Journal of Loss 

Prevention in the Process Industries, 24(1), 90-97.

Leont'ev, A. N. (1977). Activity, Consciousness and Personality. Moscow: Moscow University 

Publishers.

Letsky, M. (2007). Macrocognition in Collaboration and Knowledge Ineroperability. Paper 

presented at the 51st Annual Meeting of the Human Factors and Ergonomics Society, 

Baltimore, MD.

Letsky, M., Warner, N., Fiore, S. M., Rosen, M. A., & Salas, E. (2007). Macrocognition in 

complex team problem solving. Paper presented at the Twelfth International Command and 

Control Research and Technology Symposium (12th ICCRTS), Newport, RI.

Letsky, M., Warner, N., Fiore, S. M., & Smith, C. A. P. (Eds.). (2008). Macrocognition in Teams: 

Theories and Methodologies. Hampshire, England: Ashgate Publishing Limited.

Levi, D. M. (2008). Crowding--An essential bottleneck for object recognition: A mini-review. 

Vision Research, 48(5), 635-654. doi:10.1016/j.visres.2007.12.009

Lin, C. J., Chang, H.-C., Chen, H.-J., & Yang, C.-W. (2009). Human signal-detection 

performance in multiscreen monitoring tasks. Perceptual and Motor Skills, 109(2), 546-550.
117

Lin, M.-H. (2009). Analytic-Holistic thinking, information use, and sensemaking during unfolding 

events. 69, ProQuest Information & Learning, US. Available from EBSCOhost psych 

database.

Lipshitz, R. (1993). Converging themes in the study of decision making in realistic settings. In G. 

A. Klein, J. Orasanu, R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: 

Models and methods. (pp. 103-137). Westport, CT US: Ablex Publishing.

Lipshitz, R., Klein, G. A., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision 

making. Journal of Behavioral Decision Making, 14(5), 331-352. doi:10.1002/bdm.381

Liu, C.-C., Doong, J.-L., Lin, C.-Y., Lai, C.-H., Jeng, M.-C., & Hwang, J.-R. (2009). Responses in 

a manual tracking and visual detection task. Perceptual and Motor Skills, 109(3), 899-910.

Lois, E. L., Dang, V. N., Forester, J. A., Broberg, H., Massaiu, S., Hildebrandt, M., . . . Bye, A. 

(2009). International HRA Empirical Study – Phase 1 Report: Description of Overall 

Approach and Pilot Phase Results from Comparing HRA Methods to Simulator Performance 

Data. (NUREG/IA-0216). Washington, DC: US Nuclear Regulatory Commission.

Lundberg, C. G. (2000). Made sense and remembered sense: Sensemaking through abduction. 

Journal of Economic Psychology, 21(6), 691-709. doi:10.1016/s0167-4870(00)00027-1

MacLean, K. A., Aichele, S. R., Bridwell, D. A., Mangun, G. R., Wojciulik, E., & Saron, C. D. 

(2009). Interactions between endogenous and exogenous attention during vigilance. 

Attention, Perception, & Psychophysics, 71(5), 1042-1058. doi:10.3758/app.71.5.1042

Malcolmson, K. A., Reynolds, M. G., & Smilek, D. (2007). Collaboration during visual search. 

Psychonomic Bulletin & Review, 14(4), 704-709.

Massaiu, S., Hildebrandt, M., & Bone, A. (2011). The Guidance-Expertise Model: Modeling 

Team Decision Making with Emergency Procedures. Paper presented at the 10th 

International Conference on Naturalistic Decision Making (NDM 2011), Orlando, FL.

Massaiu, S., Hildebrandt, M., & Bone, A. (2011). The guidance-expertise model: Modeling team 

decision making with emergency procedures. Paper presented at the 10th International 

Conference on Naturalistic Decision Making, Orlando, FL.

Massaro, D. W., & Cohen, M. M. (1991). Integration versus interactive activation: The joint 

influence of stimulus and context in perception. Cognitive Psychology, 23(4), 558-614. 

doi:10.1016/0010-0285(91)90006-a

Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon-Bowers, J. A. (2000). The 

influence of shared mental models on team process and performance. Journal of Applied 

Psychology, 85(2), 273-283. doi:10.1037/0021-9010.85.2.273

Mellers, B. A., Schwartz, A., & Cooke, A. D. J. (1998). Judgment and decision making. Annual 

Review of Psychology, 49(1), 447.

Meyer, O. (1992). Trip Report: Onsite Analysis of the Human Factors of an Event at Crystal 

River 3 December 8, 1991. (EGG-HFRU-10085). Idaho National Engineering Laboratory.

Milkman, K. L., Chugh, D., & Bazerman, M. H. (2009). How can decision making be improved? 

Perspectives on Psychological Science, 4(4), 379-383.

Morray, N. (1997). Human factors in process control. In G. Salvendy (Ed.), Handbook of Human 

Factors and Ergonomics. New York: Wiley, Inc.

Mumaw, R. J., Roth, E. M., & Burns, C. M. (2000). There is more to monitoring a nuclear power 

plant than meets the eye. Human Factors, 42(1), 36-55. doi:10.1518/001872000779656651
118

Mumaw, R. J., Swatzler, D., Roth, E. M., & Thomas, W. A. (1994). Cognitive Skill Training for 

Nuclear Power Plant Operational Decision Making. (NUREG/CR-6126). Washington, DC: 

U.S. Nuclear Regulatory Commission.

Murphy, E. D., & Mitchell, C. M. (1986). Cognitive attributes: Implications for display design in 

supervisory control systems. International Journal of Man-Machine Studies, 25, 411-438.

Neisser, U. (1967). Cognitive psychology. East Norwalk, CT US: Appleton-Century-Crofts.

Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. 

New York, NY US: W H Freeman/Times Books/ Henry Holt & Co.

Nicholson, W. (1995). Microeconomic Theory: Basic Principles and Extensions (6th ed.). Fort 

Worth, TX: Dryden Press.

Nikolic, M. I., Orr, J. M., & Sarter, N. B. (2004). Why Pilots Miss the Green Box: How Display 

Context Undermines Attention Capture. International Journal of Aviation Psychology, 14(1), 

39-52. doi:10.1207/s15327108ijap1401_3

NRC, U.S. (2000). Technical Basis and Implementation Guidelines for A Technique for Human 

Event Analysis (ATHEANA), Rev. 1. (NUREG-1624). Washington, DC: U.S. Nuclear 

Regulatory Commission.

NRC, U.S. (2006). Staff Requirements—Meeting with Advisory Committee on Reactor 

Safeguards. (SRM-M061020). Washington, DC: U.S. Nuclear Regulatory Commission.

O’Connor, P., O’Dea, A., Flin, R., & Belton, S. (2008). Identifying the team skills required by 

nuclear power plant operations personnel. International Journal of Industrial 

Ergonomics, 38(11), 1028-1037.

O'Hara, J. M., Brown, W. S., Lewis, P. M., & Persensky, J. J. (2002). Human-System Interfaced 

Design Review Guidelines NUREG-0700, Rev. 2. Washington, DC: US Nuclear Regulatory 

Commission.

O’Hara, J. M., & Roth, E. M. (2005). Operational concepts, teamwork, and technology in 

commercial nuclear power stations. In Clint Bowers, Eduardo Salas and Florian Jentsch 

(Eds) Creating High-Tech Teams: Practical guidance on work performance and technology. 

(pp. 139-159).Washington, D. C.: American Psychological Association.

O'Hara, J. M., Higgins, J. C., Brown, W. S., Fink, R., Persensky, J. J., Lewis, P. M., . . . Szabo, 

A. (2008). Human Factors Considerations with Respect to Emerging Technology in Nuclear 

Power Plants. (NUREG/CR-6947). Washington, D.C.: U.S. Nuclear Regulatory Commission.

Orasanu, J. M. (1993). Decision-making in the cockpit. In E. L. Wiener, B. G. Kanki & R. L. 

Helmreich (Eds.), Cockpit Resource Management (pp. 137-172). Boston, MA: Academic 

Press.

Orasanu, J. M., & Martin, L. (1998). Errors in aviation decision making: A factor in accidents and 

incidents. Paper presented at the HESSD 98: 2nd Workshop on Human Error, Safety, and 

System Development, Seattle, WA.

Paris, C. R., Salas, E., & Cannon-Bowers, J. A. (2000). Teamwork in multi-person systems: a 

review and analysis. Ergonomics, 43(8), 1052-75.

Park, J., & Jung, W. (2007). A study on the development of a task complexity measure for 

emergency operating procedures of nuclear power plants. Reliability Engineering & System 

Safety, 92(8), 1102-1116. doi:10.1016/j.ress.2006.03.009
119

Patrick, J., James, N., & Ahmed, A. (2006). Human processes of control: Tracing the goals and 

strategies of control room teams. Ergonomics, 49(12-13), 1395-1414. 

doi:10.1080/00140130600613042

Patterson, E. S., & Hoffman, R. R. (2012). Visualization Framework of Macrocognition 

Functions. Cognition, Technology & Work, Special Issue on Adapting to Change and 

Uncertainty.

Perry, C. M., Sheik-Nainar, M. A., Segall, N., Ma, R., & Kaber, D. B. (2008). Effects of physical 

workload on cognitive task performance and situation awareness. Theoretical Issues in 

Ergonomics Science, 9(2), 95-113. doi:10.1080/14639220600959237

Phillips, W. A. (1974). On the distinction between sensory storage and short-term visual 

memory. Perception & Psychophysics, 16(2), 283-290.

Pirolli, P. (2007). Information foraging theory: Adaptive interaction with information. New York, 

NY US: Oxford University Press.

Platanov, K. K. (1982). System of Psychology and Theory of Reflection. Moscow: Science 

Publishers.

Poldrack, R. A., Sabb, F. W., Foerde, K., Tom, S. M., Asarnow, R. F., Bookheimer, S. Y., & 

Knowlton, B. J. (2005). The Neural Correlates of Motor Skill Automaticity. The Journal of 

Neuroscience, 25(22), 5356-5364. doi:10.1523/jneurosci.3880-04.2005

Powers, W. T. (1973). Behavior: The Control of Perception. New York, NY: Hawthorne.

Prince, C., & Salas, E. (2000). Team situation awareness, errors, and crew resource 

management: Research integration for training guidance. In M. R. Endsley & D. J. Garland 

(Eds.), Situation awareness analysis and measurement. (pp. 325-347). Mahwah, NJ US: 

Lawrence Erlbaum Associates Publishers.

Proctor, R. W., & Van Zandt, T. (2008). Human Factors in Simple and Complex Systems (2nd 

ed.): Taylor & Francis.

Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs, and symbols, and other 

distinctions in human performance models. IEEE Transactions on Systems, Man, and 

Cybernetics, smc-13(3), 257-266.

Reason, J. (1990). Human error. New York, NY US: Cambridge University Press.

Reason, J. (1997). Managing the Risks of Organizational Accidents. Burlington, VT: Ashgate.

Reason, J. (2002). Combating omission errors through task analysis and good reminders. 

Quality & Safety in Health Care, 11(1), 40-44.

Reason, J. (2008). The Human Contribution: Unsafe Acts, Accidents, and Heroic Recoveries: 

Ashgate Publishing Limited.

Reason, J., & Bernsen, N. O. (1991). Errors in a team context. In B. Brehmer (Ed.), Distributed 

Decision Making. Proceedings of the Third MOHAWC Workshop (Vol. 1, pp. 90-107). 

Belgirate: European Esprit Basic Research Project MOHAWC Deliverable D4, Risoe 

National Laboratory.

Reicher, G. M. (1969). Perceptual recognition as a function of meaningfulness of stimulus 

material. Journal of Experimental Psychology, 81(2), 275-280. doi:10.1037/h0027768

Roth, E. M. (1997). Analysis of decision making in nuclear power plant emergencies: An 

investigation of aided decision making. In C. E. Zsambok & G. Klein (Eds.), Naturalistic

decision making. (pp. 175-182). Hillsdale, NJ England: Lawrence Erlbaum Associates, Inc.
120

Roth, E. M. (2010a). Generic List of Complicating Situational Factors, Individual and Team 

‘Macro-Cognitive’ Performance Problems, and Unsafe Actions. Internal project report 

prepared for the HRA Data project, Office of Research, US Nuclear Regulatory Commission.

Roth, E. M. (2010b). Macrocognitive model for HRA. Presentation given at the July 20-22, 2010 

SRM Hybrid HRA Project Meeting. Presentation given at the SRM Hybrid HRA Project 

Meeting. Washington, DC.

Roth, E. M., Chang, J. Y. H., & Richards, R. E. (2011). Untitled. Draft NRC Data Collection 

Method: Characterization of Action Errors (Unpublished project document). U.S. Nuclear 

Regulatory Commission.

Roth, E. M., Multer, J., & Raslear, T. (2006). Shared Situation Awareness as a Contributor to 

High Reliability Performance in Railroad Operations. Organization Studies, 27(7), 967-987. 

doi:10.1177/0170840606065705

Roth, E. M., Mumaw, R. J., & Lewis, P. M. (1994). An Empirical Investigation of Operator 

Performance in Cognitively Demanding Simulated Emergencies. (NUREG/CR-6208). 

Washington, DC: U.S. Nuclear Regulatory Commission.

Rouse, W. B. (1983). Models of human problem solving: Detection, diagnosis, and 

compensation for system failures. Automatica, 19(6), 613-625.

Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: 

Discoveries and developments. Human Factors, 50(3), 540-547. 

doi:10.1518/001872008x288457

Salas, E., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992). Toward an 

understanding of team performance and training. In R. W. Swezey & E. Salas (Eds.), 

Teams: Their training and performance. (pp. 3-29). Westport, CT US: Ablex Publishing.

Salas, E., & Fiore, S. M. (2004). Team cognition: Understanding the factors that drive process 

and performance. Washington, DC US: American Psychological Association.

Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a “Big Five” in teamwork?. Small group 

research, 36(5), 555-599.

Salmon, P. M., Stanton, N. A., Walker, G. H., Baber, C., Jenkins, D. P., McMaster, R., & Young, 

M. S. (2008). What really is going on? Review of situation awareness models for individuals 

and teams. Theoretical Issues in Ergonomics Science, 9(4), 297-323. 

doi:10.1080/14639220701561775

Sarter, N. B., & Woods, D. D. (1995). Autonomy, authority, and observability: properties of 

advanced automation and their impact on human-machine coordination. Paper presented at 

the Analysis, Design and Evaluation of Man-Machine Systems 1995. Postprint volume from 

the Sixth IFAC/IFIP/IFORS/IEA Symposium, Place of Publication: Oxford, UK; Cambridge, 

MA, USA. Country of Publication: UK.

Sasou, K., & Reason, J. (1999). Team errors: definition and taxonomy. Reliability Engineering & 

System Safety, 65(1), 1-9. doi:10.1016/S0951-8320(98)00074-X

Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in 

three paradigms suggest new concepts for training. Psychological Science, 3(4), 207-217. 

doi:10.1111/j.1467-9280.1992.tb00029.x

Shappell, S. A., & Wiegmann, D. A. (2000). The Human Factors Analysis and Classification 

System - HFACS. (DOT/FAA/AM-00/7). Washington, DC: Office of Aviation Medicine.
121

Shattuck, L. G., & Miller, N. L. (2006). Extending naturalistic decision making to complex 

organizations: A dynamic model of situated cognition. Organization Studies, 27(7), 989-

1009.

Shen, S.-H., & Mosleh, A. (1996). Human error probability methodology report, RAN: 96-002. 

Calvert Cliffs Nuclear Power Plant: BGE.

Shebilske, W. L., Goettl, B. P., & Garland, D. J. (2000). Situation awareness, automaticity, and 

training. In M. R. Endsley & D. J. Garland (Eds.), Situation awareness analysis and 

measurement. (pp. 303-323). Mahwah, NJ US: Lawrence Erlbaum Associates Publishers.

Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 

69(1), 99-118.

Simon, H. A. (1959). Theories of decision-making in economics and behavioral science. 

American Economic Review, 49, 253-283.

Smidts, C., Shen, S.-H., & Mosleh, A. (1997). The IDA cognitive model for the analysis of 

nuclear power plant operator response under accident conditions. I. Problem solving and 

decision making model. Reliability Engineering & System Safety, 55(1), 51-71.

Smith, K., & Hancock, P. A. (1995). Situation awareness is adaptive, externally directed 

consciousness. Human Factors, 37(1), 137-148. doi:10.1518/001872095779049444

Sohn, Y.-W. (2000). The role of expertise, working memory capacity, and long-term memory 

retrieval structure in situation awareness. 60, ProQuest Information & Learning, US.

Available from EBSCOhost psyh database.

Sohn, Y. W., & Doane, S. M. (2004). Memory Processes of Flight Situation Awareness: 

Interactive Roles of Working Memory Capacity, Long-Term Working Memory, and Expertise. 

Human Factors, 46(3), 461-475. doi:10.1518/hfes.46.3.461.50392

Soliman, A. M. (2010). Exploring the central executive in situation awareness. Psychological 

Reports, 106(1), 105-118. doi:10.2466/pr0.106.1.105-118

Steelman-Allen, K. S., McCarley, J. S., Wickens, C. D., Sebok, A., & Bzostek, J. (2009). NSEEV: A Computational Model of Attention and Noticing. Paper presented at the Human 

Factors and Ergonomics Society Annual Meeting Proceedings, San Antonio, TX. 

http://www.ingentaconnect.com/content/hfes/hfproc/2009/00000053/00000012/art00003

Sternberg, R. J., & Davidson, J. E. (1986). Conceptions of giftedness. New York, NY US: 

Cambridge University Press.

Swain, A. D., & Guttman, H. E. (1983). Handbook of Human Reliability Analysis with Emphasis 

on Nuclear Power Plant Applications. (NUREG/CR-1278). Washington, DC: U.S. Nuclear 

Regulatory Commission.

Szalma, J. L., Warm, J. S., Matthews, G., Dember, W. N., Weiler, E. M., Meier, A., & 

Eggemeier, F. T. (2004). Effects of Sensory Modality and Task Duration on Performance, 

Workload, and Stress in Sustained Attention. Human Factors, 46(2), 219-233. 

doi:10.1518/hfes.46.2.219.37334

Tales, A., Porter, G., & Butler, S. (2009). Automatic change detection during the performance of 

a continuous visual task. NeuroReport: For Rapid Communication of Neuroscience 

Research, 20(18), 1638-1642.

Treisman, A. (1991). Search, similarity, and integration of features between and within 

dimensions. Journal of Experimental Psychology: Human Perception and Performance, 

17(3), 652-676. doi:10.1037/0096-1523.17.3.652
122

Tsang, P. S., & Vidulich, M. A. (2006). Mental workload and situation awareness. In G. 

Salvendy (Ed.), Handbook of Human Factors and Ergonomics (3rd ed.). (pp. 243-268). 

Hoboken, NJ US: John Wiley & Sons Inc.

Tversky, A., & Fox, C. R. (1995). Weighing risk and uncertainty. Psychological Review, 102(2), 

269-283. doi:10.1037/0033-295x.102.2.269

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. 

Science, 185(4157), 1124-1131. doi:10.1126/science.185.4157.1124

Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. 

Science, 211(4481), 453-458. doi:10.1126/science.7455683

van Zoest, W., Lleras, A., Kingstone, A., & Enns, J. T. (2007). In sight, out of mind: The role of 

eye movements in the rapid resumption of visual search. Perception & Psychophysics, 

69(7), 1204-1217.

Varakin, D. A., Levin, D. T., & Collins, K. M. (2007). Comparison and representation failures 

both cause real-world change blindness. Perception, 36(5), 737-749. doi:10.1068/p5572

Vicente, K. J. (2007). Monitoring a nuclear power plant. In A. F. Kramer, D. A. Wiegmann & A. 

Kirlik (Eds.), Attention: From theory to practice. (pp. 90-99). New York, NY US: Oxford 

University Press.

Vicente, K. J., & Burns, C. M. (1996). Evidence for direct perception from cognition in the wild. 

Ecological Psychology, 8(3), 269-280. doi:10.1207/s15326969eco0803_5

Vicente, K. J., Mumaw, R. J., & Roth, E. M. (2004). Operator monitoring in a complex dynamic 

work environment: a qualitative cognitive model based on field observations. Theoretical 

Issues in Ergonomics Science, 5(5), 359-384. doi:10.1080/14039220412331298929

Vicente, K. J., Roth, E. M., & Mumaw, R. J. (2001). How do operators monitor a complex, 

dynamic work domain? The impact of control room technology. International Journal of 

Human-Computer Studies, 54(6), 831-856. doi:10.1006/ijhc.2001.0463

Vierck, E., & Kiesel, A. (2008). Change detection: Evidence for information accumulation in 

flicker paradigms. Acta Psychologica, 127(2), 309-323. doi:10.1016/j.actpsy.2007.06.004

Von Neumann, J., & Morgenstern, O. (1947). Theory of games and economic behavior (2d rev. 

ed.). Princeton, NJ US: Princeton University Press.

Vygotsky, L. S. (1960). The Development of the Higher Mental Functions. Moscow: Academy 

Pedagogical Science.

Vygotsky, L. S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.

Weick, K. E. (1995). Sensemaking in organizations: Sage Publications.

Wickens, C. D. (1976). The effects of divided attention on information processing in manual 

tracking. Journal of Experimental Psychology: Human Perception and Performance, 2(1), 1-

13. doi:10.1037/0096-1523.2.1.1

Wickens, C. D. (2001). Workload and situation awareness. In P. A. Hancock & P. A. Desmond 

(Eds.), Stress, workload, and fatigue. (pp. 443-450). Mahwah, NJ US: Lawrence Erlbaum 

Associates Publishers.

Wickens, C. D., & Hollands, J. G. (2000). Engineering Psychology and Human Performance

(3rd ed.). New Jersey: Prentice-Hall.
123

Wickens, C. D., Lee, J. D., Liu, Y., & Becker, S. E. G. (2004). An Introduction to Human Factors 

Engineering (2nd Ed): Pearson, Prentice Hall.

Xing, J. (2004). Measures of Information Complexity and the Implications for Automation 

Design. (DOT/FAA/AM-04/17). Washington, DC: Office of Aerospace Medicine.

Xing, J. (2007). Information complexity in air traffic control displays. Paper presented at the 

Human-Computer Interaction. HCI Applications and Services. 12th International 

Conference, HCI International 2007, Place of Publication: Berlin, Germany; Beijing, China. 

Country of Publication: Germany.

Xing, J., Lois, E., Chang, Y. J., & Roth, E. M. (2011). Cognitive basis for using non-domain 

participants in part-task experiments to study human errors in nuclear power plants. Paper 

presented at the Enlarged Halden Program Group Meeting; Roros, Norway.

Yates, J. F. (2003). Decision Management: How to Assure Better Decisions in Your Company. 

San Francisco, CA: Jossey-Bass.

Zajonc, R. B. (1965). Social facilitation. Science, 149(Whole No. 3681), 269-274. 

doi:10.1126/science.149.3681.269

Zsambok, C. E. (1997). Naturalistic decision making: Where are we now? In C. E. Zsambok & 

G. Klein (Eds.), Naturalistic Decision Making (pp. 3-16). New York, NY: Routledge.

A-1

Appendix A

Cognitive Mechanism Tables

This appendix contains detailed information about the behavioral and cognitive sciences 

literature reviewed to establish a psychological foundation for human reliability analysis. The 

purpose of these tables and the related cognitive framework diagrams in Appendix B is to map 

out the cognitive mechanisms and performance influencing factors (PIFs) that explain why a 

macrocognitive function may fail. 

Section A-1 contains the cognitive mechanism tables for failures of Detecting and Noticing. 

A-1.1 Cue/information not perceived

A-1.2 Cue/information not attended to

A-1.2 Cue/information misperceived

Section A-2 contains the cognitive mechanism tables for failures of Understanding and 

Sensemaking. 

A-2.1 Incorrect data

A-2.2 Incorrect integration of data, frames, or data with a frame

A-2.3 Incorrect frame

Section A-3 contains the cognitive mechanism tables for failures of Decisionmaking. 

A-3.1 Incorrect goals or priorities set

A-3.2 Incorrect internal pattern matching

A-3.3 Incorrect mental simulation

Section A-4 contains the cognitive mechanism tables for failures of Action. 

A-4.1 Failure to take required action (action not attempted)

A-4.2 Execute desired action incorrectly

Section A-5 contains the cognitive mechanism tables for failure of Teamwork. 

A-5.1 Failure of team communication

A-5.2 Error in leadership/supervision

Each section in this appendix provides a brief review of the macrocognitive function, proximate 

causes of failure of the function, and associated cognitive mechanisms. The sub-sections are 

organized by proximate cause, and include a table listing the cognitive mechanisms for each 

proximate cause. 

The first column in each table lists the cognitive mechanisms. The second column includes a

discussion and explanation of each mechanism. The third column provides a real-world 

example of the mechanism. The fourth column lists the PIFs that have been identified as 

relevant to the mechanism. The fifth column provides additional discussion of why the PIF is 

relevant. The last column contains the literature sources referenced in the row. Sometimes 

there is more than one row dedicated to each mechanism; this represents different literature 

sources that highlight different aspects of the mechanisms. 

This appendix should be used in conjunction with the cognitive framework diagrams in 

Appendix B. The diagrams are a graphical summary of the information in these tables.
A-2

A-1. DETECTING AND NOTICING

This section of Appendix A contains the cognitive mechanism tables for the Detecting and 

Noticing macrocognitive function. Detecting and Noticing is the process of perceiving important 

information in the work environment. This macrocognitive function emphasizes the sensory and 

perceptual processes that allow humans to perceive large amounts of information and focus 

selectively on those pieces of information that are pertinent to present activities.

Detecting and Noticing represents the process of detecting meaningful signals from a large bulk 

of information received by the sensory organ. Sensory information can only be retained for a 

short interval before being replaced by new sensory information. Moreover, a person can only 

attend to a limited amount of information at a time. When there is too much meaningful sensory 

information, an individual may not be able to detect and notice all of that information, resulting in 

sensory overload. Conversely, a lack of salient sensory information may cause important plant 

information to go undetected or unnoticed. When meaningful cues fail to be sensed or 

perceived it automatically results in a failure of the subsequent macrocognitive function of 

Understanding and Sensemaking. 

Chapter 3 discusses Detecting and Noticing in more detail.

In general, failure of the Detecting and Noticing macrocognitive function means that the person 

does not have the necessary information, or has incorrect information, to appropriately respond 

to the situation. Without the correct information, the operator is likely to have an incorrect 

diagnosis of the situation, and is likely to make incorrect decisions about how to respond to the 

situation. In these cases, however, the root failure is in Detecting and Noticing. Three 

proximate causes have been identified as potentially leading to this failure. Those proximate 

causes and cognitive mechanisms are:

1. Cue/Information not perceived (Table A-1.1)

a. Cue content

b. Vigilance in monitoring

c. Attention

d. Expectation

e. Working memory

2. Cue/Information not attended to (Table A-1.2)

a. Cue content

b. Vigilance in monitoring

c. Attention

d. Expectation

e. Working memory

3. Cue/Information misperceived (Table A-1.3)

a. Cue content

b. Vigilance in monitoring

c. Attention

d. Expectation

e. Working memory
A-3

A-1.1 Cue/Information Not Perceived

This table contains the cognitive mechanisms for the Cue/Information Not Perceived proximate 

cause. When this is the cause of Detecting and Noticing failure, the cue or information may 

simply be missed, not seen, or not heard, in which case it is not perceived. For example, a 

control room operator may not notice that an important indicator is trending downward. This 

proximate cause can be explained by the following cognitive mechanisms:

1. Cue content: Cue salience is low and not detected. For example, the operator fails to detect 

“low” indication because scale of trend display makes subtle changes difficult to see.

2. Vigilance in monitoring: Unable to maintain vigilance. For example, the operator is fatigued 

at the end of the night shift and has difficulty monitoring the plant. Alternatively, the operator 

must attend to several indications following a plant transient, thereby missing important 

cues.

3. Attention: Inattentional blindness—missing something that is in plain sight. For example, the 

operator may not perceive a plant indicator because it is not the focus of the operator’s 

attention.

4. Expectation: Mismatch between expected and actual cues. The operator may not notice a 

plant indicator because he or she is not looking for it. This may occur because of the 

operator’s mindset—if the operator has a particular hypothesis about the plant condition, he 

or she will look for information to confirm that hypothesis but may not equally look for 

information to disconfirm it.

5. Working memory: Working memory capacity overload. If the operator attends to multiple 

alarms simultaneously, they may reach a state of information overload in which he or she is 

focused on a particular alarm panel and may miss additional alarms or indicators elsewhere 

in the control room.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Attention Change blindness is failure 

to detect changes in the 

visual environment due to 

limitations of working 

memory. Change 

blindness is inability to 

predict the level of missed 

information and an 

overconfidence that the 

visual environment has 

been perceived correctly. 

Experiment 1 of this study 

looked at the effect of 

intentional search on 

change blindness. If 

participants were 

deliberately/intentionally 

looking for differences, 

they were significantly 

more likely to detect 

changes in the 

environmental than if they 

were not specifically 

looking for changes. 

An operator actively 

looking for changes 

in the environment/ 

instrumentation is 

more likely to detect 

such a change than 

an operator who is 

not actively looking 

for changes. 

• Intention 

(motivation)

• HSI

Participants who were 

intentionally looking for 

changes in the 

environment had 91% 

accuracy in detecting 

changes in the 

environment, while 

participants who were 

incidentally but not 

specifically looking for 

changes had 38% 

accuracy.

Beck, Levin & 

Angelone 

(2007)

Attention The potential for change 

detection failure during the 

monitoring of a military 

digital situation awareness 

map was investigated. 

If an operator is 

monitoring a 

complex group of 

indicators, the 

disappearance of a 

• HSI The ability to detect 

changes is dependent 

on the context of 

surrounding 

information. If only a 

Durlach, Kring 

& Bowens 

(2008)
A-4

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Participants were asked to 

monitor the map for icon 

appearance or 

disappearance. A change 

accompanied by two other 

changes was detected 

69.3% of the time, while 

the same change occurring 

alone was detected 79.6% 

of the time. When three 

changes occurred 

simultaneously, all three 

were detected only 37% of 

the time. Detection of icon 

appearance was superior 

to detection of icon 

disappearance. 

single indicator will 

be perceived better 

than if a group of 

indicators 

disappears.

single item was 

changed in a simplified 

display, it was detected 

79.6% of the time. If 

three changes 

occurred 

simultaneously, 

information context 

was lost, and the 

changes were detected 

only 37% of the time. 

Change detection 

varied as a function of 

whether information 

was added or deleted. 

Change detection 

performance was 

significantly higher for 

added items than for 

deleted items. 

Attention Change detection was 

monitored as a function of 

visual mismatch negativity, 

a negative reflection in the 

visual event-related 

potential evoked by 

infrequent deviant stimuli. 

The study found that 

automatic change 

detection continued 

despite high workload. 

However, as the frequency 

of the change was 

increased, the sensitivity of 

the brain to the change 

decreased. 

If an operator sees a 

frequent change, he 

or she may become 

desensitized to it. 

This effect is seen in 

nuisance alarms.

• HSI Increasing frequency 

of visual mismatch 

results in a decreased 

brain response to 

change.

Tales, Porter 

& Butler 

(2009)

Attention Change detection exists as 

a method of testing human 

perception through rapid 

presentation of alternating 

images. Across three 

experiments, this paper 

offers insights on the 

optimal presentation for 

rapid presentation of 

alternating images. It was 

found that increasing the 

number of presentations 

strengthens the ability of 

participants to detect 

changes, as does 

maintaining an alternating 

source-target order for 

presenting images. While 

primarily a methodological 

improvement study, the 

article highlights the 

importance of training to 

improved change 

detection. 

Because of training, 

an operator may 

learn to detect 

changes in 

indications more 

quickly than without 

training.

• HSI Color and position of 

changes did not vary 

significantly in time to 

detect, but presence 

differences (appear 

versus disappear) 

were quickest to be 

detected. Training on 

change detection 

significantly decreased 

detection time.

Vierck & 

Kiesel (2008)

Attention This experiment reviewed 

the performance of 

participants for change 

detection across different 

situations. The primary 

situational distinction 

involved top-down versus 

An operator who 

employs an active 

search strategy 

guided by attentional 

focus is more likely 

to detect changes to 

instrumentation than 

• HSI: Type of 

cue salience

• Motivation: 

Type of strategy 

used to detect 

changes

Those participants 

employing a top-down 

strategy were 9% more 

accurate at change 

detection than those 

relying on bottom-up 

cues. Top-down 

Bubić (2008)
A-5

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

bottom-up processes. 

Top-down here is defined 

as a cue specifically 

requiring attentional 

selection, while bottom-up 

is defined as a salient cue 

that does not require 

specific attentional 

mechanisms to draw focus. 

The results showed that 

change detection was 

more effective when 

relying on a top-down 

strategy of focused 

attention than when relying 

on salient cues from the 

environment. 

one who relies on 

the salience of the 

instrumentation 

cues.

participants were also 

on average .74 

seconds faster at 

change detection than 

bottom-up participants.

Attention If a person's attention is 

locked onto a particular 

location or object, then the 

onset of signals in different 

locations is less likely to 

attract attention. 

Operator focuses on 

a specific indicator 

and misses 

additional 

information 

elsewhere on the 

panel.

• Attention Perceptual focus 

determines what is 

perceived

Nikolic, Orr, & 

Sarter (2004)

Vigilance in 

monitoring

In this model, vigilance or 

sustained attention is 

resource demanding and 

effortful, and humans have 

only a limited amount of 

“attentional resources.” As 

attentional resources are 

used, stress levels can 

increase over time 

because the requirements 

to remain vigilant are 

unrelenting. Once those 

attentional resources are 

depleted, our ability to 

maintain attention or be 

vigilant is compromised.

When attentional 

resources are 

depleted, or being 

used for other tasks, 

even a clearly 

presented cue may 

not be perceived.

• All of the PIFs 

listed under the 

category 

"Personal PIFs"

Szalma et al. 

(2004)

MacLean et al. 

(2009)

Davies & 

Parasuraman 

(1982)

Vigilance in 

monitoring

Attention as a 

psychological construct (a 

construct is a hypothetical 

explanatory variable which 

is not directly observable) 

is a key aspect of detect/ 

notice. External stimulus or 

stimuli that attract our 

attention also tends to 

guide our visual gaze (e.g., 

a flashing light; even a loud 

noise tends to cause us to 

look in the direction from 

which the sound 

emanated).

A cue such as 

flashing light, even a 

loud noise tends to 

cause us to look in 

the direction from 

which the sound 

emanated. However, 

when the stimuli 

does not have 

sufficient "activation 

energy" that exceeds 

a biologically 

determined threshold 

that triggers a 

sensory response in 

us, we fail to 

perceive the cue.

• HSI The external stimulus 

or stimuli must have an 

"activation energy" that 

exceeds a biologically 

predetermined 

threshold for 

detecting/noticing/sens

ing objects in the world 

around us, causing our 

attention to focus on 

that stimulus.

Lavine et al. 

(2002)

Vigilance in 

monitoring

The presence of a 

demanding central task 

effectively narrows or 

"tunnels" the functional 

field of view, making it 

more difficult to extract 

information from the 

periphery. 

When an operator is 

engaged in a 

cognitively 

demanding task, he 

is less likely to notice 

cues that are in the 

periphery of his 

visual focus.

• Task load Refers to visual cues. 

Mental workload 

(anything that loads 

the central executive) 

creates attentional 

narrowing.

Nikolic, Orr, & 

Sarter (2004)
A-6

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Vigilance in 

monitoring

The sympathetic response 

is the action to mobilize the 

body's resources under 

stress to induce the 

fight-or-flight response. It is 

what causes people to 

become more anxious 

under stressful conditions. 

1. Not being 

sufficiently stressed 

causes an operator 

to not have sufficient 

attentional resources 

at the ready to 

perceive cues.

2. Too many cues 

being presented 

simultaneously 

causes one to be 

over-stressed, and in 

this highly anxious 

state, an operator 

may not perceive a 

critical cue amongst 

the multitude of other 

cues

• Stress Depending on the 

magnitude of the 

sympathetic response, 

an individual may not 

be sufficiently 

stressed, or 

over-stressed. Not 

being sufficiently 

stressed may lead to 

the cue not being 

perceived. Being 

over-stressed, for 

example, by an alarm 

flooding situation, may 

lead to a critical cue 

not being perceived.

Aarts & 

Pourtois 

(2010)

Vigilance in 

monitoring

In this experiment, the 

interaction of the central 

executive (related to 

working memory) and 

situation awareness was 

explored. Participants for 

low, medium, and high 

situation awareness were 

given a driving task. 

Participants with low 

situation awareness 

performed significantly 

worse at tasks involving 

the central executive. 

When taxing the central 

executive with increased 

workload, the low situation 

awareness group 

committed significantly 

more driving errors than 

the high situation 

awareness group. The 

driving errors were related 

to cue detection, primarily 

in the form of detecting 

information too late to 

respond properly in the 

driving simulator, resulting 

in a simulated impact or 

road departure.

Operators may have 

different levels of 

situation awareness, 

and individual 

operators' situation 

awareness may be 

impacted by external 

factors such as 

stress. For low 

situation awareness, 

the operators' ability 

to detect relevant 

cues is particularly 

diminished when 

there is high 

workload.

• Task load Individual difference in 

situation awareness. 

Beyond that, a task 

involving increased 

workload in working 

memory/central 

executive will result in 

significant decrease in 

performance on 

detection-related tasks.

Soliman 

(2010)
A-7

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Vigilance in 

monitoring

Humans are remarkably 

adaptable. Humans can 

cope with the constraints of 

their environment through 

strategically adapting their 

behavior to maximize their 

success (survivability). In 

the context of NPPs, 

operators implement 3 

strategies to facilitate their 

task performance (e.g., 

monitoring). They (a) 

enhance information 

extraction by increasing 

the salience of important 

indicators and reducing the 

background “noise,” (b) 

create new information, 

and (c) offload some of the 

cognitive processing onto 

the interface (e.g., creating 

external aids and 

reminders for monitoring).

Operators may (a) 

enhance information 

extraction by 

increasing the 

salience of important 

indicators and 

reducing the 

background “noise,” 

(b) create new 

information, and (c) 

offload some of the 

cognitive processing 

onto the interface 

(e.g., creating 

external aids and 

reminders for 

monitoring). 

• Morale/ 

motivation/ 

attitude

If any of the 3 

strategies the 

operators use fail, it 

can lead to a cue not 

being perceived.

Mumaw et al. 

(2000)

Vicente et al. 

(2001)

Vicente et al. 

(2007)

Vigilance in 

monitoring

Szalma et al. (2004) 

showed that the interaction 

of factors such as sensory 

modality and workload 

(time on task) affect stress 

levels differentially. Stress 

can either be exacerbated 

or attenuated depending 

on how the factors 

interacted with each other.

Observers became 

more stressed over 

time, with evidence 

of recovery in the 

auditory but not the 

visual condition 

toward the end of the 

watch.

• Stress

• Task load

As stress and workload 

increase, ability to 

perceive or attend 

cues decreases

Szalma et al. 

(2004)

Vigilance in 

monitoring

Perceptual Control Theory 

(PCT) states that people 

act on their surroundings, 

the environment, so as to 

control the effects the 

environment is having on 

them. In PCT, organisms 

generate actions affecting 

the environment near 

them, thus altering the 

environment and creating 

or changing experiences at 

many levels in the way 

desired by the organism.

PCT relates to the findings 

from Mumaw et al. (2000), 

Vicente et al. (2001), and 

Vicente (2007) in that what 

they describe as the key 

"adaptable" behaviors 

operators engage in, 

namely: (a) enhance 

information extraction by 

increasing the salience of 

important indicators and 

reducing the background 

“noise,” (b) create new 

information, and (c) offload 

some of the cognitive 

processing onto the 

interface, are all examples 

of the operator attempt to 

exert more control on their 

Operators may (a) 

enhance information 

extraction by 

increasing the 

salience of important 

indicators and 

reducing the 

background “noise,” 

(b) create new 

information, and (c) 

offload some of the 

cognitive processing 

onto the interface 

(e.g., creating 

external aids and 

reminders for 

monitoring). 

• Morale/ 

motivation/ 

attitude

If any of the 3 

strategies the 

operators use fail, it 

can lead to a cue not 

being perceived.

Powers (1973)

Hendy et al. 

(2001)

Mumaw et al. 

(2000)

Vicente et al. 

(2001)

Vicente et al. 

(2007)
A-8

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

environment so as to 

improve their performance 

(or minimize the likelihood 

of committing an error)

Vigilance in 

monitoring

Failure to perceive the 

status, attributes, and 

dynamics of relevant 

elements in the 

environment. 

Distractions by other 

relevant tasks, an 

overall high level of 

workload, 

distractions by 

unrelated 

situations/tasks, 

vigilance problems, 

or overreliance on 

automation. 

• Attention (to 

task and to 

surroundings)

• Physical & 

psychological 

abilities

• Task load

• Non-task loads 

Decrease in perception 

due to PIFs

Endsley 

(1995)

Vigilance in 

monitoring

The relationship between 

workload and vigilance is 

similar to that of stress and 

attention. 

1. Tasks too low in 

complexity cause an 

operator to not be 

engaged in the task 

and have sufficient 

attentional resources 

at the ready to 

perceive cues.

2. Tasks too high in 

complexity may 

cause an operator to 

not perceive a critical 

cue amongst the 

multitude of other 

cues

• Task complexity Depending on the 

task's complexity, an 

individual may not be 

sufficiently engaged to 

perceive to the cue, or 

over-worked. 

Donald (2001)

Liu et al. 

(2009)

Vigilance in 

monitoring

Hitchcock et al. (2003) 

found a main effect for type 

of automated warning cue: 

correct detection of critical 

signals was best with 

warning cues of 100% 

reliability and progressively 

poorer as warning cue 

reliability decreased to 

80%, 40%. Correct 

detection performance was 

worst with no automated 

warning cue.

Operators are less 

likely to detect 

warning cues that 

are unreliable.

Refers to visual cues. 

Warning cues that are 

highly reliable lead to 

significantly better 

operator performance 

than warning cues that 

are unreliable or 

unavailable.

Hitchcock et 

al. (2003)

Vigilance in 

monitoring

The relationship between 

workload and vigilance is 

similar to that of stress and 

attention. 

1. Not having 

sufficient workload 

causes an operator 

to not be engaged in 

the task and have 

sufficient attentional 

resources at the 

ready to perceive 

cues.

2. Too many cues 

being presented 

simultaneously 

causes one's 

workload to be too 

high, and in this 

state, an operator 

may not perceive a 

critical cue amongst 

the multitude of other 

cues

Depending on the 

amount of work being 

loaded on to the 

operator, an individual 

may not be sufficiently 

engaged to perceive 

the cue, or 

over-worked. 

Donald (2001)

Liu et al. 

(2009)

Cue 

content

The context in which the 

target cue is displayed 

influences the likelihood of 

If the cue that 

operators need to 

attend to is the same 

• HSI Refers to visual cues. 

Cues surrounded by 

other dynamic 

Nikolic, Orr, & 

Sarter (2004)
A-9

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

detection. Specifically, 

targets embedded in 

same-color or dynamic 

displays were the most 

difficult to detect (targets in 

color AND dynamic 

displays had marginally 

worse detection rates). The 

detection rate for the 

color-dynamic displays 

was 32 percentage points 

lower than the control, and 

20 percent lower than 

mono-static displays. The 

combination of color and 

motion surrounding the 

target cue had the most 

detrimental effect on 

detection performance. 

Note that this study 

involved identifying target 

cues in peripheral vision 

when the participant was 

engaged in a cognitively 

demanding primary task.

color as irrelevant 

information in the 

same display, or if it 

is embedded in other 

dynamic indicators, 

they are less likely to 

notice the cue.

information, such as 

moving dials or 

indicators also in the

same color as the 

target cue are less 

likely to be noticed 

when the person is 

attending to another 

task. The poor 

detection performance 

may be the result of 

masking by 

surrounding moving 

display elements, the 

color similarity 

between target and 

background, or a 

combination.

Cue 

content

The SEEV model is based 

on the plausible 

assumption that four 

factors drive the acquisition 

of visual information: the 

salience (S) of events that 

might capture attention; the 

effort (E) required to 

redirect attention from one 

location to another (i.e., 

visual saccade, head 

rotation), which will inhibit 

information access; the 

expectancy (E) that a given 

location in the visual field 

will contain information; 

and the value (V) of 

information to be obtained 

at that location for the task 

or tasks at hand.

The four 

criteria/factors in this 

model must be met 

in order for a cue to 

be perceived. In 

particular, the 

salience of the cue is 

critical for perception 

to occur.

• HSI If the cue is not 

presented to the 

operator with sufficient 

strength/energy to 

distinguish itself from 

existing background 

noise such that it 

activates a sensory 

response in the 

operator, the operator 

may not perceive it.

Steelman-Alle

n et al. (2009)

Cue 

content

The consistency, 

predictability, 

distinctiveness, and 

interpretability of the signal 

affect vigilance.

1. If the cue is not 

consistently 

presented, the 

operator may not 

perceive it.

2. If the cue is 

unpredictable, the 

operator may not 

perceive it.

3. If the cue is not 

presented to the 

operator with 

sufficient 

strength/energy to 

distinguish itself from 

existing background 

noise such that it 

activates a sensory 

response in the 

operator, the 

operator may not 

perceive it.

• HSI Cue qualities 

(consistency, 

predictability/reliability, 

and strength) affect 

perception

Donald (2001)

Liu et al. 

(2009)

MacLean et al. 

(2009)
A-10

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Cue 

content

The human's ability to 

detect or sense a stimuli is 

a function of (1) how 

clearly that stimuli is 

present over and above 

existing background noise 

(e.g., the cue's signal to 

noise ratio), and (2) the 

extent to which the human 

is willing to accept an error 

of omission versus an error 

of commission. 

The cue is not 

presented to the 

operator with 

sufficient 

strength/energy to 

distinguish itself from 

existing background 

noise such that it 

activates a sensory 

response in the 

operator.

• HSI

• Workload

• Task complexity

• Stress

The signal to noise 

ratio is too low for the 

operator to be able to 

perceive the cue (i.e., 

distinguish it from 

noise)

Bustamante 

(2008)

Cue 

content

Whether the signal is 

successive (serial) or 

simultaneous, and whether 

it is sensory or cognitive 

affects vigilance. 

1. If cues are 

presented serially, 

but there is not 

adequate time to 

perceive each one 

before the next is 

presented can cause 

the operator to not 

perceive the cue.

2. If two cues are 

presented 

simultaneously, the 

operator may not be 

able to perceive 

both.

• HSI Cue presentation order 

affects perception

Donald (2001)

Liu et al. 

(2009)

Expectation Expectations of a particular 

type of signal, such as 

onsets, offsets, or color 

changes, will increase the 

likelihood of that particular 

cue to capture attention. In 

other words, visual onsets 

per se do not necessarily 

capture attention. Instead, 

the likelihood of detection 

depends on the match 

between a person's active 

attention control settings 

and properties of the 

appearing signal. 

Operator expects a 

particular indicator 

and misses a 

contrary indication 

elsewhere on the 

control panel.

• Training, 

experience, 

bias

Refers to visual cues. 

Training/experience 

leads to expectations 

or bias about the type 

of cues to watch for.

Nikolic, Orr, & 

Sarter (2004)

Expectation Van Zoest et al. (2007) 

found that people are able 

to complete a search much 

faster if they had a prior 

look at the display (i.e., 

they are able to complete 

an interrupted search 

faster than starting a new 

search). Rapid resumption 

of the search is based on 

target pre-processing and 

hypothesis formation in the 

last look before locating 

the target.

Operators know what 

information they are 

looking for, and they 

know where it is 

located, because 

they are well-trained 

on the control panel 

layout. However, in 

diagnosing a 

problem, they may 

perform a visual 

scan of control 

boards. Scanning 

without interruptions 

results in quicker 

diagnosis.

• Experience/

Training

Refers to visual cues 

and scanning.

Van Zoest 

(2007)

Working 

memory

Broadbent’s “filter theory” 

explains sensory 

bottlenecks—the alarm 

might be missed because 

humans have a limited 

ability to take in stimulus 

inputs and perceive them. 

It is possible to reach 

Filter theory has 

direct implications for 

plant environments 

like control rooms, 

where concurrent 

tasks and multiple 

simultaneous alarms 

may trigger sensory 

• Task load Sensory bottlenecks 

dictate ability to 

perceive information.

Broadbent 

(1958)
A-11

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

sensory bottlenecks, which 

can result in sensory 

overload.

overload and lead to 

missing information.

Working 

memory

Suggests that sensory 

information is held in very 

short-term memory. If that 

sensory information is not 

attended to, it will be lost. 

The implication is 

that plant information 

may be lost if the 

operator does not 

attend to it. If the cue 

is not presented in 

sufficient 

(strength/length/salie

nce) for the operator 

to register it, it is lost.

• HSI, hardware 

failure (e.g., 

bulb burned out 

upon 

illuminating)

Strength, length, and 

salience of cues affect 

ability to perceive.

Neisser (1967)

Working 

memory/Vig

ilance in 

monitoring

An experiment is 

presented that 

demonstrates the effect of 

working memory on 

vigilance. Using a 

visiospatial monitoring 

task, participants were 

given either a task that 

taxed visiospatial working 

memory or nonspatial 

working memory. Over a 

20-minute period, vigilance 

decreased significantly for 

the task using visiospatial 

working memory but not for 

the nonspatial working 

memory.

If an operator must 

maintain visual 

information in 

working memory and 

monitor visual 

indicators, the 

overlap of working 

memory and 

perception will result 

in a high workload, 

leading to mental 

fatigue and 

decreased vigilance.

• Task load There was a significant 

interaction of working 

memory type on 

vigilance. Vigilance 

decreased significantly 

over time for a type 

match between 

information in working 

memory and 

perceptual cues. In 

other words, a large 

degree of overlap 

resulted in working 

memory interference 

and increased 

workload, decreasing 

the amount of time the 

operator could 

maintain vigilance.

Caggiano & 

Parasuraman 

(2004)
A-12

A-1.2 Cue/Information Not Attended To

This table contains the mechanisms for the Cue/Information Not Attended To proximate cause. 

When this is the cause of Detecting and Noticing failure, the cues or information may be sensed 

and perceived but not attended to—in other words, the sensory-perceptual system acts as a 

filter, and this information is not propagated for further Understanding and Sensemaking. For 

example, during an incident at the plant, a control room operator may be confronted with an 

alarm flood. The raw sensory cues corresponding to the individual alarms are present, but the 

operator experiences sensory overload and cannot consciously perceive or process the large 

number of simultaneous cues. Or, the operator’s attention may be focused on a particularly 

important part of plant recovery and choose not to respond to certain alarms until a more 

pressing issue is addressed.

1. Cue content: Too many meaningful cues. For example, during a plant transient, the 

operator may have multiple simultaneous indications, such as an alarm flood involving 

hundreds of annunciators, and may not be able to attend to all indicators simultaneously. 

This mechanism focuses on the content and amount of the information.

2. Vigilance in monitoring: Divided attention. For example, if the operator has to respond to 

multiple simultaneous alarms, the operator may reach a state of so-called information 

overload or alarm flood in which he or she is unable to attend to additional alarms. Also, the 

operator may be fatigued at end of night shift and have difficulty monitoring the plant. 

Alternatively, the operator may be stressed, leading to shortened attentional focus. In 

vigilance in monitoring for Cue/Information Not Perceived, the operator simply misses 

important information. Here, the operator is aware of new cues like alarms but cannot 

attend to or address it.

3. Attention: Change blindness. For example, the operator may not notice a change in a plant 

indicator because the operator has missed the prior state of the indicator. In inattentional 

blindness for Cue/Information Not Perceived, the operator misses detecting a change in an 

indicator that is not his or her current focus. In contrast, change blindness occurs when the 

operator is focused on monitoring an indicator and simply fails to notice the change. 

4. Expectation: Overreliance on primary indicator. The operator may be monitoring a particular 

plant indicator to the exclusion of other meaningful indicators that should or need to be 

monitored. This often occurs because of the operator’s mindset—if the operator has a 

particular hypothesis about the plant condition, he or she will look for information to confirm 

that hypothesis and may discard information to disconfirm it.

5. Working memory: Working memory capacity overflow. This mechanism focuses on the 

operator’s limited working memory capacity. For example, if the operator has to respond to 

multiple simultaneous alarms, the operator may reach a state of so-called information 

overload or alarm flood in which he or she is focused on a particular alarm panel and may 

disregard additional alarms or indicators elsewhere in the control room. The distinction with 

working memory for Cue/Information Not Perceived involves the extent the operator is 

aware of information that must be attended to. In working memory for Cue/Information Not 

Perceived, the operator misses information because his or her attention and working 

memory are elsewhere. Here, the operator is aware of the additional alarms, but simply 

cannot dedicate resources to processing additional alarms.
A-13

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Cue content The consistency, 

predictability, 

distinctiveness, and 

interpretability of the signal 

affect vigilance.

If the cue is difficult 

to interpret and 

repeatedly 

presented, the 

operator may not 

attend to the next 

time the signal is 

presented.

• HSI Cue qualities 

(consistency, 

predictability/reliability, 

and strength) affect 

perception

Donald (2001), 

Liu et al. (2009),

MacLean et al. 

(2009)

Vigilance in 

monitoring

Szalma et al. (2004) 

showed that the interaction 

of factors such as sensory 

modality and workload 

(time on task) affect stress 

levels differentially. Stress 

can either be exacerbated 

or attenuated depending 

on how the factors 

interacted with each other.

Observers became 

more stressed over 

time, with evidence 

of recovery in the 

auditory but not the 

visual condition 

toward the end of 

the watch.

• Stress

• Workload

As stress and workload 

increase, ability to 

perceive or attend 

cues decreases

Szalma et al. 

(2004)

Vigilance in 

monitoring

Fatigue impairs our 

higher-level cognitive 

processes and our ability 

to perceive stimuli. 

When the operator 

is fatigued, they 

may not attend to 

cues being 

presented

• Fatigue Fatigue impairs our 

higher-level cognitive 

processes and our 

ability to attend to 

stimuli.

Donald (2001); 

MacLean, et al., 

(2009)

Vigilance in 

monitoring

The SEEV model is based 

on the plausible 

assumption that four 

factors drive the acquisition 

of visual information: the 

salience (S) of events that 

might capture attention 

(Yantis, 1993); the effort 

(E) required to redirect 

attention from one location 

to another (i.e., visual 

saccade, head rotation), 

which will inhibit 

information access 

(Wickens, 1993); the 

expectancy (E) that a given 

location in the visual field 

will contain information 

(Senders, 1964); and the 

value (V) of information to 

be obtained at that location 

for the task or tasks at 

hand (Sheridan, 1970).

The four 

criteria/factors in 

this model must be 

met in order for a 

cue to be attended 

to. Salience of the 

cue is a necessary, 

but not sufficient 

criteria for attending 

to the cue. 

Furthermore, any 

one of the 

remaining 

criteria/factors on its 

own can be 

responsible for the

operator failing to 

attend to the cue.

• All of the 

PIFs listed 

under the 

category 

"Personal 

PIFs"

1. If the cue is 

presented in such a 

way that it is difficult for 

the operator to 

change/move their 

focus of attention to it, 

the cue may not be 

attended to.

2. If the operator's 

expectancy for the cue 

and the presentation of 

the cue do not match, 

the operator may not 

attend to the cue.

3. If the operator 

believes the cue does 

not add value to their 

understanding of the 

system and/or 

situation, they may not 

attend to the cue.

Yantis (1993), 

Wickens (1993), 

Senders (1964), 

Sheridan (1970), 

Steelman-Allen 

et al. (2009), 

Wickens et al. 

(2003)

Vigilance in 

monitoring

Whether the signal is 

successive (serial) or 

simultaneous, and whether 

it is sensory or cognitive 

affects vigilance. 

1. If cues are 

presented serially, 

but there is not 

adequate time to 

attend to each one 

before the next is 

presented can 

cause the operator 

to not attend to the 

cue.

2. If two cues are 

presented 

simultaneously, the 

operator may not be 

able to attend to 

both.

• HSI Cue presentation order 

affects perception

Donald (2001); 

Liu, et al. (2009)

Vigilance in 

monitoring

Whether the signal is 

coming from a single 

source or multiple sources 

affect vigilance. The 

1. Operators may 

not be able to 

attend to all 

signals/cues 

• HSI 1. Operators' vigilance 

is related to attention, 

and as a general rule, 

tasks requiring split 

Donald (2008), 

Xing (2007), Levi 

(2009)
A-14

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

complexity of information 

being presented on a 

display affects 

performance. Visual 

crowding creates an 

information-processing 

bottleneck for recognizing 

(perceiving) objects in 

peripheral vision.

coming from 

multiple sources.

2. Operators may 

not be able to 

attend to all 

signals/cues 

coming from a 

cluttered display.

3. Operators may 

not attend to cues 

from displays that 

are in their 

peripheral vision.

attention can lead to 

cues/information not 

being attended to.

2. Cues coming from 

cluttered displays may 

not be salient enough 

for the operator to 

attend to them.

3. In crowding, the 

cue’s target and flank 

features are detected 

independently and, 

when both fall within 

the “integration field”, 

they are merged into a 

percept that is often 

described as jumbled 

or indistinct.

Vigilance in 

monitoring

The relationship between 

workload and vigilance is 

similar to that of stress and 

attention. 

1. Workload being 

too low to keep the 

operator engaged 

causes an operator 

to not have 

sufficient attentional 

resources at the 

ready to attend to 

cues.

2. Too many cues 

being presented 

simultaneously 

causes the 

operator's workload 

to be too high, and 

in this state, an 

operator may not 

attend to a critical 

cue amongst the 

multitude of other 

cues.

• Workload Depending on the 

amount of work being 

loaded on to the 

operator, an individual 

may not be sufficiently 

engaged to attend to 

the cue, or 

over-worked. 

Donald (2001), 

Liu, et al. (2009)

Vigilance in 

monitoring, 

Cue 

content, 

Attention

The human's ability to 

detect or sense, and then 

attend to a stimuli is a 

function of (1) how clearly 

that stimuli is present over 

and above existing 

background noise (e.g., the 

cue's signal to noise ratio), 

(2) the extent to which the 

human is willing to accept 

an error of omission versus 

an error of commission, 

and (3) the human's 

attentional capacity at the 

time the cue is presented. 

1. An operator that 

has responded 

frequently to false 

alarms may choose 

not to attend to a 

subsequent alarm.

2. The cue is 

presented with 

sufficient 

strength/energy to 

distinguish itself 

from existing 

background noise, 

but it is not 

presented long 

enough for the 

operator to attend 

to it.

• HSI

• Workload

• Task 

complexity

• Stress

• Fatigue

1. The operator has 

changed the threshold 

to which he/she is 

willing to accept an 

error of commission 

because the system's 

signal to noise ratio is 

so low that the 

probability of false 

alarms is unacceptably 

high.

2. Operator misses 

cue—it is perceived 

but not attended to 

before the cue 

disappears.

Bustamante 

(2008)

Expectation Confirmation bias: People 

tend to seek out evidence 

that confirms their current 

position and to disregard 

evidence that conflicts with 

their current position. 

Operator seeks 

information to 

confirm his or her 

mental model of 

plant condition.

• Bias

• HSI

The greater the 

congruence between 

environmental cues 

and mental model 

(information being 

sought), the more the 

information will be 

attended to.

Einhorn & 

Hogarth (1978)
A-15

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Working 

memory

Research demonstrates 

the effect of cue relevance 

for high and low working 

memory conditions on 

formulating the intent to 

complete an action that 

cannot be completed 

immediately. A high focal 

task is one that provides 

readily detectable cues 

that reinforce the planned 

action, whereas a low focal 

task is one that does not 

readily reinforce the action.

According to the working 

memory model, high focal 

tasks should require 

minimal working memory 

because the perceptual 

cues reinforce memory. 

Participants with high and 

low working memory 

capacity performed equally 

well on the high focal task, 

but participants with a low 

working memory capacity 

performed much better for 

focal tasks than for 

nonfocal tasks. This finding 

confirmed the theory that 

low cue focality was 

affected by working 

memory capacity.

An operator who is 

monitoring several 

plant states 

concurrently will be 

cued on appropriate 

tasking if the 

instrumentation 

reinforces the 

action required. 

Otherwise, 

additional working 

memory will be 

required to attend to 

the instrumentation, 

and both 

instrumentation 

cues and follow on 

tasking may be 

missed.

• HSI

• Workload

Participants with low 

working memory 

capacity saw a 20% 

performance 

decrement for nonfocal 

cues.

Brewer (2010)
A-16

A-1.3 Cue/Information Misperceived

This table contains the mechanisms for the Cue/Information Misperceived proximate cause. 

When this is the cause of Detecting and Noticing failure, the cues or information may be sensed 

but misperceived. In other words, the sensed information is tagged with the incorrect meaning. 

For example, a control room operator may misread a pump as being on when, in fact, it is off. 

Or, the operator may mis-read an indicator by transposing the numbers.

1. Cue content: Cues are too complex. For example, during a plant transient, the operator 

may have multiple simultaneous indications that have multiple facets or dimensions, and 

may not be able to attend to all indicators simultaneously. The operator may arrive at the 

wrong conclusion about the plant state because he or she is unable to interpret the entire 

cue content.

2. Vigilance in monitoring: Degraded focus. The operator may have high workload, high 

fatigue, or high stress, which can cloud judgment and diagnosis, leading to a 

misinterpretation of a cue about plant status. Alternately, during a period of sustained high 

workload involving a high focus of attention, vigilance in scanning, detecting, or monitoring 

degrades to the point where operators misperceive available cues. 

3. Attention: Inattentional blindness. The technical term for the phenomenon of missing 

something that is in plain sight. For example, the operator may misperceive obvious plant 

information because that information is not the focus of the operator’s attention. In this 

case, the operator may fail to integrate important information that would otherwise be 

obvious (e.g., changing trends, because the operator is focused on understanding different 

information). The distinction between Attention for Cue/Information Not Perceived and here 

deals with how inattentional blindness manifests. The operator may simply not perceive 

changed but important information in Cue/Information Not Perceived, whereas here they 

may misinterpret it.

4. Expectation: Mismatch between expected and actual cues. The operator may be monitoring 

a particular plant indicator to the exclusion of other meaningful indicators that should or 

need to be monitored, leading to a misperception of a cue about plant status. This may 

occur because of the operator’s mindset—if the operator has a particular hypothesis about 

the plant condition, he or she will look for information to confirm that hypothesis and may 

discard or miss information to disconfirm it.

5. Working memory: Memory segmenting error. Information may be parsed or processed 

incorrectly, leading to a misperception of the plant state. This can occur especially in 

response to high task load or when the operator is distracted. Based on his or her 

expectations for the progression of a plant transient, the operator may fill in missing or 

expected information, leading to the wrong perception.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Attention, 

Working 

memory, 

Cue content

Template-matching 

theory accounts for the 

perceptual match 

between a stimulus and 

a pattern in memory. 

Items in memory serve 

as templates to which 

stimuli are compared. 

The theory accounts 

for the difficulty 

operators may have in 

mapping the HSI—or a 

specific plant 

configuration—to a 

mental template. The 

theory also accounts 

for misreading 

indicators when there 

is a mis-map between 

the stimulus and the 

mental template.

• HSI Proper cue perception 

depends on the degree 

the information being 

perceived matches 

previously experienced 

information

Phillips 

(1974)
A-17

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Vigilance in 

monitoring

The sympathetic 

response is the action to 

mobilize the body's 

resources under stress 

to induce the 

fight-or-flight response. 

It is what causes people 

to become more anxious 

under stressful 

conditions. 

Under high stress, an 

operator may 

misperceive a cue. For 

example, a pilot 

mistaking the out of 

fuel light with the 

intercom light.

• Stress The stress response 

being caused by the 

autonomic nervous 

system can interfere 

with higher level 

cognitive processing of 

information. In effect, 

the autonomic 

response overrides 

rational thinking

Aarts & 

Pourtois 

(2010)

Vigilance in 

monitoring

Fatigue impairs our 

higher-level cognitive 

processes and our 

ability to perceive 

stimuli. 

When the operator is 

fatigued, they may 

misperceive cues being 

presented.

• Fatigue Fatigue impairs our 

higher-level cognitive 

processes and our 

ability to perceive 

stimuli correctly.

Donald 

(2001), 

MacLean, et 

al. (2009)

Vigilance in 

monitoring

The SEEV model is 

based on the plausible 

assumption that four 

factors drive the 

acquisition of visual 

information: the salience 

(S) of events that might 

capture attention 

(Yantis, 1993); the effort 

(E) required to redirect 

attention from one 

location to another (i.e., 

visual saccade, head 

rotation), which will 

inhibit information 

access (Wickens, 1993); 

the expectancy (E) that 

a given location in the 

visual field will contain 

information (Senders, 

1964); and the value (V) 

of information to be 

obtained at that location 

for the task or tasks at 

hand (Sheridan, 1970).

Failing to meet any one 

of the four 

criteria/factors in this 

model can lead to a 

cue being 

misperceived.

• All of the PIFs 

listed under the 

category 

"Personal PIFs"

1. If the cue is not 

presented to the 

operator with sufficient 

strength/energy to 

distinguish itself from 

existing background 

noise such that it 

activates a sensory 

response in the 

operator, the operator 

may misperceive it.

2. If the cue is 

presented in such a 

way that it is difficult for 

the operator to 

change/move their 

focus of attention to it, 

the cue may be 

misperceived.

3. If the operator's 

expectancy for the cue 

and the presentation of 

the cue do not match, 

the operator may 

misperceive the cue.

4. If the operator 

believes the cue does 

not add value to their 

understanding of the 

system and/or 

situation, they may 

misperceive the cue.

Yantis 

(1993), 

Wickens 

(1993), 

Senders 

(1964), 

Sheridan 

(1970), 

Steelman-All

en et al. 

(2009), 

Wickens et 

al. (2003)

Vigilance in 

monitoring

Whether the signal is 

coming from a single 

source or multiple 

sources affect vigilance.

The complexity of 

information being 

presented on a display 

affects performance.

Visual crowding creates 

an

information-processing 

bottleneck for 

recognizing (perceiving) 

objects in peripheral 

vision.

1. Operators may 

misperceive some 

signals/cues coming 

from multiple sources.

2. Operators may 

misperceive 

signals/cues coming 

from a cluttered 

display.

3. Operators may 

misperceive cues from 

displays that are in 

their peripheral vision.

• Attention

• HSI

1. Operators' vigilance 

is related to attention, 

and as a general rule, 

tasks requiring split 

attention can lead to 

cues/information being 

misperceived.

2. Operators may 

misperceive cues from 

cluttered displays that 

are not salient enough.

3. In crowding, the 

cue’s target and flank 

features are detected 

independently and, 

when both fall within 

the “integration field,” 

they are merged into a 

Donald 

(2008) 

Xing (2007), 

Levi (2009)
A-18

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

percept that is often 

described as jumbled 

or indistinct.

Cue content This study used 

multi-dimensional (up to 

ten dimensions) visual 

icons to test whether 

participants could 

accurately detect the 

state of the various 

dimensions (binary) of 

the icons. They found 

that as the number of 

dimensions increased, 

sensitivity (number of 

correct hits per number 

of times the dimension 

was present) decreased 

exponentially (no 

specific numbers 

provided, but from the 

trend graph, it appears 

that sensitivity dropped 

from 100% to about 95% 

as the number of 

dimensions increased 

from 0 to 10). 

As the complexity of 

the cue increases, the 

less likely operators 

are to correctly identify 

crucial aspects of the 

cue.

• HSI Refers to visual cues. 

The quality and 

amount of information 

provided in the cue has 

an effect on whether 

the cue will be 

accurately detected. 

Specifically, the more 

complex the cue, the 

less likely operators 

are to correctly 

recognize specific 

parts of the cue.

Repperger 

et al. (2007)

Cue content Feature-integration 

theory. Counterpart to 

feature analysis and 

recognition-by-compone

nts theory. The theory 

posits that people must 

pay attention to a 

stimulus before they can 

synthesize its features 

into a pattern. The 

implication is that if the 

person does not attend 

to an object, they may 

not perceive relevant 

features necessary to 

understand it. It links 

attention and workload 

with perception. 

The theory accounts 

for errors such as 

misreading indicators 

in busy control rooms 

due to distraction or 

high workload.

• HSI

• Attention

Proper cue perception 

depends on paying 

proper attention to 

those cues.

Treisman 

(1991)

Cue content Gestalt Principles. (1)

Elements close together 

organize into perceptual 

units. Objects that can 

form a continuum will 

tend to be perceived in 

this manner. Offers 

possibility that control 

room elements, 

especially instrument 

clusters, may be 

misgrouped and 

misperceived. (2)

Objects are seen as a 

whole, even if parts are 

missing or occluded. (3)

Wrong control activated: 

Elements close together 

organize into perceptual 

units. Objects that can 

form a continuum will 

tend to be perceived in 

1. Operators can 

misread instruments 

among other 

instruments. 

2. Has implications 

especially for 

disconnected 

information sources or 

for discrete sampling 

situations, affording 

possibility of 

misreading information. 

(Applies primarily to 

advanced HSIs.) 

3. Offers possibility that 

control room elements, 

especially controls, 

may be misgrouped 

and misperceived. Can 

identify wrong control 

among other controls. 

4. This property of 

• HSI None provided Köhler 

(1947)
A-19

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

this manner. (4) Rather 

than perceive parts of an 

object, the object is 

perceived as a whole—

the whole object 

automatically "emerges" 

from the parts during 

perception. 

(5) Multistable 

perception explains that 

when there is 

ambiguous information 

present in the 

environment, a person 

may cycle between two 

interpretations of what 

the cues represent. 

While certain optical 

illusions such as the 

Rubin vase are often 

used to demonstrate this 

phenomenon, it more 

broadly represents the 

opportunity for confusion 

in the perception of 

objects when there is 

ambiguity present. 

(6) Reification explains 

how objects are formed 

based on partial 

information. Since the 

mind seeks to recognize 

the environment in terms 

of objects, the mind may 

construct objects even 

when only partial 

information is available 

to support that object 

percept. 

Gestalt psychology 

explains how operators 

may form an incorrect 

picture of a situation by 

automatically forming 

an overall percept from 

environmental cues 

without decomposing 

individual cues.

5. Given the high 

number of cues 

present in control room 

I&C, ambiguous 

instrument cues are a 

distinct possibility. The 

operator's response to 

such ambiguity in 

terms of shuffling 

between interpretations

can lead to misreading 

the instruments and to 

delay in taking 

appropriate actions. 

6. This tendency to 

construct objects can 

lead to overgeneralizations or oversimplifications of 

environmental cues 

and may result in filling 

in missing information 

according to operator 

expectations.

Cue content The consistency, 

predictability, 

distinctiveness, and 

interpretability of the 

signal affect vigilance.

1. If the cue is not 

consistently presented, 

the operator may 

misperceive it.

2. If the cue is 

unpredictable, the 

operator may 

misperceive it.

3. If the cue is not 

presented to the 

operator with sufficient 

strength/energy to 

distinguish itself from 

existing background 

noise such that it 

activates a sensory 

response in the 

operator, the operator 

may misperceive it.

• HSI Cue qualities 

(consistency, 

predictability/reliability, 

and strength) affect 

perception

Donald 

(2001), Liu 

et al. (2009),

MacLean et 

al. (2009)

Cue content The human's ability to 

detect or sense a stimuli 

is a function of (1) how 

clearly that stimuli is 

present over and above 

existing background 

noise (e.g., the cue's 

signal to noise ratio), 

and (2) the extent to 

1. An operator that has 

responded frequently 

to false alarms may 

misperceive a 

subsequent alarm as 

another false alarm 

when it is in fact a real 

alarm.

• HSI

• Workload

• Task complexity

• Stress

• Fatigue

• Fitness for duty

1. The operator has 

changed the threshold 

to which he/she is 

willing to accept an 

error of commission 

because the system's 

signal to noise ratio is 

so low that the 

probability of false 

Bustamante 

(2008)
A-20

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

which the human is 

willing to accept an error 

of omission versus an 

error of commission. 

2. The operator is 

fatigued or under high 

stress and does not 

correctly perceive the 

cue (which may or may 

not have a clear signal)

alarms is unacceptably 

high.

2. When fatigued or 

under high stress, 

cognitive abilities are 

hindered. For example, 

the stress response 

can interfere with 

higher level cognitive 

processing.

Cue content Collaboration affects 

visual search (i.e., 

vigilance/attentional) 

performance. 

Operators working 

together on a visual 

search task are less 

likely to correctly detect 

targets and make false 

alarms (compared to 

operators performing

the task side by side, 

but not collaboratively). 

Working collaboratively 

tends to make the 

operators more 

conservative, which 

can lead them to 

misperceive a cue.

• Team PIFs Effects of team 

dynamics on team 

search

Malcolmson, 

et al. (2007)

Cue 

content, 

Vigilance in 

monitoring

The relationship 

between workload and 

vigilance is similar to 

that of stress and 

attention. 

When task complexity 

is too high, an operator 

may misperceive a 

cue. For example, a 

pilot mistaking the out 

of fuel light with the 

intercom light.

• Task complexity Task complexity being 

too high can interfere 

with higher level 

cognitive processing of 

information.

Donald 

(2001), Liu 

et al. (2009)

Cue 

content, 

Vigilance in 

monitoring

The ability to monitor 

tasks effectively (i.e., 

maintain vigilance) is 

affected by the amount 

of information presented 

and the number of 

displays it is presented 

in.

An operator may have 

difficulty monitoring 

plant status indicators 

that are distributed 

across multiple panels. 

(In practice, related 

functions are grouped 

on panels, but during a 

plant transient involving 

multiple symptoms, the 

operator may need to 

monitor multiple areas.)

• HSI As the number of 

displays increases, the 

number of errors of 

commission the 

operator makes 

increases. Participants 

committed errors of 

commission only 1-2% 

of the time when there 

was only one split 

screen, regardless of 

how frequently or 

infrequently the signal 

was presented. 

Participants committed 

errors of commission 

16-17% of the time 

when there were 16 

split screens and the

signal was presented 

at a rate of 1 time 

every 30 seconds. The 

error of commission 

rate dropped to 7-8% 

when the signal rate 

was 1 time every 60 

seconds, and generally 

improved the more 

infrequently the signal 

was presented.

Lin et al. 

(2009)

Expectation Verbal communication 

misunderstood: 

Accounts for how we 

recognize words better 

than individual sounds, 

Accounts for hearing 

wrong word (either 

hear it wrong because 

expecting something 

else in that context or 

• All of the PIFs 

listed under the 

category 

"Personal PIFs"

Verbal (word) salience 

can affect proper 

perception

Reicher 

(1969)
A-21

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

but we may perceive the 

incorrect word 

depending on the 

context. Sentence 

context fills in missing 

information, but it can 

also prime the wrong 

word. 

hear it wrong because 

sounds that make up 

word are not clearly 

audible).

• External 

Environment

Expectation Verbal communication 

misunderstood: People 

fill in missing speech 

sounds based on 

context. 

Accounts for hearing 

wrong word (either 

hear it wrong because 

expecting something 

else in that context or 

hear it wrong because 

sounds that make up 

word are not clearly 

audible).

• Task load Verbal (word) salience 

can affect proper 

perception; missing 

information may be 

inserted based on 

context

Warren 

(1970)

Expectation Fuzzy Logic Model of 

Perception. Verbal 

communication 

misunderstood: General 

theory that suggests we 

combine context and 

features to perceive 

speech and objects. The 

boundaries categorically 

distinguishing objects 

are fuzzy, causing 

ambiguity in many 

cases. 

For example, the 

McGurk-MacDonald 

effect explains how 

visual and auditory 

information are 

integrated (e.g., if a 

visual /ga/ is combined 

with an auditory /ba/, 

the combination is 

perceived as /da/). The 

effect generalizes to 

other areas where 

there is a 

visual-auditory 

mismatch and can lead 

to misinterpretation of 

communications.

• Task load Verbal (word) salience 

can affect proper 

perception; missing 

information may be 

inserted or falsely 

categorized based on 

context

Massaro 

and Cohen 

(1992)

Working 

memory

As new information is 

perceived, it is 

segmented into events. 

Event segmenting 

occurs automatically and 

helps define information 

to be processed in 

working memory or the 

intersection points for 

accessing and 

comparing to working 

memory.

If an operator 

encounters a novel 

situation, the operator 

may not segment 

information about the 

situation properly, 

resulting in an incorrect 

mental model of the 

situation, potentially 

increasing the 

likelihood of 

misperceived cues.

• Experience/ 

training

Segmenting for familiar 

activities is at a 

coarser level than for 

novel activities. 

Segmenting will 

require greater working 

memory, or 

segmenting may occur 

at false boundaries for 

novel activities.

Kurby & 

Zacks 

(2007)
A-22

A-2. UNDERSTANDING AND SENSEMAKING 

This section of Appendix A contains the cognitive mechanism tables for the Understanding and 

Sensemaking macrocognitive function. The macrocognitive function Understanding and 

Sensemaking is the process of understanding the meaning of the information that has been 

detected. Cognition in this function ranges from automatic, effortless recognition and 

understanding to more effortful thinking and deliberate attempts to make sense of multiple 

pieces of information. Sensemaking allows people to question what is known, evaluate what is 

conjectured, hypothesize and diagnose, and integrate facts with theory (Klein, Phillips, Rall, & 

Peluso, 2007).

The model used to integrate all of the various approaches to understanding for the purposes of 

this effort is the data/frame theory of sensemaking (Klein, Moon, & Hoffman, 2006; Klein, et al., 

2007). This model is an extension of Neisser’s (1967) perceptual cycle theory, in which a 

person’s sampling of the environment updates their cognitive map of the environment, which in 

turn directs further exploration. The data-frame theory of sensemaking posits that information 

coming into the sensemaking process is data. This data is integrated with an existing frame, 

which is a person’s current understanding that links data with other elements to explain and 

describe the relationship of the data with other entities. A frame encompasses the concepts of 

a mental representation, a mental model, a story, a map, a schema, a script, or a plan, and 

serves as a structure for explaining the data and guiding the search for more data (Klein, et al., 

2007). The data identify or construct the frame, and the frame determines which data are 

attended to. Neither the data nor the frame comes first; rather, it is a constant process of 

moving back and forth from data to frame. This dynamic aspect is the central theme of the 

model.

Chapter 4 discusses Understanding and Sensemaking in more detail.

In general, failure of the Understanding and Sensemaking macrocognitive function means that 

the person has an incorrect understanding of the situation. In other words, she or he has an 

incorrect mental model of what is happening. When this occurs, the operator is unlikely to make 

appropriate decisions in response to the situation, but the root of this error is in the 

Understanding and Sensemaking macrocognitive function. The literature review identified three 

proximate causes as potentially leading to this failure. Those proximate causes, along with their 

cognitive mechanisms, are:

1. Incorrect data (Table A-2.1)

a. Information available in the environment (including procedures) is not complete, correct, 

or otherwise sufficient to create understanding of the situation

b. Attention to wrong or inappropriate information 

c. Improper data or aspects of the data selected for comparison with or identification of a 

frame 

d. Incorrect or inappropriate or inadequate frame used to search for, identify, or attend to 

information 

e. Data not properly recognized, classified, or distinguished.

2. Incorrect integration of data, frames, or data with a frame (Table A-2.2)

a. Data not properly recognized, classified, or distinguished

b. Improper integration of information

c. Improper aspects of the frame selected for comparison with the data
A-23

d. Improper data or aspects of the data selected for comparison with or identification of a 

frame

e. Incorrect or failure to match data or information to a frame or mental model

f. Mental manipulation of the information (including projection of future status) is 

inadequate, inaccurate, or otherwise inappropriate

g. Working memory limitations impair processing of information

h. Improper control of attention.

3. Incorrect frame (Table A-2.3)

a. Incorrect or inadequate frame or mental model used to interpret or integrate information

b. Frame or mental model inappropriately preserved or confirmed when it should be 

rejected or reframed

c. Frame or mental model inappropriately rejected or reframed when it should be preserved 

or confirmed

d. Incorrect or inappropriate frame used to search for, identify, or attend to information

e. No frame or mental model exists to interpret the information or situation.
A-24

A-2.1 Incorrect Data

This table contains the mechanisms for the Incorrect Data proximate cause. When this is the 

cause of Detecting and Noticing failure, the data the person is comparing with a frame is 

incomplete, incorrect, or otherwise insufficient to understand the situation. This may be due to

the information itself being faulty, external to the person, errors in the perceptual process (in 

which case the failure is in Detecting and Noticing), or the person attending to inappropriate 

information, or focusing on inappropriate aspects of the information (e.g., operators are focused 

on coolant level and overlook rate of change). This proximate cause can be explained by the 

following mechanisms:

1. Information available in the environment (including procedures) is not complete, correct, or 

otherwise sufficient to create understanding of the situation

2. Attention to wrong or inappropriate information 

3. Improper data or aspects of the data selected for comparison with or identification of a frame 

4. Incorrect or inappropriate or inadequate frame used to search for, identify, or attend to 

information 

5. Data not properly recognized, classified, or distinguished.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Information 

available in 

the 

environment 

(including 

procedures) 

is not 

complete, 

correct, 

accurate, or 

otherwise 

sufficient to 

create 

understandin

g of the 

situation

The quality, accuracy, 

structure, and 

availability of 

information in the 

situation—the better 

and more accurate the 

information available, 

the better people are 

able to make sense of 

it. This means that 

information from the 

human-system 

interface is of crucial 

importance for 

properly 

understanding an 

event.

This also includes 

issues with the 

procedures, 

specifically the 

procedure formatting, 

having simple or 

complex logic, being 

difficult to read and 

follow. It also includes 

issues of vagueness 

or lack of specificity. 

If operators do not have 

the right information, 

they will be less likely to 

have an accurate 

understanding of what is 

going on. For example, 

in an event at the Crystal 

River 3 nuclear power 

plant, a pressurizer spray 

valve stuck open during 

a power increase 

following a shutdown, 

causing a slow 

depressurization. The 

indication for this valve in 

the control room showed 

that it was closed. The 

plant behavior was 

perceived as 

inexplicable, as all they 

could see was evidence 

of a depressurization 

with no indications of 

problems with the 

pressurizer or any other 

reason to explain it. Their 

operating procedures did 

not help them determine 

the cause of the problem 

or prompt them to verify 

that all relevant spray 

valves were closed. 

Without information from 

the plant indicators or 

procedures to guide 

them, operators did not 

understand the nature of 

the situation and made 

several inappropriate 

knowledge-based 

decisions.

• HSI output

• System 

complexity

• Situation 

dynamics or 

complexity

• Procedure 

availability, 

quality

• Quality and 

availability of 

information

1. The quantity and quality of 

the information provided by 

the system interface is of 

critical importance. The “Las 

Vegas Strip” phenomenon—

when systems display flashing 

lights, moving icons, overuse 

bright colors, or simply 

present too much information, 

information overload can 

occur, as well as misplaced 

salience, when information 

that is objectively irrelevant (or 

inaccurate) is physically more 

salient than important 

information, drawing operator 

attention away from needed 

information. Furthermore, the 

framing effect shows that 

aspects of information 

presentation can affect 

interpretation of the 

information.

2. Highly complex systems 

make it difficult for people to 

develop sufficient internal 

representations of how the 

systems work. System 

complexity can slow down the 

ability of people to take in 

information (due to the 

overwhelming amount of 

information), and it 

undermines their ability to 

correctly interpret the 

information and project what 

is likely to happen next.

3. If the situation or 

information in the environment 

is too complex or changing 

Durso et al. 

(2007)

Endsley (1995)

Endsley (2000)

Endsley ( 2006)

Endsley, Bolté 

et al. (2003)

Endsley & 

Garland (2000)

Jones & 

Endsley (1996)

Jones & 

Endsley (2000)

Kim et al.(1999)

Meyer (1992)

Salmon et al. 

(2008)

Smith & 

Hancock (1995)

Tversky & 

Kahneman 

(1981)
A-25

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

too rapidly, then it will be 

beyond the operators’ 

cognitive abilities to process.

4. If the information available 

from the environment, 

including from the procedures, 

is not correct, appropriate, or 

sufficient for the person to 

adequately update his/her 

knowledge, then they are less 

likely to understand what is 

going on.

5. Information that is very 

inconsistent with the current 

understanding of the situation 

is more likely to be identified 

as contradictory and often 

leads to recognizing that 

either the data or the frame is 

in error, whereas information 

that is seen as irrelevant to 

the frame (but is objectively 

relevant to the situation) is 

simply not noticed, and an 

incorrect frame persists.

6. System automation can 

undermine situation 

awareness by taking the 

operator out of the loop. If 

complex, automated systems 

do not provide operators with 

enough information to keep 

them in the loop, operators 

may struggle to understand 

what is going on when the 

automation fails or reaches 

situations for which it is not 

equipped to handle.

Attention to 

wrong/

inappropriate 

information 

This mechanism refers 

to the problem when 

attention is on the 

wrong information, 

due to bottom-up 

issues such as 

information salience. 

Specifically, this refers 

to the problem that 

occurs when 

particularly “loud” or 

salient information that 

is objectively incorrect 

or irrelevant catches 

the operator’s 

attention. This 

attention capture 

diverts operators’ 

attention away from 

the information that is 

relevant and important 

to the situation.

This also includes 

issues such as 

accidentally gathering 

the wrong information, 

like getting information

from Train A instead of 

Train B.

If operators are attending 

to inappropriate 

information, they may 

misclassify the situation. 

For example, when faced 

with a large number of 

alarms, operators may 

miss one important alarm 

(such as a RCP seal 

temperature alarm) 

because they are 

focusing on other alarms.

• Cue salience

• HSI Output

1. Misplaced salience, when 

information that is objectively 

irrelevant (or inaccurate) is 

physically more salient than 

important information, can 

draw operator attention away

from needed information.

2. The information that is 

sampled in the environment 

feeds into and updates the 

operators’ frame/mental 

model. If the information 

provided by the interface is 

incorrect, so will the operators’ 

understanding of the situation 

be incorrect.

Adams et al. 

(1995) 

Endsley, Bolté 

et al. (2003)

Klein (1993)

Lipshitz (1993)

Salmon et al. 

(2008)

Smith & 

Hancock (1995)
A-26

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Improper 

data/aspects 

of the data 

selected for 

comparison 

with/

identification 

of a frame 

This mechanism refers 

to the situation where 

the operator focuses 

on the wrong 

information, or wrong 

parts of the 

information. This is 

less a bottom-up issue 

of cue salience 

capturing attention, 

but instead describes 

the situation when 

operators have a 

frame or mental model 

about what is 

happening, but 

because of the way 

information initially 

presented itself, focus 

on inappropriate data. 

This can also occur 

due to operators 

placing too much 

subjective significance 

on information that is 

objectively 

unimportant.

If operators focus on one 

indicator to the exclusion 

of others, they may not 

fully understand the 

situation. For example, if 

operators focus on RCS 

level but do not account 

for the rate of change in 

level, they may 

misunderstand the 

urgency of the situation.

• Knowledge/ 

experience/ 

expertise

• Training

• HSI Output, 

Situation 

dynamics

1. Having insufficient 

knowledge and experience 

can lead operators to 

selecting an incorrect or 

inadequate model of the 

situation, focus on 

inappropriate aspects of the 

frame to understand the 

situation, and come up with an 

incorrect understanding of the 

information. Experts are better 

able to make use of their 

mental models to impose form 

on sensory data in real time.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with the 

necessary information or 

practice, knowledge will 

suffer.

3. The initial one or two key 

data elements that present 

themselves in the situation 

often serve as anchors for 

developing understanding. 

These anchors elicit the initial 

frame, and we use that frame 

to search for more data 

elements, which in turn 

elaborate that frame or anchor 

a new frame. If the first few 

pieces of information are 

misleading or spurious, 

understanding will suffer. 

Bedny & Meister

(1999)

Durso et al. 

(2007)

Klein et al. 

(2007)

Incorrect/ 

inappropriate/ 

inadequate 

frame used to 

search for, 

identify, or 

attend to 

information 

This refers to when 

the operator has an 

incorrect frame or 

mental model for the 

situation, which directs 

their attention toward 

information that is not 

objectively important. 

This, in turn, further 

perpetuates the 

incorrect 

understanding of the 

situation.

This can occur 

because of habit 

intrusion, in which the 

habitual frame 

interferes with the 

ability to attend to 

non-habitual 

information;

expectations for how 

the situation will 

unfold; the goals the 

operator has in the 

situation, including 

goals of personal 

significance to the 

operator (such as 

saving face or not 

being perceived as 

wrong); a lack of 

For example, people 

tend to seek out 

evidence that confirms 

their current position or 

mental model and to 

disregard evidence that 

conflicts with their 

current position, a 

phenomenon known as 

confirmation bias.

• Knowledge/ 

experience/ 

expertise

• Training

1. Having insufficient 

knowledge and experience 

can lead operators to 

selecting an incorrect or 

inadequate model of the 

situation, focus on 

inappropriate aspects of the 

frame to understand the 

situation, and come up with an 

incorrect understanding of the 

information. Experienced 

people use their knowledge 

base to develop expectancies 

about situations. Experts are 

more likely to be able to work 

with multiple frames at once. 

When faced with data that has 

more than one explanation, 

experts can deliberately 

elaborate two to three frames 

simultaneously, looking for 

information that will rule out 

one of the frames. Experts are 

more prone to question the 

data, and are also more 

confident and skeptical in the 

face of contradictory 

information.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with the 

Bedny & Meister 

(1999)

Einhorn & 

Hogarth (1981)

Endsley (1995)

Endsley (2006)

Feltovich et al., 

(1984)

Klein et al. 

(2007)

Niesser (1976)

Salmon et al. 

(2008)

Smith & 

Hancock (1995)
A-27

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

relevant training, 

knowledge, or 

experience; or 

knowledge, training, or 

experience 

predisposing the 

operator to an 

incorrect frame for the 

situation.

NOTE: This can also 

lead to not checking 

information frequently 

enough. If the frame 

doesn’t specify 

checking information 

often enough, then 

important parameters 

may not be checked 

with adequate 

frequency.

necessary information or 

practice, knowledge will 

suffer.

Data not 

properly 

recognized, 

classified, or 

distinguished 

This mechanism refers 

to classification or 

categorization of 

information at the 

recognition level, 

before the information 

is more deeply or 

consciously 

processed.

Once a scene is 

identified (perceived), 

top-down influences 

from existing 

knowledge on 

identification of 

aspects of the 

situation begin.

Operators can get the 

“gist” of a situation 

very quickly, and that 

“gist” may be 

inaccurate if 

information has not 

been properly 

classified.

If information is 

improperly categorized 

(e.g., as irrelevant or

oversimplifications), 

then operators will not 

make proper use of 

the information, and 

understanding will 

suffer.

This misclassification 

can occur due to 

misperceiving the 

information (see 

Detecting and 

Noticing), or 

misinterpreting the 

information based on 

existing knowledge.

For example, if a 

particular alarm has a 

history of sounding 

spuriously, operators 

may be likely to dismiss 

it as a false alarm if it 

activates in the case of a 

real problem.

Another example is the 

case of operators 

misreading procedures. 

Crews attend to the 

procedures while 

performing actions. 

During this time the crew 

will judge whether the 

strategies embodied in 

the procedures are 

appropriate to the 

situation. The crew may 

make a mistake and 

inappropriately misjudge 

the procedures as 

inapplicable due to 

misreading the 

procedures.

• Knowledge/ 

experience/ 

expertise

• Training

• HSI Output

• Procedure 

quality

1. Generally, novices or less 

experienced people tend to be 

less certain about the 

relevance of pieces of 

information, and are more 

·likely to interpret information 

that is noise in the situation as 

important signals. Insufficient 

expertise or knowledge of 

critical cues in the 

environment may prevent very 

fine classifications of incoming 

data.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with the 

necessary information or 

practice, knowledge will 

suffer.

3. The manner in which 

information is presented can 

affect how it is interpreted. For 

example, the framing effect 

shows that word choices, 

colors, images, and other 

aspects of presentation can all 

greatly affect resulting 

interpretation

4. If procedures are of poor 

quality, such as being difficult 

to follow, being inaccurate, or 

failing to prioritize important 

steps, they may be more likely 

to cause misunderstanding.

Durso et al. 

(2007)

Endsley (1995)

Klein (1993)

Klein et al.

(2007)

Lipshitz (1993)

Roth (1997)

Tversky & 

Kahneman 

(1981)
A-28

A-2.2 Incorrect Integration of Data, Frames, or Data With a Frame

This table contains the mechanisms for the Incorrect Integration of Data, Frames, or Data with a 

Frame proximate cause. This refers to errors in integration that lead to failure of the 

Understanding and Sensemaking macrocognitive function. The person does not properly 

integrate pieces of information together, does not correctly match data with a frame, or does not 

appropriately integrate multiple frames, such as when a person does not properly merge a 

frame for a system (i.e., a system model) with the frame for the ongoing event (i.e., a situation 

model). In addition, because sensemaking is a continuous, dynamic process, the person must 

integrate new data into the frame periodically as the situation evolves. In this proximate cause, 

the data is correct, the frame is correct, but the integration, matching, or updating process goes 

awry. This proximate cause can be explained by the following mechanisms:

1. Data not properly recognized, classified, or distinguished

2. Improper integration of information or frames

3. Improper aspects of the frame selected for comparison with the data

4. Improper data or aspects of the data selected for comparison with or identification of a frame

5. Incorrect or failure to match data or information to a frame or mental model

6. Mental manipulation of the information (including projection of future status) is inadequate, 

inaccurate, or otherwise inappropriate

7. Working memory limitations impair processing of information

8. Improper control of attention.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Data not 

properly 

recognized, 

classified, or 

distinguished 

As information is perceived, 

more meaning is applied to 

it. This is the blurry boundary 

between Detecting and 

Noticing and Understanding 

and Sensemaking. This 

mechanism refers to 

classification or 

categorization of information 

at the recognition level, 

before the information is 

more deeply or consciously 

processed.

Once a scene is identified 

(perceived), top-down 

influences from existing 

knowledge on identification 

of aspects of the situation 

begin. Operators can get the 

“gist” of a situation very 

quickly, and that “gist” may 

be inaccurate if information 

has not been properly 

classified.

If information is improperly 

categorized (e.g., as 

irrelevant or 

oversimplifications), then 

operators will not make 

proper use of the 

information, and 

understanding of the 

situation will suffer.

For example, if a 

particular alarm has 

a history of sounding 

spuriously, operators 

may be likely to 

dismiss it as a false 

alarm if it activates 

in the case of a real 

problem.

Another example is 

the case of 

operators 

misreading 

procedures. Crews 

attend to the 

procedures while 

performing actions. 

During this time the 

crew will judge 

whether the 

strategies embodied 

in the procedures 

are appropriate to 

the situation. The 

crew may make a 

mistake and 

inappropriately 

misjudge the 

procedures as 

inapplicable due to 

misreading the 

procedures.

• Knowledge/ 

experience/ 

expertise

• Training

• HSI Output

• Procedure 

quality

1. Generally, novices or 

less experienced people 

tend to be less certain 

about the relevance of 

pieces of information, and 

are more ·likely to interpret 

information that is noise in 

the situation as important 

signals. Insufficient 

expertise or knowledge of 

critical cues in the 

environment may prevent 

very fine classifications of 

incoming data.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with 

the necessary information 

or practice, knowledge will 

suffer.

3. The manner in which 

information is presented 

can affect how it is 

interpreted. For example, 

the framing effect shows 

that word choices, colors, 

images, and other aspects 

of presentation can all 

greatly affect resulting 

interpretation

4. If procedures are of poor 

quality, such as being 

Durso et al. 

(2007)

Endsley 

(1995)

Klein (1993)

Klein et al. 

(2007)

Lipshitz (1993)

Roth (1997)

Tversky & 

Kahneman 

(1981)
A-29

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

This misclassification can 

occur due to misperceiving 

the information (see 

Detecting and Noticing), or 

misinterpreting the 

information based on 

existing knowledge.

difficult to follow, being 

inaccurate, or failing to 

prioritize important steps, 

they may be more likely to 

cause misunderstanding.

Improper 

integration of 

information

As people process 

information into 

understanding, they 

assemble or integrate pieces 

of information together and 

map it to existing knowledge. 

If they put the information 

together improperly, they can 

come to incorrect or 

inadequate conclusions 

about the situation. They 

may develop an incorrect 

holistic picture from the 

separate perceived 

elements, producing an 

inappropriate mental model 

or frame for the situation.

For example, if 

operators are 

dealing with one 

faulted system, they 

may have difficulty 

handling a 

subsequent 

secondary faulted 

system, and 

integrating the 

information about 

the second system 

into the mental 

model for handling 

the situation.

• Attention, 

working 

memory 

capacity

• Knowledge/ 

experience/ 

expertise

1. People can only process 

so much information at 

once. Errors in 

understanding can occur 

due to insufficient working 

memory span, or the decay 

of information in working 

memory over time. 

2. Experts have a richer 

repertoire of frames and a

much stronger 

understanding of context. 

Experts are better able to 

make use of their mental 

models to impose form on 

sensory data in real time.

Experts have greater skill in 

producing inferences. This 

can be seen in the 

inferences of the values of 

variables during monitoring, 

in the use of covert 

variables in the building-up 

of a representation during 

diagnosis and in the use of 

inferential strategies during 

executive control.

Experts are better at 

organizing information into 

meaningful units.

Baddeley 

(1992)

Baddeley & 

Hitch (1974)

Cellier, 

Eyrolle, & 

Marine (1997)

Doane et al. 

(2004)

Durso et al. 

(2007)

Endsley et al. 

(2003)

Sternberg & 

Davidson 

(1986)

Improper 

aspects of the 

frame selected 

for comparison

with the data

When comparing information 

with the active frame, it is 

possible for people to focus 

on inappropriate parts of the 

frame for the evaluation of 

the data. 

For example, a 

frame may have 

“default” and 

“exception” values, 

such as a rule that 

“Scenario X has 

symptoms of A, B, 

and C, EXCEPT 

when variant Y 

occurs, and then the 

symptoms of A, B, 

D, and E are 

relevant.” An error 

can occur if the 

person focuses 

inappropriately on 

either the default or 

the exception 

values.

• Knowledge/ 

experience/ 

expertise

Having insufficient 

knowledge and experience 

can lead operators to 

selecting an incorrect or 

inadequate model of the 

situation, focus on 

inappropriate aspects of the 

frame to understand the 

situation, and come up with 

an incorrect understanding 

of the information. Experts 

are better able to make use 

of their mental models to 

impose form on sensory 

data in real time.

Durso et al. 

(2007)

Endsley 

(1995)

Improper 

data/aspects of 

the data selected 

for comparison 

with/identification 

of a frame 

This mechanism refers to the 

situation where the operator 

focuses on the wrong 

information, or wrong parts 

of the information. This is 

less a bottom-up issue of 

cue salience capturing 

attention, but instead 

describes the situation when 

operators have a frame or

If operators focus on 

one indicator to the 

exclusion of others, 

they may not fully 

understand the 

situation. For 

example, if 

operators focus on 

RCS level but do not 

account for the rate 

• Knowledge/ 

experience/ 

expertise

• Training

• HSI Output

1. Having insufficient 

knowledge and experience 

can lead operators to 

selecting an incorrect or 

inadequate model, focus on 

inappropriate aspects of the 

frame to understand the 

situation, and come up with 

an incorrect understanding 

of the information. Experts 

Bedny & 

Meister, 

(1999)

Durso et al. 

(2007)

Klein et al. 

(2007)
A-30

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

mental model about what is 

happening, but because of 

the way information initially 

presented itself, focus on 

inappropriate data. This can 

also occur due to operators 

placing too much subjective 

significance on information 

that is objectively 

unimportant.

of change in level, 

they may 

misunderstand the 

urgency of the 

situation.

are better able to make use 

of their mental models to 

impose form on sensory 

data in real time.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with 

the necessary information 

or practice, knowledge will 

suffer.

3. The initial one or two key 

data elements that present 

themselves in the situation 

often serve as anchors for 

developing understanding. 

These anchors elicit the 

initial frame, and we use 

that frame to search for 

more data elements, which 

in turn elaborate that frame 

or anchor a new frame. If 

the first few pieces of 

information are misleading 

or spurious, understanding 

will suffer.

Incorrect or 

failure to match 

data/ information 

to a 

frame/mental 

model

When evaluating the 

information available in a 

situation, people may fail to 

match the information to a 

frame or mental model, or 

they may make an incorrect 

match. This may be due to 

overexplanation, or finding 

connections that don’t 

actually exist, or to failing to 

recognize that the data 

matches a particular frame.

For example, a 

situation may be 

misinterpreted by 

the crew and the 

procedures are, 

therefore, deemed 

inappropriate. Crews 

will follow the 

procedures while 

responding to an 

event. During this 

time the crew will 

judge/decide 

whether the 

strategies embodied 

in the procedures 

are appropriate to 

the situation. The 

crew may make a 

mistake either due to 

a misinterpretation 

of the situation or 

due to misreading 

the procedures.

• HSI Output

• Knowledge/ 

experience/ 

expertise

1. If operators are focused 

on other, overly salient 

information and neglect 

relevant but less salient 

information, they may not 

map the important 

information to a frame.

2. People are explanation 

machines. People will 

employ whatever tactics are 

available to help them find 

connections and identify 

anchors. This means that 

people may find 

connections between all 

sorts of pieces of 

information, even those that 

aren’t actually connected. 

Experts are better able to 

work with the information;

they have a richer repertoire 

of frames, and are less 

likely to find 

pseudocorrelations.

Anderson (

2000)

Chi, Feltovich, 

& Glaser 

(1981)

Endsley 

(1995)

Klein et al. 

(2007)

Roth (1997)

Mental 

manipulation of 

the information 

(including 

projection of 

future status) is 

inadequate, 

inaccurate, or 

otherwise 

inappropriate

People don’t just put pieces 

of information together; they 

mentally manipulate, 

transform, and turn the 

information around in their 

heads. This includes 

projecting the current status 

into the future to imagine 

how things will play out, 

engaging in hypothetical 

postulation, and abstracting, 

simplifying, or combining 

information to better make 

sense of it. If this mental 

transformation is done 

improperly, for example, if 

information is overly 

Overly optimistic 

estimations of the 

amount of time one 

has to work with can 

lead to problems in 

responding 

appropriately to an 

event.

• Knowledge/ 

experience/ 

expertise

• Training

1. To ensure the proper 

interpretation of information 

by the operator, the 

operator must have 

professional experience and 

knowledge. Having 

insufficient knowledge and 

experience can lead 

operators to selecting an 

incorrect or inadequate 

model of the situation, focus 

on inappropriate aspects of 

the frame to understand the 

situation, and come up with

an incorrect understanding 

of the information.

Bedny & 

Meister (1999)

Cellier et al.

(1997)

Durso et al. 

(2007)

Endsley 

(1995)

Klein et al. 

(2007)
A-31

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

simplified, or if projection of 

future status is overly 

optimistic or conservative, or 

if the mental manipulation 

focuses on inappropriate 

aspects of the situation, then 

an incorrect understanding of 

the situation can develop.

NOTE: This can also lead to 

not checking information 

frequently enough.

Experts have greater skill in 

anticipating. This can be 

seen in the gathering of 

anticipated cues during 

monitoring and diagnosis, in 

the preventive rather than 

reactive processing of 

disturbances and in better 

predictions of process 

evolution.

2. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with 

the necessary information 

or practice, knowledge will 

suffer.

Working memory 

limitations impair 

processing of 

information

Only a limited amount of the 

situation can be processed 

at one time due to limited 

cognitive capacity.

Operators dealing 

with multiple alarms,

multiple system 

faults, or multiple 

tasks may have 

difficulty keeping 

everything in 

memory. This will 

impair processing 

and understanding 

of the situation.

• Situation 

complexity

• Working 

memory 

capacity

• Experience/ 

knowledge/ 

expertise

• Training

• HSI output

• Workload

1. If the environment is well 

structured, then brief 

samples can take 

advantage of redundancies 

in the world. If it is 

unstructured, or the 

operator does not have the 

experience to take 

advantage of the structure, 

then situation awareness 

will be impaired.

2. Working memory span, 

the number of chunks of 

information one can hold 

active in memory, is 

correlated with situation 

awareness performance. 

People with larger working 

memory spans tend to 

develop better situation 

awareness. The more 

information a person can 

work with, the more likely 

they are to develop a full 

picture of what is 

happening.

3. Experts’ chunks are more 

complex and structured 

than novices, so experts 

can work with more 

information successfully 

than novices. 

4. Training is a method for 

developing knowledge and 

experience. If training does 

not provide operators with 

necessary information or 

practice, knowledge will 

suffer.

5. Significant errors can 

result from systems that 

force operators to rely 

heavily on their memory.

6. Maintaining 

understanding requires 

expending cognitive 

resources, which may 

compete with other parallel 

cognitive tasks. Heavy 

Anderson

(2000)

Cook (2001)

Durso et al. 

(2007)

Endsley 

(2000)

Endsley, et al. 

(2003) 

Endsley & 

Garland 

(2000)

Gugerty & 

Tirre (2000)

Wickens

(2001)
A-32

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

concurrent task demands 

may in turn divert resources 

from the maintenance of 

situation awareness. This 

relationship between 

workload and

understanding can also 

depend on expertise: an 

experienced operator can 

maintain understanding with 

less investment of cognitive 

resources, often due to their 

richer mental models. 

However, a more 

experienced operator does 

not necessarily have a 

better understanding of the 

situation; heavy reliance on 

resource-free mental 

models may mean that the 

operator incorrectly 

interprets or fails to notice 

an unexpected event.

Improper control 

of attention

People limited amounts of 

attention. There is only so 

much information a person 

can attend to at once. 

Problems in attention control 

can manifest by failing to 

maintain attention to multiple 

tasks, goals, or pieces of 

information. 

Procedures often 

require attention to 

multiple goals, for 

example, continuous 

action steps that 

must be carried out 

in parallel with other 

steps. Or, when 

operators are 

dealing with multiple 

faults, multiple 

systems, they may 

have difficulty 

properly maintaining 

attention on the 

appropriate items.

• Knowledge/ 

experience/ 

expertise

• Cue 

salience/HSI 

output

• Stressors

• Workload

1. Experienced operators 

have mental models that 

inform them of the 

information they need to 

attend to and which 

information to disregard. 

Expert’s search patterns 

through even complex 

displays for information can 

be quite efficient. However, 

even experienced operators 

can make errors in the 

process of prioritizing 

attention and neglect to 

attend to certain 

information, particularly 

when they have to juggle 

numerous competing tasks. 

2. Stressors can narrow the 

attentional field, or the area 

of the visual field that 

operators can attend to. 

This means that potentially 

information in the periphery 

may be neglected, leading 

to poorer situation 

awareness

3. Maintaining a high level 

of understanding requires 

expending cognitive 

resources, which may 

compete with other parallel 

cognitive tasks. Heavy 

concurrent task demands 

may in turn divert resources 

from the maintenance of 

situation awareness.

Expertise can reduce 

workload due to use of 

richer mental models, but 

reliance on less resourceintensive mental models 

may introduce other errors.

Adams, 

Tenney, & 

Pew (1995)

Baddeley & 

Hitch (1974)

Cook (2001)

Durso & 

Gronlund 

(1999)

Endsley et al. 

(2003)

Jones & 

Endsley 

(1996)

Wickens 

(2001)
A-33

A-2.3 Incorrect Frame

This table contains the mechanisms for the Incorrect Frame proximate cause. When this is the 

cause of Detecting and Noticing failure, the frame used to understand the situation is incorrect, 

incomplete, or otherwise insufficient to properly interpret the data. In this proximate cause, the 

mental model in use is not adequate to provide a correct understanding of the situation. This 

can be due to a number of factors, such as the person does not have the necessary experience 

to develop appropriate frames, or because the situation seems similar to a different frame. 

Whatever the reason, the cause of failure of Understanding and Sensemaking is that the person 

is using an inappropriate frame for the situation. This proximate cause can be explained by the 

following mechanisms:

1. Incorrect or inadequate frame or mental model used to interpret or integrate information

2. Frame or mental model inappropriately preserved or confirmed when it should be rejected or 

reframed

3. Frame or mental model inappropriately rejected or reframed when it should be preserved or 

confirmed

4. Incorrect or inappropriate frame used to search for, identify, or attend to information

5. No frame or mental model exists to interpret the information or situation.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Incorrect or 

inadequate 

frame/mental 

model used to 

interpret/

integrate 

information

This mechanism refers 

to instances in which the 

person is using an 

incorrect, incomplete, or 

otherwise inadequate 

frame or mental model to 

understand the situation. 

This includes having an 

inappropriate goal for the 

situation, as goals are 

frames or parts of 

frames that indicate how 

things should happen or 

the ideal end state. This 

can occur if the person 

selects the wrong frame, 

if habit intrusion 

interferes with the 

selection of an 

appropriate frame, or if a 

subjectively important 

goal takes precedence 

over objectively 

important goals.

Frames or mental 

models can be 

inadequate in a number 

of ways, including but 

not limited to:

• The frame may be 

insufficient to integrate 

the information or 

determine which cues 

are actually relevant.

• The frame may have 

inaccurate uncertainty 

features, which makes 

projection unreliable.

An example is the 

availability cognitive bias.

People believe those 

values or items which are 

easier to retrieve from 

memory, or occur more 

frequently in memory, are 

more likely to occur. Items 

that are easier to retrieve 

from memory include those 

that the person is more 

familiar with, things that 

have been recently 

thought of, things that are 

frequently thought of.

An example of this in NPP 

domain: operators who 

have been trained heavily 

on a particular event may 

diagnose a different event 

with similar symptoms as 

the event they have 

recently trained on.

Another example is what is 

known as a “mode” error: 

operators believe that a 

particular system is

operating in a particular 

mode, such as in 

automatic, or on standby, 

when it is not. They may 

then not recognize 

information that suggests 

the system is not operating 

correctly.

• Knowledge/ 

experience/ 

expertise

• Training

• Motivation

1. To ensure the proper 

interpretation of 

information by the 

operator, the operator 

must have professional 

experience and 

knowledge. Lack of 

experience can influence 

the options available to 

the decision maker in 

deciding upon a course 

of action or in the 

evaluation of the options.

Experts’ mental models 

are richer in terms of 

having more detail, more 

variety, finer degrees of 

differentiation, and more

coverage of the domain.

Experts are also more 

sensitive to the context 

of the situation. Experts 

are more prone to 

question data than 

novices because they 

are more familiar with 

cases of faulty 

information. This may 

also mean that experts 

are more confident in 

their frames and 

skeptical of contrary 

evidence (making them 

more susceptible to 

fixation errors) than 

novices, who are less 

confident in frames and 

more sensitive to 

contrary data. Experts 

Anderson (2000)

Bedny & Meister

(1999)

Durso et al. 

(2007)

Endsley (1995)

Feltovich, 

Johnson, Moller, 

& Swanson 

(1984)

Klein (1998)

Klein et al. (2006)

Klein et al. (2007)

Tversky & 

Kahneman (1981)
A-34

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

• The model may not be 

sufficiently developed 

to support projection.

• Operators may have 

an adequate mental 

model for the situation 

(e.g., they may know 

what event is in 

progress), but an 

inadequate mental 

model of the system, 

plant configuration, 

system dynamics, or 

subtle dependencies, 

that may lead to errors 

in responding the 

situation.

are more likely to be 

able to work with 

multiple frames at once. 

When faced with data 

that has more than one 

explanation, experts can 

deliberately elaborate 

two to three frames 

simultaneously, looking 

for information that will 

rule out one of the 

frames.

3. Training is a method 

for developing 

knowledge and 

experience. If training 

does not provide 

operators with necessary 

information or practice, 

knowledge will suffer.

4. The higher the level of 

motivation or the more 

intense the motivation, 

the more mental and 

physical effort the 

person expends directed 

toward the achievement 

of that goal.

Frame/mental 

model 

inappropriately 

preserved/ 

confirmed when 

it should be 

rejected/ 

reframed

When faced with data 

from the situation that 

does not match the 

frame or mental model, 

people go through a 

process of questioning 

the data and/or the 

frame. Eventually, they 

either reframe or they 

reject the data as 

inaccurate, spurious, or 

faulty and confirm their 

existing mental model. 

This mechanism refers 

to the case when people 

inappropriately reject the 

data and persist in an 

incorrect frame or mental 

model.

Persistence in an 

inappropriate mental 

model may be difficult to 

overcome, because new 

information is interpreted 

in light of the model and 

is susceptible to 

confirmation bias.

Operators actively 

generate situation 

assessments that include 

possible explanations for 

the set of symptoms they 

observe. They then form 

expectations with regard to 

additional symptoms they 

should observe if their 

explanation is correct. 

These expectations are an 

example of a “frame” that 

may then be 

inappropriately confirmed 

or rejected.

For example, if a particular 

alarm has a history of 

sounding spuriously, 

operators may be likely to 

dismiss it as a false alarm 

if it activates in the case of 

a real problem.

• Knowledge/ 

experience/ 

expertise

• Trust in the 

data source

To ensure the proper 

interpretation of 

information by the 

operator, the operator 

must have professional 

experience and 

knowledge. Experts are 

more prone to question 

data than novices 

because they are more 

familiar with cases of 

faulty information. This 

may also mean that 

experts are more 

confident in their frames 

and skeptical of contrary 

evidence (making them 

more susceptible to 

fixation errors) than 

novices, who are less 

confident in frames and 

more sensitive to 

contrary data. 

Anderson (2000)

Bedny & Meister 

(1999)

Endsley (1995)

Klein et al. (2006)

Klein et al. (2007)

Roth (1997)
A-35

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Frame/mental 

model 

inappropriately 

rejected/ 

reframed when it 

should be 

preserved/ 

confirmed

When faced with data 

from the situation that 

does not match the 

frame or mental model, 

people go through a 

process of questioning 

the data and/or the 

frame. Eventually, they 

either reframe—say the 

data is correct and the 

model is wrong, and 

switch to a different

model—or reject the 

data and confirm their 

existing mental model. 

This mechanism refers 

to the case when people 

inappropriately reframe 

by rejecting a correct 

mental model and failing

to recognize faulty data. 

Instead, they change to 

a new, incorrect model 

of the situation.

When the frame is 

rejected (e.g., their 

expectations are 

violated), operators will 

search for additional or 

alternative factors to 

explain the observed 

system behavior.

Operators actively 

generate situation 

assessments that include 

possible explanations for 

the set of symptoms they 

observe. They then form 

expectations with regard to 

additional symptoms they 

should observe if their 

explanation is correct. 

These expectations are an 

example of a “frame” that 

may then be 

inappropriately confirmed 

or rejected.

• Knowledge/ 

experience/ 

expertise

To ensure the proper 

interpretation of 

information by the 

operator, the operator 

must have professional 

experience and 

knowledge. Experts are 

more prone to question 

data than novices 

because they are more 

familiar with cases of 

faulty information. This 

may also mean that 

experts are more 

confident in their frames 

and skeptical of contrary 

evidence (making them 

more susceptible to 

fixation errors) than 

novices, who are less 

confident in frames and 

more sensitive to 

contrary data. 

Anderson ( 2000)

Bedny & Meister

(1999)

Endsley (1995)

Klein et al. (2006)

Klein et al. (2007)

Roth (1997)

Incorrect/ 

inappropriate/ 

inadequate 

frame used to 

search for, 

identify, or 

attend to 

information

This mechanism refers 

to the situation when 

the operator has an 

incorrect frame or 

mental model for the 

situation, which directs 

their attention toward 

information that is not 

objectively important. 

This then further 

perpetuates the 

incorrect 

understanding of the 

situation.

This can occur 

because of habit 

intrusion, in which the 

habitual frame 

interferes with the 

ability to attend to 

non-habitual 

information, 

expectations for how 

the situation will unfold, 

the goals the operator 

has in the situation, 

including goals of 

personal significance 

to the operator (such 

as saving face or not 

being perceived as 

wrong), a lack of 

relevant training, 

knowledge, or 

experience, or 

For example, people 

tend to seek out 

evidence that confirms 

their current position or 

mental model and to 

disregard evidence that 

conflicts with their 

current position, a 

phenomenon known as 

confirmation bias.

• Knowledge/ 

experience/ 

expertise

• Training

1. Having insufficient 

knowledge and 

experience can lead 

operators to selecting 

an incorrect or 

inadequate model of 

the situation, focus on 

inappropriate aspects 

of the frame to 

understand the 

situation, and come up 

with an incorrect 

understanding of the 

information. 

Experienced people 

use their knowledge 

base to develop 

expectancies about 

situations. Experts are 

more likely to be able 

to work with multiple 

frames at once. When 

faced with data that 

has more than one 

explanation, experts 

can deliberately 

elaborate two to three 

frames simultaneously, 

looking for information 

that will rule out one of 

the frames. Experts 

are more prone to 

question the data, and 

are also more 

confident and skeptical 

Bedny & Meister 

(1999)

Einhorn & 

Hogarth (1981)

Endsley (1995)

Endsley (2006)

Feltovich et al. 

(1984)

Klein et al. 

(2007)

Niesser (1976)

Salmon et al. 

(2008)

Smith & 

Hancock (1995)
A-36

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

knowledge, training, or 

experience 

predisposing the 

operator to an incorrect 

frame for the situation.

NOTE: This can also 

lead to not checking 

information frequently 

enough—if the frame 

doesn’t specify 

checking information 

often enough, then 

important parameters 

will not be checked 

with adequate 

frequency.

in the face of 

contradictory 

information.

2. Training is a method 

for developing 

knowledge and 

experience. If training 

does not provide 

operators with the 

necessary information 

or practice, knowledge 

will suffer.

No frame/mental 

model exists to 

interpret the 

information/

situation

This mechanism refers 

to the case where the 

operator has no mental 

model or frame for the 

situation. This can be 

due to insufficient 

experience or training, or 

it can be due to the 

sheer bizarreness of the 

situation. In the absence 

of an appropriate mental 

model, people will often 

fail to solve a new 

problem, even though 

they would have to apply 

the same logic as that 

used for a familiar 

problem. 

This is likely rare, but may 

occur if the plant and the 

procedures do not provide 

the necessary information 

for operators to identify a 

model and make a 

diagnosis. An example is 

an event at the Crystal 

River 3 NPP, a pressurizer 

spray valve stuck open 

during a power increase 

following a shutdown, 

causing a slow 

depressurization. The 

indication for this valve in 

the control room showed 

that it was closed. The 

plant behavior was 

perceived as inexplicable, 

as all they could see was 

evidence of a 

depressurization with no 

indications of problems 

with the pressurizer. Their 

operating procedures did 

not help them determine 

the cause of the problem 

or prompt them to verify 

that all relevant spray 

valves were closed. 

Without information from 

the plant indicators or 

procedures to guide them, 

operators did not 

understand the nature of 

the situation and made 

several inappropriate 

knowledge-based 

decisions. When someone 

with more expertise joined 

the event response, that 

person was able to 

recognize the likelihood of 

a stuck open valve from 

knowledge, and thus the 

crew was able to terminate 

the event.

• Experience, 

knowledge, 

expertise

• Training

• Situation 

complexity

• HSI output, 

availability 

and quality of 

information

1. To ensure the proper 

interpretation of 

information by the 

operator, the operator 

must have professional 

experience and 

knowledge. Having 

insufficient knowledge 

and experience can lead 

operators to fail to 

understand the situation.

2. Training is a method 

for developing 

knowledge and 

experience. If training 

does not provide 

operators with the 

necessary information or 

practice, knowledge will

suffer.

3. If the situation or 

information in the 

environment is too 

complex changing too 

rapidly, then it will be 

beyond the operators’ 

cognitive abilities to 

process.

4. If the information 

available from the 

environment, including 

from the procedures, is 

not correct, appropriate, 

or sufficient for the 

person to adequately 

update his/her 

knowledge, then they 

are less likely to 

understand what is going 

on.

Bedny & Meister 

(1999)

Durso et al. 

(2007)

Endsley (1995)

Klein (1998)

Meyer (1992)
A-37

A-4. DECISIONMAKING

This section of Appendix A contains the cognitive mechanism tables for the macrocognitive 

function of Decisionmaking. Decisionmaking involves goal selection, planning, re-planning and 

adapting, evaluating options, and selection. In IDHEAS decisionmaking can refer to things such 

as deciding a piece of information is irrelevant, deciding between two gauges to obtain 

information, or deciding to ask a question. It is not limited to making a decision about what 

action to take in response to an event.

Decisionmaking within a nuclear power plant (NPP) is characterized as involving experts and 

being largely driven by procedures. Although procedures usually dictate the actions of the 

operators, Roth (1997) explains that the operators still maintain a mental model of the situation 

and will plan their course of action semi-independently of the procedures. That is, they will have 

an idea of what must be accomplished and how it should be done and will look to the 

procedures to confirm these beliefs. Furthermore, situations may arise that procedures do not 

cover. In these instances, operators must rely on their expert knowledge to solve the problem 

and implement the appropriate decision.

Two models that are particularly useful when examining Decisionmaking within an NPP are a 

recognition-primed decision (RPD) model (Klein, 1993), and an integrated naturalistic 

decisionmaking model proposed by Greitzer et al. (2010). RPD was primarily developed to 

explain decisionmaking in stressful situations and under time pressure. The integrated model 

includes concepts from SA, RPD, and recognition/metacognition. This integrated model seems 

to work well in identifying the process an experienced operator goes through in making a 

decision, even in the presence of procedures. 

Chapter 5 discusses Decisionmaking in more detail.

In general, failure of the Decisionmaking means that a wrong decision has been made about the 

nature of the event, whether information is relevant, what the response should be, or what 

action should be taken. In other words, an inappropriate response plan has been constructed. 

The literature review identified three proximate causes as potentially leading to this failure.

Those proximate causes, along with their cognitive mechanisms, are:

1. Incorrect goals or priorities set (Table A-3.1)

a. Incorrect goals selected

b. Missing a goal

c. Incorrect prioritization of goals

d. Incorrect judgment of goal success

2. Incorrect pattern matching (Table A-3.2)

a. Not updating the mental model to reflect the changing state of the system

b. Fail to retrieve previous experiences

c. Incorrect recall of previous experiences

d. Incorrectly comparing the mental model to previously encountered situations

e. Cognitive biases

3. Incorrect mental simulation or evaluation of options (Table A-3.3)

a. Inaccurate portrayal of action

b. Incorrect inclusion of alternatives

c. Inaccurate portrayal of the system response to the proposed action

d. Misinterpretation of procedures

e. Cognitive biases
A-38

A-4.1 Incorrect Goals or Priorities Set

This table contains the mechanisms for the Incorrect Goals or Priorities Set proximate cause. 

When this is the cause of Decisionmaking failure, the operator is working toward an 

inappropriate goal, or has goals improperly prioritized. Goals are set as the objectives that the 

decision should achieve. Goals are the measure for viewing the decision as successful or not. 

Although goals are formed during any decisionmaking process, they are especially relevant 

during novel situations when there is no previous experience to which the current situation and 

the outcome of the decided upon action can be compared against to measure success. If more 

than one goal is selected, priorities are assigned to the goals to determine the order in which 

they are to be addressed. This proximate cause includes errors that occur in either what goals 

are set or what priorities are assigned. Failure mechanisms include:

1. Incorrect goals selected. Errors may arise if the operators select the wrong goal to work 

toward. A variant of this failure mechanism is if the operator selects an implausible goal that 

cannot be achieved.

2. Goal conflict. A conflict may arise in the operator’s mind between the goals of safety and 

the continued viability of the plant.

3. Incorrect prioritization of goals. Goals may be ordered incorrectly in the operators’ mind or 

given the wrong priority, such that less important goals are addressed first.

4. Incorrect judgment of goal success. The threshold used by the operator to judge goal 

success may be incorrectly set too low, or be incorrectly determined as met when it was not.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Incorrect 

goals 

selected

During goal setting, the 

operator chooses the 

wrong goal(s) to work 

toward. The wrong 

goal(s) may be selected 

due to an improper 

understanding of the 

situation.

Although the operator 

may initially have 

classified the situation 

correctly (i.e., had a 

correct mental 

model), the situation 

may evolve to 

something different 

and the operator does 

not update the goals 

to reflect this new 

situation.

• Procedures

• Knowledge/ 

Experience/ 

Expertise

• Training

• System 

Reponses

• Safety culture

1. Procedures may mislead 

the operator to believe the 

initial situation is changing 

slower than it really is.

2. Experience with this type 

of situation may be lacking 

and the operator does not 

expect the situation to 

change so quickly or to 

evolve to the new state at all.

3. Training with this type of 

situation may be 

non-existent or have been 

given too long ago to be 

relevant.

The cues and responses 

being presented by the 

system may be ambiguous 

making it difficult for the 

operator and crew to 

diagnose the situation and 

develop the correct response 

plan.

Cacciabue et al. 

(1990)

Klein (1993)

Lipshitz (1993)

Orasanu (1993)

Reason (1997)

Incorrect 

goals 

selected

During goal setting, the 

operator may select 

implausible goals not 

realizing they cannot be 

achieved given the state 

of the system.

The operator has 

chosen a goal to 

reach success, but 

there is not enough 

time to enact all the 

necessary actions to 

achieve that goal.

• Knowledge/ 

Experience/ 

Expertise

• Time load

• Training

1. Experience with this 

particular situation is lacking 

and, therefore, the operator 

does not have an accurate 

sense of how long 

implementing actions and 

seeing results will take.

2. There may not be enough 

time to allow the operator to 

Jenkins, Stanton, 

Salmon, Walker, & 

Rafferty (2010)

Klein (1993)

Lipshitz (1993)
A-39

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

implement the solution as 

planned.

3. Training may be lacking 

on the manner in which the 

action should be 

implemented and on the 

situation in general, and 

therefore, the operator 

wastes time in implementing 

the solution and may have 

an inaccurate estimate of 

time available.

Goal conflict A conflict may exist in the 

operator’s mind between 

the goals of safety and 

continued viability of the 

plant.

An improper balance 

of priorities may lead 

them to choose a 

response option that 

is less optimal (with 

regards to plant 

integrity or safety). 

The consequences of 

the actions may be 

less than desirable in 

one sense (e.g., 

reduces system life 

expectancy; will result 

in significant plant 

outage duration), so 

the crew would be 

reluctant to execute a 

specific response 

path.

• Procedures

• Knowledge/ 

Experience/ 

Expertise

• Training

• System 

responses

• Perceived 

decision 

impact [plant] 

(awareness of 

the economic 

consequences)

• Safety culture

1. Procedures may be poorly 

written or have complicated 

logic such that the crew does 

not fully understand the 

seriousness of the situation.

2. Experience and 

knowledge may be lacking 

such that the operator does 

not recognize the 

seriousness of the situation 

or understand the 

ramifications of the decision.

3. Training may be 

infrequent and the operator 

does not know how to 

balance the priorities 

appropriately.

4. System responses may be 

difficult to understand or 

misleading causing the 

operator to misunderstand 

the seriousness of the 

situation.

5. The crew or operator may 

have an incorrect 

assessment of the impact of 

the decision and value the 

continued viability of the 

plant more.

Orasanu (1993)

Reason (1997)

Incorrect 

prioritization 

of goals

Goals may be ordered 

incorrectly and assigned 

the wrong priority either 

because the operator 

didn’t understand the 

importance of the goal or 

didn’t understand the 

impact of the action.

The operator or team 

become distracted by 

problems with the 

secondary system 

and devote time and 

resources solving that 

issue and do not 

prioritize the issue 

with the primary 

system.

• Training

• Knowledge/ 

Experience/ 

Expertise

• Safety culture

1. Training may be 

incomplete in how to 

prioritize goals and what 

systems should be 

recovered first and what 

actions should be performed 

first.

2. Experience with the plant 

may be lacking, and 

therefore, the operator 

doesn’t know how to 

prioritize the goals and 

actions and doesn’t fully 

understand the impact the 

actions will have on future 

goals.

Amendola, Bersini, 

Cacciabue, & 

Mancini (1987)

Kasbi & de 

Montmollin (1991)

Rouse (1983)

Reason (1997)
A-40

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Incorrect 

prioritization 

of goals

The operator or crew 

may not know how to 

devote their time and 

resources to the problem 

and waste precious time 

trying to prioritize the 

goals.

The crew must decide 

whether it is more 

important to chase 

down the cause of the 

problem being 

observed (e.g., why is 

feedwater low) or 

respond to the 

situations being 

created by the 

problem (e.g., the 

need to secure more 

feedwater).

• Training

• Procedures

• Resources

1. Training may not direct the 

team to understand how to 

approach a situation and 

prioritize the goals correctly.

2. Procedures may be 

misleading or unclear as to 

which goal should be 

accomplished first. 

Procedures also often 

represent scenarios in a 

linear fashion.

3. Resources may be limited 

such that the crew cannot 

attend to both goals 

simultaneously. 

Espinosa-Paredes 

et al. (2008)

Orasanu (1993)

Patrick, James, & 

Ahmed (2006)

Incorrect 

judgment of 

goal success

During goal setting, 

errors may occur if the 

threshold for determining 

goal satisfaction is set at 

the wrong level, and the 

goal is judged as being 

achieved before it 

actually is.

Actions may be 

implemented and 

then abandoned or 

terminated too early if 

the goal is considered 

attained when it is 

not.

• Procedures

• Training

• Knowledge/ 

Experience/ 

Expertise

1. Procedures may be 

written poorly such that it is 

hard for the operator to 

determine when success has 

been achieved.

2. Training on determining a 

value for a parameter may 

be lacking such that the 

operator is unsure if success 

has been achieved.

3. Experience with the 

system may be lacking such 

that the operator believes the 

system is in a safe state (or 

moving toward a safe state) 

when it is not.

Cacciabue et al. 

(1990)

Vicente et al.

(2004)
A-41

A-4.2 Incorrect Internal Pattern Matching

This table contains the cognitive mechanisms for the Incorrect Internal Pattern Matching 

proximate cause. When this is the cause of Decisionmaking failure, the operator has mapped 

the situation to an inappropriate mental model. During the Understanding and Sensemaking 

macrocognitive function, an operator forms a mental model of the current situation. During 

pattern matching, the operator compares this mental model to previously encountered situations 

to judge the typicality of the situation and help in devising a plan. If the operator judges the 

situation as being typical, a previous response plan can be used again. If the situation is novel, 

the operator may find a similar situation that can be adapted to fit the current situation. This 

proximate cause includes errors that occur during the mental exercise of pattern matching. 

Cognitive mechanisms include:

1. Not updating the mental model to reflect the changing state of the system. Events within an

NPP may evolve quite quickly, and the operator must update his or her mental model to 

reflect this dynamic nature.

2. Fail to retrieve previous experiences. During pattern matching, the operator compares the 

current situation to previously encountered situations to devise an appropriate response 

plan. Errors may occur in this recollection process if the operator fails to evoke appropriate 

previous experiences.

3. Incorrect recall of previous experiences. Similar to the previous failure mechanism dealing 

with the recollection of previous experiences, in this case the error may occur due to an 

incorrect recollection of the previous experience. In other words, the operator may 

incorrectly remember how the previous experience was responded to.

4. Incorrectly comparing the mental model to previously encountered situations. The 

comparison with previously encountered situations may cause an error either because the 

comparison was incomplete or simply because a mistake occurred in the comparison.

5. Cognitive biases. Confirmation bias and availability bias may be particularly pertinent to 

causing errors in this phase of decisionmaking (Einhorn & Hogarth, 1978; Tversky & 

Kahneman, 1974). Confirmation bias states that people tend to seek out evidence that 

confirms their current position. Availability bias states that the ease with which an item can 

be brought out of memory will influence the value assigned to the memory. These biases 

may affect the recollection of previously encountered situations, the comparison of the 

mental model to the previously encountered situations, or the updated mental model.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Not updating 

the mental 

model to 

reflect the 

changing 

state of the 

system

An operator may become 

fixated on the problem 

presented by the system and 

not observe how the system 

is changing and, therefore, 

not update the response plan 

to account for the changes in 

the system.

An inadequate 

monitoring strategy is 

employed by the 

team.

• Procedures

• Training

• Knowledge/ 

Experience/ 

Expertise

1. An operator may become 

fixated on the procedures to 

the point that the plant is 

not being monitored 

closely.

2. The crew has not been 

trained on what an 

adequate monitoring 

strategy should be and how 

often they should check 

various readings and plant 

states.

Burns (2000)

Murphy & 

Mitchell (1986)

Patrick et al.

(2006)

Shappell & 

Wiegmann

(2000)

Fail to 

retrieve 

previous 

experiences

An operator may fail to 

retrieve an applicable 

previously encountered 

scenario. 

The previously 

encountered scenario 

occurred (or was 

trained on) so long 

• Training

• System 

responses

1. Training is not offered 

frequently enough so that 

these relevant cases are 

Burns (2000)

Shappell & 

Wiegmann 

(2000)
A-42

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

ago, that it is no 

longer memorable. • Knowledge/ 

Experience/ 

Expertise

fresh to the memory of the 

operator.

2. The system responses 

may be ambiguous enough 

that the operator doesn’t 

recognize the situation as 

being one that was 

encountered before.

Incorrect 

recall of 

previous 

experiences

A previous encountered 

scenario may be recalled 

incorrectly either because 

the situation isn’t 

remembered right or the 

responses used in the 

previous situation are not 

recalled correctly. 

An operator is 

pressed for time and 

quickly does a mental 

comparison, but given 

the time stress, the 

operator does not 

correctly recall the 

previously 

encountered 

situation.

• Training

• Knowledge/ 

Experience/ 

Expertise

• Time load

1. Training may either be 

incomplete and does not 

include the relevant 

previously encountered 

situation, or the training is 

infrequent and the relevant 

factors about the 

previously encountered 

situation are forgotten.

2. Experience by the 

operator may be lacking 

such that he or she hasn’t 

had previous experiences 

to compare to the current 

situation.

3. If there is not much time, 

a comprehensive review 

may not be possible and 

the operator may incorrectly 

recall details about the 

previously encountered 

scenario.

Cacciabue et al.

(1990)

Klein (2008)

Incorrectly 

comparing 

the mental 

model to 

previously 

encountered 

situations

The comparison may be 

incomplete and compare 

only some of the situation to 

the mental model. For 

instance, the comparison 

may have terminated early.

The comparison to 

previously 

encountered 

situations may 

terminate early due to 

time pressure felt by 

the operator.

• Time load

• Knowledge/ 

Experience/ 

Expertise

• Training

• Attention to task

1. Time load may pressure 

the operator into making a 

decision too quickly and 

terminating the comparison 

prematurely.

2. Experienced operators 

are more likely to quickly 

recognize a typical situation 

or one that has been 

encountered previously. 

Therefore, if the operator 

lacks experience, he or she 

may spend too long in 

searching for a similar 

previous experience and 

run out of time.

3. Training may not 

correctly prepare the 

operator for comparing the 

context to the mental 

model.

4. The operator may be 

distracted such that his/her 

attention to the task of 

mental simulation is 

incomplete and the task 

ends premature.

Greitzer et al. 

(2010)

Lipshitz et al. 

(2001)

Incorrectly 

comparing 

the mental 

model to 

previously 

The mental model may be 

correct and the previously 

encountered scenario may 

be remembered correctly, 

but an error may occur in the 

comparison of the two. 

Due to time pressure, 

the operator may 

simply make a 

mistake in the 

comparison and 

therefore, not 

• Training

• Time load

• Task complexity

1. Training may not 

correctly prepare the 

operator for comparing the 

context to the mental 

model.

Greitzer et al. 

(2010)

Hoc & Amalberti 

(2005)
A-43

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

encountered 

situations

consider the situation 

typical when it is or 

think it does match a 

previously 

encountered scenario 

when it does not.

2. Time load may pressure 

the operator into making a 

decision too quickly and the 

operator therefore makes a 

mistake during the 

simulation.

3. The task, context, or

mental model may be so 

complex that the operator 

makes a mistake in doing 

the comparison.

Cognitive 

Biases

The confirmation bias states 

that people tend to seek out 

evidence that confirms their 

current position and to 

disregard evidence that 

conflicts with their current 

position.

An operator may 

recall a previously 

encountered situation 

that he or she judges 

to be very similar to 

the current situation. 

The operator will tend 

to find evidence 

supporting this view 

and ignore evidence 

that would prove 

against the view.

• Time load

• Training

1. As time pressure 

increases, the operator 

feels more pressure to form 

a response plan. Therefore, 

the operator is likely to run 

with the first similar 

situation recalled even if it 

is not that similar.

2. Training may not be 

frequent enough to keep 

multiple previously 

encountered situations 

fresh in the mind of the 

operator.

Einhorn & 

Hogarth (1978)

Klein (1998)

Milkman, 

Chugh, & 

Bazerman

(2009)

Cognitive 

Biases

The 

retrievability/imaginability/ or 

availability bias states that 

the ease with which an item 

can be brought out of 

memory will influence the 

value assigned to the 

memory. People tend to 

believe and proceed forward 

with those items that are 

easier and quicker to retrieve 

from memory.

The first remembered 

previously 

encountered scenario 

is likely to be judged 

by the operator as 

being at least 

somewhat similar to 

the current situation.

• Training

• Knowledge/ 

Experience/ 

Expertise

1. Training may have been 

limited so that the operator 

is not familiar with a large 

number of previous 

scenarios to compare 

against the present 

situation.

2. The operator may not 

have much experience and, 

therefore, is biased toward 

thinking everything is 

similar to one of the limited 

situations he or she does 

have experience with.

Lipshitz et al. 

(2001)

Tversky & 

Kahneman

(1974)
A-44

A-4.3 Incorrect Mental Simulation or Evaluation of Options

This table contains the cognitive mechanisms for the Incorrect Mental Simulation or Evaluation 

of Options proximate cause. When this is the cause of Decisionmaking failure, the operator has 

engaged in incorrect projection of a possible course of action, or has unrealistically evaluated 

the options. To evaluate the appropriateness of the different proposed actions, a mental 

simulation is done in which the operator runs through the application of the actions. The 

operator will probably not do an exhaustive test of all proposed solutions, but will choose the 

first acceptable option. This proximate cause includes errors that occur during the mental 

simulation or evaluation of options. Cognitive mechanisms include:

1. Inaccurate portrayal of action. This failure mechanism includes incorrectly characterizing 

the action (i.e., forgetting a step of the action during the mental simulation) or incorrectly 

predicting how the action will be implemented.

2. Incorrect inclusion of alternatives. The operator may forget to include some alternatives that 

should be considered.

3. Inaccurate portrayal of the system response to the proposed action. This failure mechanism 

manifests in the operator incorrectly predicting how the system will respond to the proposed 

action.

4. Misinterpretation of procedures. Response planning within the NPP is done by consulting 

procedures. An error may occur because either the wrong procedures are used to address 

the situation or the procedures have complicated logic making them difficult to use and 

understand.

5. Cognitive biases. The cognitive biases of overconfidence and the anchoring effect may be 

especially prevalent for this failure mechanism. Overconfidence affects the operator’s 

confidence in the ability of an action to work. Especially if the operator has had previous 

success with an action, he or she may be overconfident in its ability to work in the present 

case. The anchoring effect states that people are biased toward the first option they see or 

the first judgment they make. Therefore, an operator may be biased toward choosing the 

first action that occurs to him or her, and apply an unsuitable action.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Inaccurate 

portrayal of 

action

During the mental simulation 

of applying a multi-step 

action, the operator may 

forget one or more critical 

steps of the action. 

Therefore, the mental 

simulation does not 

accurately predict what the 

situation’s response to the 

action would be.

An operator 

dismissed an action 

thinking it would not 

work. However, it did 

not work because the 

operator forgot the 

action had a second 

step done after 

monitoring the 

system.

• Knowledge/ 

Experience/ 

Expertise

• Training

• Time load

1. Experience is limited in 

applying this action, so 

the operator forgot all the 

steps involved.

2. Training on the action 

is limited, and therefore, 

the operator does not 

understand the action and 

does not realize multiple 

steps are included.

3. Time pressure is 

increasing and this 

additional pressure 

causes the operator to 

make a mistake in the 

mental simulation and 

leave a step out of the 

action.

Klein (1993)

Lipshitz (1993)

Orasanu & 

Martin (1998)

Inaccurate 

portrayal of 

action

The operator may make a 

mistake during the mental 

simulation and not correctly 

The operator 

incorrectly performed 

math in his or her 

• Memory load If the action or response 

is very complex, it 

requires the operator to 

Klein (1993)

Lipshitz (1993)
A-45

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

predict the outcomes from 

taking the proposed

action(s).

mind and thought a 

certain pressure 

would be needed to 

solve the problem 

when in fact that 

pressure was too low. 

keep a lot in mind. The 

working memory has 

limited capacity; 

therefore, the operator 

may forget pieces or 

misrepresent the outcome 

of the proposed actions.

Murphy & 

Mitchell (1986)

Rasmussen 

(1983)

Inaccurate 

portrayal of 

action

An error may occur if the 

operator does not realize an 

action must be implemented 

in stages of steps. 

Therefore, the operator does 

not accurately portray how 

the action would play out.

A continuous action 

step that requires 

monitoring and 

feedback from the 

system while 

implementing the 

action may be 

imagines wrong 

during the evaluation 

of the action as an 

option.

• Knowledge/ 

Experience/ 

Expertise

Experience may be 

lacking in understanding 

plant responses and the 

need to control the plant 

in stages/steps followed 

by monitoring the change.

Gonzalez, 

Lerch, & 

Lebiere (2003)

Greitzer et al.

(2010)

Incorrect 

inclusion of 

alternatives

The construction of the 

response plan may be 

incorrect because wrong 

responses are included.

Wrong actions may 

be included because 

the response of the 

previous situation to 

the actions was not 

recalled correctly and 

the operator thought 

the action had been 

successful when it 

was not.

• Training

• Knowledge/ 

Experience/ 

Expertise

1. Training may be 

incomplete and the 

operator is unable to 

recognize the similarity to 

previously encountered 

scenarios or recall the 

previous scenarios.

2. Experience is lacking in 

dealing with NPP 

operations, and therefore, 

the operator lacks 

understanding regarding 

how the plant will react to 

certain actions and 

recognizing similarity to 

previously encountered 

scenarios.

Cacciabue et al. 

(1990)

Clemen (2001)

Greitzer et al. 

(2010)

Incorrect 

inclusion of 

alternatives

The construction of the 

response plan may be 

incorrect because responses 

that should be included are

not.

For example, the 

operator ran out of 

time for conducting 

the mental simulation 

and had to go 

forward with the best 

alternative presented 

at the time. 

Experience will help 

the operator learn to 

do a directed search 

and simulation of 

relevant course of 

action instead of a 

random and 

exhaustive search.

• Training

• Time load

• Knowledge/ 

Experience/ 

Expertise

1. Training may be 

incomplete such that 

similar previously 

encountered situations 

(either encountered by 

this plant or at other 

plants) are not trained 

well enough so that the 

operator can bring up the 

appropriate responses 

when needed.

2. The operator is 

pressed on time and there 

is not enough time to 

perform a more complete 

mental simulation and 

compare against more 

previously encountered 

scenarios.

3. Experience may be 

lacking in being familiar 

with previous scenarios 

encountered by NPPs, 

and in recognizing a 

similar situation.

Cacciabue et al. 

(1990)

Gonzalez et al. 

(2003)

Inaccurate 

portrayal of the 

system 

Incorrect judgment of time 

needed to implement the 

solution.

Especially in a novel 

situation in which the 

operator is imaging 

the impact of 

• Procedures

• Training

1. Procedures may not 

have sufficient clarity or 

detail to help the operator 

in such a novel situation.

Klein (1993)

Lipshitz (1993)

Mumaw, 

Swatzler, Roth, 
A-46

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

response to the 

proposed action

proposed actions on 

the situation, the 

operator may not 

have a correct sense 

of the time needed to 

implement the 

solution.

• Knowledge/ 

Experience/ 

Expertise

2. Training may be 

lacking on similar 

situations to help the 

operator have a better 

estimate of how the 

system will progress and 

how much time is 

available.

3. Experience may be 

lacking on similar 

situation that would help 

the operator make a 

better prediction of the 

progression of the plant 

state and a more accurate 

estimate of the time 

remaining.

& Thomas

(1994)

Shappell & 

Wiegmann

(2000)

Inaccurate 

portrayal of the 

system 

response to the 

proposed action

The operator or crew may 

incorrectly judge the amount 

of time available to 

implement the solution. The 

misjudgment may be either 

thinking there is more time 

available than there actually 

is or thinking there is less 

time available and artificially 

imposing a time constraint.

A supervisor may 

impose time pressure 

on the crew to 

resolve an issue. If 

the crew is stressed 

from time pressure, 

they are more likely 

to make mistakes 

such as taking 

shortcuts with the 

procedures and not 

fully evaluating 

outcomes from 

proposed solutions. 

• Training

• Knowledge/ 

Experience/ 

Expertise

1. Training on the 

situation was not detailed 

enough to give the 

operator a realistic sense 

of how much time would 

be available to implement 

a solution.

2. Experience is lacking 

with this particular 

scenario, and therefore, 

the operator does not 

have a realistic sense of 

how much time is 

available.

Klein (1993)

Lipshitz (1993)

Orasanu & 

Martin (1998)

Shattuck & 

Miller (2006)

Inaccurate 

portrayal of the 

system 

response to the 

proposed action

During the mental 

simulation, the operator may 

fail to include a problem 

likely to occur within the 

system with the application 

of the action. 

Especially for 

creative solutions 

applied to novel 

situations, it is difficult 

for the operator to 

imagine all that might 

go wrong with certain 

actions. 

• Knowledge/ 

Experience/ 

Expertise

• Training

1. Experience may be 

lacking, and therefore, the 

operator is faced with a 

novel situation.

2. Training on the 

situation may be lacking.

Klein (1993)

Lipshitz (1993)

Orasanu & 

Martin (1998)

Misinterpretation 

of procedures

Response planning within 

the NPP is done by 

consulting the procedures 

and being directed by them. 

An error may occur because 

the wrong procedures are 

being used for addressing 

the situation or the 

procedures are difficult to 

understand in terms of what 

actions should be included 

in the decision set.

A complex situation 

arises with multiple 

alarms and multiple 

operations being 

required. The 

operator and crew 

may be overwhelmed 

and unsure what 

procedure is correct 

for responding to the 

issues.

• Procedures

• Training

• Time load

• System 

responses

1. The procedures may 

be written poorly and are 

difficult for the operator to 

use.

2. Training on the use of 

the procedures, or how to 

switch between 

procedures, or how to use 

multiple procedures may 

be poor or too infrequent.

3. Time may be 

compromised if the 

operator has to spend 

much time trying to 

determine which is the 

appropriate procedure.

4. If the system 

responses are ambiguous 

or overly complex, it can 

mislead the operator into 

using the wrong 

procedure.

Cacciabue et al. 

(1990)

Mumaw et al.

(1994)

Roth (1997)
A-47

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Cognitive biases The operator may be 

overconfident in the ability of 

an action to work. 

(overconfidence bias)

Due to an operator’s 

previous success 

with a certain 

response plan, the 

team employs the 

same response plan 

without fully realizing 

the differences 

between the 

situations and the 

likelihood of the 

actions failing.

• Training

• Knowledge/ 

Experience/ 

Expertise

1. Training may be 

lacking on this particular 

situation, and therefore, 

the operator does not fully 

understand and 

accurately predict the 

response of the situation 

to the proposed action.

2. Experience may be 

lacking with other forms of 

this type of situation, and 

therefore, the operator 

does not understand the 

subtly of the differences 

and the impact those 

differences can have on 

the success rate of the 

proposed solution.

Klein (1998)

Milkman et al. 

(2009)

Murphy & 

Mitchell (1986)

Orasanu & 

Martin (1998)

Cognitive biases The anchoring effect occurs 

because people are biased 

toward the first option they 

see or the first judgment 

they make.

A wrong action may 

be chosen just 

because it was the 

first one imagined

• Training

• Knowledge/ 

Experience/ 

Expertise

• Time load

1. Training is lacking on 

the situation to 

understand appropriate 

response.

2. Experience is lacking 

on the situation to 

understand the 

appropriate response.

3. Time pressure is 

leading the operator to 

make a quick selection, 

and therefore, the 

operator is biased to one 

of the first actions 

imagined.

Milkmanbet al. 

(2009)

Orasanu & 

Martin (1998)

Tversky & 

Kahneman 

(1974)
A-48

A-5. ACTION

This section of Appendix A contains the cognitive mechanism tables for the macrocognitive 

function of Action. As discussed in Section 2.2.3 above, Action is not listed as a macrocognitive 

function in either the Klein et al. (2003) or Patterson & Hoffman (2012) models. However, for 

the purposes of human reliability analysis (HRA), it is important to consider performance of the 

actual task. It is necessary to be able to predict pressing the wrong button or turning the wrong 

dial, for example. Therefore, the current model includes the macrocognitive function of Action.

When considering the scope of Action, it was necessary to define the level of action 

implementation that is included in the function. A large portion of the cognitive activities 

involved in executing a procedure (or procedure steps) includes detecting appropriate cues, 

understanding the situation given those cues and the mental model of the process, and deciding 

upon the correct response (i.e., the macrocognitive functions reviewed above). For the 

purposes of this effort, Action is defined as implementing an action on the level of a single 

manual action (such as operating a valve) or a predetermined sequence of manual actions. The 

action(s) must involve the manipulation of the hardware and/or software that would 

consequently alter plant status. Therefore, communication (e.g., giving commands) and 

collecting information about the plant status are not within the scope of this function, as they are 

encompassed in Teamwork. It is assumed that the other macrocognitive functions (e.g.,

Detecting and Noticing, Understanding and Sensemaking, and Decisionmaking) have been 

carried out successfully.

Operator actions can take the form of individual control actions (e.g., turning a switch to a 

particular position; turning a pump on or off) or a sequence of actions intended to achieve a 

particular goal. An example of a sequence of actions is realigning a set of valves to change a 

flow path. In some cases, all that is required is a single, discrete, control action to achieve the 

goal. For example, manually tripping the plant generally requires a single button press (or turn 

of a switch). However, in other cases more sustained control is required. For example, in many 

situations operators are required to make continuous adjustments to maintain a parameter 

within a specified set of limits. 

Reason (1990) presented a detailed psychological analysis of the psychological causes of 

errors of execution. He divided errors of execution into two major forms: slips and lapses. 

According to Reason (1990) “Slips and lapses are errors which result from some failure in the 

execution and/or storage of an action sequence, regardless of whether or not the plan which 

guided them was adequate to achieve its objective.” Slips are errors where actions executed 

are “not as planned.” They typically occur in the performance of largely automatic tasks 

performed by a skilled individual in familiar conditions, especially when attention is diverted 

(e.g., because of distraction or preoccupation). Lapses involve failures of memory, where an 

individual may forget to perform an intended action. This is often due to a failure in prospective 

memory—memory for intended actions in the future.

Chapter 6 discusses Action in more detail.

In general, failure of the Action macrocognitive function means that the implementation of a 

correct chosen action was omitted or done incorrectly. The literature review identified two 

proximate causes as potentially leading to this failure. Those proximate causes, along with their

cognitive mechanisms, are:

1. Failure to take required action (did not attempt action) (Table A-4.1)

a. Working memory failure

b. Prospective memory failure
A-49

c. Divided attention

2. Executed desired action incorrectly (Table A-4.2)

a. Error monitoring and correction

b. Dual task interference

c. Task switching interference

d. Negative transfer/habit intrusion

e. Automaticity control

f. Mode confusion

g. Population stereotypes

h. Motor learning

i. Recognition errors

j. Stimulus response compatibility

k. Manual control issues

l. Continuous control deficiencies
A-50

A-5.1 Failure To Take Required Action (Action Not Attempted)

This table contains the cognitive mechanisms for the Failure to Take Required Action (Action 

Not Attempted) proximate cause. When this is the cause of Action failure, the operator neglects 

to implement the chosen action. This is an error of omission. In the Action macrocognitive 

function, only omissions associated with action execution are considered. For example, failing 

to take an action because of a failure in decisionmaking would be an error dealt with in the 

Decisionmaking macrocognitive function. This proximate cause can be explained by the 

following mechanisms:

1. Working memory failure

2. Prospective memory failure

3. Divided attention

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Working 

memory 

failure

Working memory has a 

limited capacity, and the 

more information that must

be kept online to complete a 

task, the more likely it is that 

an error will be committed 

due to working memory 

failure Information can be 

combined into chunks to 

increase the amount of 

information that can be 

stored in working memory. 

Researchers generally agree 

that the typical span of 

working memory is 7±2 

chunks.

An operator must 

remember a 

sequence of 10 

predetermined 

actions specified in a 

procedure. By the 

time he gets to Action 

8, he has forgotten 

the remaining actions 

in the sequence.

• Task load

• Non-task load

• Knowledge & 

experience

1. Task demands and 

non-task demands 

influence the amount of 

information that must be 

kept in working memory.

2. Experts are able to 

construct more complex 

chunks allowing for them 

to make use of greater 

amounts of information 

in working memory.

Anderson 

(2000)

Baddeley 

(1992)

Doane et al. 

(2004)

Reason 

(2008)

Prospective 

Memory 

Failure

Once an action plan has 

been decided upon, 

execution of that plan may 

fail due to forgetting (also 

known as a failure of 

prospective memory). 

Actions that are planned to 

be carried out at some point 

in the future require that an 

operator recognize a cue 

that signals that action and 

have successfully stored the 

action plan in long-term 

memory for retrieval

An operator intends

to activate a system 

as specified by a 

procedure step, but 

before he does, he is 

interrupted by the 

shift supervisor to 

complete another 

task. When he returns 

to the procedure, he 

has still not activated 

the system, but he 

continues on with the 

procedure as though 

he has.

• Task load

• Non-task load

• HSI

1. High task demands 

and interruptions make 

forgetting-the-plan more 

likely

2. Salient cues from the 

HSI or routine reminders 

may help to reduce 

failures of prospective 

memory

Dohdia & 

Dismuskes 

(2009)

Reason 

(2008)

Reason 

(2002)

Divided 

attention

Performance is degraded on 

tasks when attention is 

divided among two or more 

tasks (Proctor and van 

Zandt, 2008).

An operator must 

monitor one 

parameter 

continuously while 

executing procedure 

steps. The actions 

specified in the 

procedure steps may 

be overlooked if 

attention is captured 

by the monitoring 

task.

• Task load

• Non-task load

Task demands affect the 

degree to which 

attention has to be 

divided among tasks. 

Proctor & van 

Zandt (2008)
A-51

A-5.2 Execute Desired Action Incorrectly

This table contains the cognitive mechanisms for the Execute Desired Action Incorrectly 

proximate cause. When this is the cause of Action failure, the operator executes the chosen 

action incorrectly. This is an error of commission. This proximate cause includes errors of 

selecting the wrong control and qualitative errors such as turning a dial too much or turning a 

switch to the wrong position. Actions that are carried out too slowly, too quickly, too soon, or too 

late are also captured by this proximate cause. It is important to note that manual execution of 

most actions in the control room and in the plant do not require precise (approximately seconds 

or milliseconds) timing. This proximate cause can be explained by the following mechanisms:

1. Error monitoring and correction

2. Dual task interference

3. Task switching interference

4. Negative transfer/habit intrusion

5. Automaticity control

6. Mode confusion

7. Population stereotypes

8. Motor learning

9. Recognition errors

10. Stimulus response compatibility

11. Manual control issues

12. Continuous control deficiencies.

Mechanism Discussion Example Relevant 

PIF(s)

PIF Explanation References

Error 

monitoring 

and 

correction

When executing a motor 

response, error monitoring 

and correction depends on 

appropriate feedback from the 

system. Errors may occur if 

feedback regarding control 

state is delayed or missing

An operator may 

unintentionally 

duplicate an action 

if feedback that the 

action was carried 

out by the system is 

delayed.

• HSI

• Fatigue

• Anxiety

1. The degree to which the HSI 

provides appropriate and timely 

feedback will affect error monitoring 

and correction. Errors may occur if 

feedback regarding control state is 

delayed or missing.

2. Sleep deprivation impairs error 

monitoring and correction.

3. Anxiety impairs error monitoring 

and correction.

Wickens 

(2004)

Hsieh et al. 

(2009)

Aarts & 

Purtis 

(2010)

Dual task 

interference

Execution of one task may 

interfere with aspects of a 

second task due to “crosstalk” 

between the two tasks. This 

interference can occur in 

cognition or in coding the 

motor response.

An operator intends 

to activate a system 

as specified by a 

procedure step, but 

before he does, he 

is interrupted by the 

shift supervisor to 

complete another 

task. When he 

returns to the 

procedure, he has 

still not activated 

the system, but he 

continues on with 

the procedure as 

though he has.

• Task load

• Non-task 

load

• HSI

1. Task load and non-task load will 

influence the degree to which 

operators must perform multiple 

tasks.

2. Interference from one task to 

another depends on the similarity in 

modality of the two tasks. For 

example, two tasks requiring vocal 

responses would introduce more 

interference than one requiring a 

vocal response and the other 

requiring manual tracking. The 

modalities of the required cues and 

responses are influenced by the 

HSI. 

Reason 

(2008)

Wickens 

(1976)

Task 

switching 

interference

Switching between two tasks 

can result in interference of 

the two tasks. This 

An operator is 

delayed in taking an 

action because he 

• Task load 1. Task load and non-task load will 

determine the amount of task 

switching that must be done.

Kiesel et al. 

(2010)
A-52

Mechanism Discussion Example Relevant 

PIF(s)

PIF Explanation References

interference can occur in 

cognition or in coding the 

motor response.

took too long to 

realize he needed 

to switch.

• Non-task 

load

• HSI

2. Preparation for task switching 

(e.g., if a cue is provided prior to a 

switch to a secondary task) can 

reduce the interference, but not 

eliminate it, this can be 

accomplished via a cue or signal 

from the HSI.

Negative 

transfer/habi

t intrusion

Highly practiced responses 

may interfere with the desired 

response if the current 

situation demands an 

alternative. Repetition may 

also temporarily produce the 

same effect. 

As part of normal 

operation, an 

operator almost 

always opens Valve 

A and closes Valves 

B & C, but the 

current task 

requires him to 

open Valves B & C 

and close Valve A. 

He accidentally 

opens Valve A like 

he always does.

• Task load

• Training 

program

• Knowledge 

and 

experience

1. High task demands may narrow 

attention such that negative transfer 

is more likely

2. The training program will affect 

the types of tasks that are highly 

practiced

3. Individual knowledge and 

experience will dictate the degree to 

which certain tasks are habitual.

Reason 

(2008)

Automaticity 

control

Well practiced sequences of 

actions often become 

automated. Automaticity on a 

primary task is defined as the 

ability for an individual to 

complete both a primary and 

secondary task without a 

performance decrement to the 

primary task. When an 

automatic motor sequence 

must be changed or stopped 

due to a novel situation, the 

automatic process may be 

difficult to interrupt and/or 

control. Controlling an 

automatic sequence can 

occur based on feedback from 

the system or based on a 

contextual cue in the 

environment.

An operator must 

carry out a 

sequence of actions 

multiple times per 

day, so he can do it 

without thinking. 

However, this 

particular time, one 

of the actions failed 

because a system 

was not available. 

Because he was 

operating on 

“autopilot,” he did 

not detect this and 

finished the 

sequence.

• Training 

program

• Knowledge

and 

experience

• Task load

• HSI

1. Training programs and individual 

knowledge and experience will 

affect the degree to which actions 

are automated

2. High task demands can narrow 

attention such that operators do not 

notice environmental or contextual 

cues that signal they need to 

respond differently

3. The HSI may not provide 

appropriate cues (or they may not 

be sufficiently salient) for operators 

to stop an automatic response

Poldrack et 

al. (2005)

Population 

stereotypes

Population stereotypes dictate 

the expectancies of certain 

control behaviors (e.g., one 

expects that when a valve is 

turned counterclockwise, flow 

will increase). Violating 

population stereotypes can 

lead to errors.

A control is typically 

turned 

counterclockwise to 

increase a value; 

however, the HSI is 

designed so that 

you turn it clockwise 

to increase the 

value.

• Training 

program

• HSI

• Knowledge 

and 

experience

• Stress

1. The training program and 

individual knowledge and 

experience will influence the 

expectancies of control behaviors 

(thus affecting the population 

stereotypes).

2. The HSI should be designed such 

that controls and displays do not 

violate population stereotypes

3. Even with extensive training, 

these violations are difficult to 

overcome

4. In high stress situations, 

operators may revert back to their 

original expectations.

Swain & 

Guttman 

(1983)

Mode 

confusion

If controls (or displays) 

operate in different modes, it 

may lead to mode confusion. 

Errors may be more severe if 

the controls accomplish 

drastically different things in 

the different modes. 

Mode A activates a 

system and Mode B 

prints information 

about that system. 

The current mode is 

not conspicuously 

indicated and an 

operator who 

intends to activate 

• HSI

• Knowledge 

and 

experience

1. If the HSI is designed such that 

displays and/or controls operate in 

multiple modes, it may lead to mode 

errors. Some major factors that 

contribute to mode errors are user 

expectations, user knowledge and 

sensing of triggering events. The 

HSI can influence sensing of 

triggering events (for mode 

Degani et al. 

(1999)
A-53

Mechanism Discussion Example Relevant 

PIF(s)

PIF Explanation References

the system fails 

because it is 

actually in Mode B. 

switches) by making conditions or

events conspicuous to users. If 

mode changes and the conditions 

that cause them are not 

conspicuously indicated by the HSI, 

mode errors are more likely to occur.

2. User expectations and user 

knowledge are influenced by 

knowledge and experience.

Motor 

learning

Retention of motor skills can 

is influenced by several 

factors associated with 

practice including amount of 

practice, practice schedule 

(i.e., distributed versus 

massed and random versus 

blocked).

An operator 

performs a 

sequence of actions 

incorrectly because 

he has not practiced 

the sequence in 

training for over a 

year. 

• Training 

program

The training program will influence 

how well motor skills and sequences 

are retained. Distribution of practice 

(i.e., spreading training sessions out 

over time) leads to better retention 

of motor skills.

Schmidt & 

Bjork 

(1992).

Recognition 

errors

Misidentification of objects 

such as controls can lead to 

errors.

Two controls that do 

drastically different 

things look similar 

and are located 

side-by-side.

• HSI The method of control coding in the 

HSI (e.g., location, color coding, 

labeling, shape coding or size 

coding) will affect the likelihood that 

the right control is selected. 

The coding scheme selected for the 

HSI should be consistent with user 

expectations and conventions.

Reason 

(2008)

Proctor & 

van Zandt 

(2008)

Wickens et 

al. (2004) 

Wickens & 

Hollands 

(2000) 

Stimulus 

response 

compatibility

The relationship between the 

stimulus (e.g., a display) and 

the desired response (e.g., a 

control action) affects the 

likelihood of failure of that 

action.

An operator 

increases a value 

that he meant to 

decrease because 

the control was a 

lever that when 

pressed down 

actually increased 

the value. 

• HSI If the HSI is designed such that 

controls and/or displays violate 

compatibility principles, then it may 

increase the likelihood of errors. 

Major compatibility factors include 

location compatibility and movement 

compatibility.

Wickens et 

al. (2004)

Proctor & 

van Zandt 

(2008)

Manual 

control 

issues

The type of control (keyboard, 

selector switch, button, etc.) 

as well as specific aspects of 

the design of that control (the 

surface or size of a button, the 

resistance of a knob, etc.) has 

an impact on the likelihood of 

committing an error. See 

Procter & van Zandt (2008) or 

Wickens et al. (2004) for 

in-depth discussion of 

controls.

An operator turns a 

dial farther than he 

intends to because 

the dial does not 

have enough 

resistance and is 

too easily turned.

• HSI The degree to which human factors 

principles have been incorporated 

into the HSI will affect the likelihood 

of errors due to controls.

Proctor & 

van Zandt 

(2008)

Wickens et 

al. (2004)

Manual 

control 

issues

Human physiology (build, 

strength) has limitations that 

should be accommodated by 

the design of the workplace. 

See Proctor and van Zandt 

(2008) for more detail. 

An operator is 

unable to open a 

valve because he 

does not have the 

strength required to 

open the valve.

• HSI

• Workplace 

adequacy

1. The HSI should be designed to 

accommodate human limitations

2. Obstructions and distractions in 

the environment (e.g., a wet floor) 

may affect the likelihood of errors 

Proctor & 

van Zandt 

(2008)

Continuous 

control 

deficiencies

When an operator must take 

continuous manual control of 

a process (e.g., keeping a 

system parameter within a 

prescribed set of limits), 

human information processing 

limits and system complexities 

may lead to errors. 

An operator 

overshoots a 

system parameter 

due to 

unpredictable

shrinks and swells. 

• System 

dynamics

• HSI

1. Complex system dynamics (such 

as shrinks and swells) may 

contribute to manual control 

deficiencies, making errors more 

likely.

2. Feedback and indications from 

the HSI may exceed human 

information processing limits.

Wickens & 

Hollands 

(2000)
A-54

A-6. TEAMWORK

This section of Appendix A contains the cognitive mechanism tables for the macrocognitive 

function of Teamwork. Teamwork is the macrocognitive function that focuses on how people 

interact with each other to coordinate as the individual or crew works on a task. In the present 

effort, this function was used to represent coordination, collaboration, and communication 

between individuals, as well as to address crew interaction issues such as command and 

control.

It is important to note that this function only encompasses the emergent aspects of team 

coordination to avoid duplicating cognitive functions already described by previous 

macrocognitive functions. Building on the team sensemaking work of Klein, Wiggins, and 

Dominguez (2010), these emergent aspects are unique to and only emerge when people work 

together in teams. For example, even in a team setting, individual functions like 

detecting/noticing and sensemaking occur in parallel in all team members, such as a crew’s 

response to an alarm annunciation. Given this context of independent parallel processing, the 

essence of teamwork is the combination of these independent process efforts via 

communication for the purpose of facilitating team coordination. Using communication to 

combine the individual macrocognitive processes is one emergent aspect of teams; the other is 

leadership. NPP control room crews are hierarchical and have a distinct leadership structure.

Errors in either communication or leadership or supervision can lead to failure of the Teamwork 

function.

Chapter 7 discusses Teamwork in more detail.

In general, failure of the Teamwork function means that the team fails to properly resolve the 

independent and parallel individual macrocognitive functions. In other words, the crew 

members do not share the same understanding of the situation or what decisions have been 

made. This can lead to poor team performance and errors in individual macrocognitive 

functions. The literature review identified two proximate causes of failure of this function.

Those proximate causes, along with their cognitive mechanisms, are:

1. Failure of team communication (Table A-5.1)

a. Source error of omission

b. Source error of commission

c. Target error of omission

d. Target error of commission

e. Incorrect timing of communication.

2. Errors in leadership/supervision (Table A-5.2)

a. Decisionmaking failures

b. Failure to verify that the RO, BOP, and/or other operator have correctly performed 

their responsibilities

c. Failure to consider the information communicated by an individual

d. Failure to iterate the communication process sufficiently.

A-6.1 Failure Of Team Communication

This table contains the cognitive mechanisms for the Failure of Team Communication proximate 

cause. When this is the cause of Teamwork failure, key information is not properly shared or 

distributed among the crew members, and as a result, the entire team does not share a mental 
A-55

model of the situation, or errors may be made when taking action in response to an event. This 

proximate cause can be explained by the following cognitive mechanisms:

1. Source error of omission. The sender of the communication does not communicate the 

information.

2. Source error of commission. The sender communicates wrong information.

3. Target error of omission. The receiver of the communication does not detect or notice the 

communication, or does not comprehend the communication (Understanding and 

Sensemaking-related error).

4. Target error of commission. The receiver of the communication is listening to the wrong 

source, or is listening for wrong information (listening biased by preconceptions).

5. Incorrect timing of communication. The communication is, for example, delayed, premature, 

or communicated too slowly.

Mechanism Discussion Example(s) Relevant PIF(s) PIF Explanation References

Source error 

of omission

In any instance of 

communication in a 

crew, there are at least 

two people involved. 

There is the person 

sending or transmitting 

the information (i.e., 

source), and the person 

receiving the 

information (i.e., target). 

This mechanism is the 

instance where the 

source should 

communicate 

information to the 

target(s), but does not.

The SRO has 

information about the 

overall state of the NPP 

system that affects 

what the BOP is doing 

with just the turbines, 

but the SRO does not 

communicate that 

information to the BOP. 

(The reasons why are 

the PIFs)

The RO is hesitant to 

speak up because:

• The SRO has an 

autocratic leadership 

style and/or

• Social pressures to 

not be a “squeaky 

wheel” and/or

• He has been wrong 

too frequently in the 

past

The BOP does not 

speak up because:

• He has an incorrect 

understanding of the 

situation

• His risk perception of 

the situation is 

incorrect

• He is unaware of his 

roles and 

responsibilities, and/or 

assumes that 

someone else will 

speak up.

• Time pressure

• Deficiency in 

resource/task 

management

• Leadership style

• Team cohesion, 

Social pressure, 

Groupthink

• Knowledge/ 

experience

• Risk perception

• Role awareness

• Physical or 

psychological 

impairment

• Training, 

communication 

protocol

• Confidence in 

the information to 

be 

communicated

• Assumption of 

target’s 

knowledge 

(misplaced trust 

in target’s 

knowledge)

1. Time pressure causes the 

SRO to rush through his 

responsibilities and he 

forgets to communicate to 

the rest of the team.

2. The SRO’s leadership 

style could discourage

operators from 

communicating “bad news,” 

or challenging the SRO’s 

beliefs.

3. The operators could feel 

social pressure to get along 

and “not rock the boat” too 

much by being a “squeaky 

wheel.”

4. The operators could feel 

social pressure to not “cry 

wolf.”

5. The BOP does not deem 

and/or understand that the 

information he needs to 

communicate is important 

and/or relevant to reducing 

the overall CDF.

6. The BOP is unaware that 

communicating a piece of 

information to the SRO is 

part of his roles and 

responsibilities.

7. The SRO is physically 

impaired and/or 

psychologically fatigued and 

is unable to transmit 

information.

8. The SRO assumes the 

RO knows what to do, and 

does not communicate the 

instruction.

Helmreich & 

Foushee (1993)

Sasou & Reason 

(1999)

Janis, I. (1972, 

1982)

Latané, Williams, 

& Harkins (1979)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Carvalho et al. 

(2007)

Lee et al. (2011)

Source error 

of 

commission

This mechanism is the 

instance where the 

source transmits the 

wrong information to the 

target, or transmits the 

correct information to 

the wrong target.

The SRO inadvertently 

transmits the wrong 

information to the RO 

and/or BOP.

The BOP

communicates the 

• Time pressure

• Deficiency in 

resource/task 

management

• Task complexity

1. Time pressure causes the 

SRO to rush through his 

responsibilities and he 

inadvertently transmits the 

wrong information to the 

target, or transmits 

incomplete information, or 

Helmreich & 

Foushee (1993)

Sasou & Reason 

(1999)

Klein (2001)
A-56

Mechanism Discussion Example(s) Relevant PIF(s) PIF Explanation References

This mechanism also 

includes communicating 

information that is not 

complete, specific 

enough, vague, overly 

simplified, done in a 

confusing way or 

otherwise insufficient, in 

addition to being 

incorrect or inadequate.

correct information, but 

transmits it to the RO 

instead of the SRO.

• Knowledge/ 

experience

transmits information in a 

confusing way, etc.

2. The complexity of the task 

causes the SRO to get 

confused as to what 

information he should 

transmit next, and he 

inadvertently transmits the 

wrong information, or 

transmits incomplete 

information, or transmits 

information in a confusing 

way, etc.

3. An upset condition that the 

crew has not trained on in 

over a year occurs, and the 

BOP lacks experience to 

know that he should have 

communicated with the RO 

instead of the SRO.

Massaiu, 

Hildebrandt, & 

Bone (2011)

Carvalho et al. 

(2007)

Lee et al. (2011)

Target error 

of omission

This mechanism has 

multiple psychological 

explanations. One 

explanation is where 

the source has 

transmitted information 

with sufficient strength 

that it could be detected 

or noticed by the target, 

but the target did not 

detect or notice that the 

information had been 

communicated.

Another explanation is 

where the target has 

detected that the source 

has sent information, 

but the target is unable 

to comprehend the 

information.

Another explanation is 

that a noisy 

environment (e.g., 

alarm annunciations) 

can interfere with, 

disrupt, or garble the 

information signal from 

the source to the target, 

causing the target to 

either not Detecting and 

Noticing or comprehend 

the information.

The RO did not 

Detecting and Noticing

what the SRO has 

instructed him to do.

Broadly speaking, 

comprehension or 

sensemaking can fail 

because of bad data, 

an incorrect mental 

model/frame, or poor 

integration of data with 

the mental model. The 

target failing to 

comprehend 

information 

communicated to him 

could be because of 

any and all of these 

sensemaking failures, 

though in this example 

we assume the 

data/information sent 

from the source is 

good.

• See Detecting 

and Noticing

PIFs

• See 

sensemaking 

PIFs for incorrect 

mental 

model/frame

• See 

sensemaking 

PIFs for poor 

integration of the 

data with the 

mental model

• Noisy 

environment/prox

imity of source to 

target

• Distraction rich 

environment

• Communication 

equipment 

malfunction

• Task load

1. The RO does not 

Detecting and Noticing a 

communication from the 

SRO because the SRO did 

not get the RO’s attention 

first.

2. The RO does not correctly 

comprehend the information 

sent by the SRO because he 

has a mental model that 

leads him to interpret the 

information differently than 

intended.

3. A noisy environment can 

interfere with the information 

signal being communicated.

4. An environment with lots 

of distractions can lead to 

the RO to attend to another 

stimulus when the 

information is transmitted.

5. A target already engaged 

in a complex task requiring 

his or her focused attention 

and does not Detecting and 

Noticing the information 

transmitted.

Klein et al. (2010)

Helmreich & 

Foushee (1993)

Sasou & Reason 

(1999)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Carvalho et al. 

(2007)

Roth (1997)

Vicente et al. 

(2001)

See Detecting 

and Noticing

references.

See 

Sensemaking 

references

Target error 

of 

commission

This mechanism has 

multiple explanations. 

One explanation is 

there are two or more 

sources of information 

sending information to 

the target, and the 

target is listening to the 

wrong source.

Another explanation is 

the target’s 

preconceptions of what 

information he thinks he 

should receive next 

The BOP is expecting 

to receive information 

from the SRO, when in 

fact it is the RO that he 

needs to listen to for the 

correct information.

The RO is expecting 

the SRO to transmit 

some information 

(because of his 

experience has created 

a preconceived notion 

of what should happen 

next), but the SRO 

• Role awareness

• Knowledge/ 

experience

• See 

sensemaking 

PIFS for poor 

integration of the 

data with the 

mental model

• Task load

1. An upset condition that the 

crew has not trained on in 

over a year occurs, and the 

BOP does not remember 

that the roles of the SRO and 

RO change in this upset 

condition, and that he needs 

to listen to the RO for 

information he normally gets 

from the SRO.

2. A target already engaged 

in a complex task requiring 

his or her focused attention 

and incorrectly processes 

Klein et al. (2010)

Helmreich & 

Foushee (1993)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Carvalho et al. 

(2007)

Roth (1997)

Vicente et al. 

(2001)
A-57

Mechanism Discussion Example(s) Relevant PIF(s) PIF Explanation References

biases what he is 

listening for, and as a 

consequence listens for 

the wrong information, 

or misinterprets the 

information transmitted.

actually transmits 

information that is 

different from the RO’s 

expectations. Yet when 

the RO receives the 

SRO’s communication, 

the RO assumes the 

information is 

consistent with his 

preconceptions, and 

misinterprets the 

information transmitted.

This is a kind of 

sensemaking error –

specifically a data and 

frame integration error.

and takes action on 

information that was 

intended for a different 

target.

See 

Sensemaking 

references

Incorrect 

timing of 

communicati

on (e.g., 

delayed, 

premature, 

communicat

ed too 

slowly)

This mechanism 

describes the instances 

where the source 

correctly transmits 

information, the target 

correctly receives (i.e., 

detects and 

comprehends) the 

information, but the 

timing of the 

communication is 

incorrect. 

The SRO 

communicates a 

command designed to 

mitigate an upset 

condition from occurring 

with insufficient lead 

time to prevent the 

initiating event.

• Time pressure

• Knowledge/ 

experience

1. Time pressure causes the 

SRO to lose track of the 

moment in time (i.e., he 

loses situation awareness) 

when he needs to 

communicate a command to 

the BOP.

2. An upset condition that the 

crew has not trained on in 

over a year occurs, and the 

BOP forgets that he needs to 

communicate a critical piece 

of information within 20 

minutes of the initiating 

event.

Helmreich & 

Foushee (1993)

Carvalho et al. 

(2007)

Lee et al. (2011)

See 

Sensemaking 

references.
A-58

A-6.2 Error in Leadership/Supervision

This table contains the cognitive mechanisms for the Error in Leadership/Supervision proximate 

cause. When this is the cause of Teamwork failure, the crew fails to work in a coordinated 

fashion, or as a cohesive team. There may be supervisory or team management problems, 

such as the leader not facilitating group discussion, or failing to identify and correct an operator 

error. This failure mode can be explained by the following mechanisms:

1. Decisionmaking failures. This is covered in the Decisionmaking function.

2. Failure to verify that the RO, BOP, and/or other operator(s) or staff has correctly performed 

their responsibilities.

3. Failure to consider the information communicated by an individual.

4. Failure to iterate the communication process sufficiently.

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

Decisionmaking

failures

The hierarchical leadership 

and supervision model used 

in NPPs dictates that the 

person in charge makes the 

final decisions, monitors the 

performance of the other 

crewmembers, and controls 

the deliberation/discussion 

processes among the team. 

Thus, one kind of error in 

leadership/supervision is 

making a bad decision.

The SRO decides to 

trip the plant when 

other options existed 

to achieve success

• See 

decisionmaking

PIFs

See Decisionmaking Massaiu, 

Hildebrandt, & 

Bone (2011)

See 

Decisionmaking

references

Failure to verify 

that the RO,

BOP, and/or 

other 

operator(s) or 

staff has

correctly 

performed their 

responsibilities.

The SRO failing to perform 

his oversight duties is 

another kind of error in 

leadership/supervision.

The SRO does not 

perform a second 

check of an action 

that the RO and/or 

BOP was directed to 

complete.

• Time pressure

• Leadership style

• Safety culture

1. Time pressure 

causes the SRO to 

rush through his 

responsibilities and

he decides to skip 

performing his 

second check.

2. The SRO’s 

leadership style 

could be more 

“hands off” (not 

micromanage), so he 

decides not to 

perform his second 

check of a routine 

action.

Helmreich & 

Foushee (1993)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Carvalho et al. 

(2007)

Carvalho et al. 

(2006)

Failure to 

consider 

information 

communicated 

by an 

individual.

Another kind of error in 

leadership/supervision is 

when the SRO does not 

consider information 

communicated from the RO 

and/or BOP during the 

resolution of the Detecting 

and Noticing and/or 

sensemaking phases of the 

team working together.

The SRO correctly 

detects/notices, and 

comprehends 

information 

communicated to him 

from the RO and/or 

BOP, but does not 

take it into 

consideration.

• Time pressure

• Leadership style

• Team dynamics

• Safety culture

1. Time pressure 

causes the SRO to 

truncate the 

discussion before the 

RO or BOP can 

communicate 

additional information 

to him.

2. The SRO’s 

leadership style 

could be highly 

autocratic, which 

leads him to 

disregard any 

information 

communicated to him 

from others.

Helmreich & 

Foushee (1993)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Janis (1972, 1982)

Carvalho et al. 

(2006)
A-59

Mechanism Discussion Example Relevant PIF(s) PIF Explanation References

3. The SRO could 

believe RO (and/or 

BOP) “cry wolf” too 

frequently, so he 

ignores their input.

Failure to 

iterate the 

communication 

process 

sufficiently.

Another kind of error in 

leadership/supervision is 

more team communication 

could help diagnose the 

cause of the upset condition 

and what the correct course 

of action is, but the SRO 

truncates the 

communication.

The SRO cuts off 

debate among the 

crew about what 

corrective actions to 

take.

• Time pressure

• Leadership style

• Knowledge/ 

experience

• Safety culture

1. The SRO 

truncates the 

discussion because 

an action must be 

taken immediately.

2. The SRO’s 

leadership style and 

prior experience 

could lead him to 

believe that he 

already understands 

what is causing a 

particular upset 

condition and what to 

do about it, when in 

fact the cause of the 

upset condition is 

different.

Helmreich & 

Foushee (1993)

Massaiu, 

Hildebrandt, & 

Bone (2011)

Janis (1972, 1982)

Carvalho et al. 

(2006)

B-1

Appendix B

Cognitive Framework Diagrams

This appendix contains the cognitive framework diagrams that the project team built based on 

the literature review. As discussed in Section 2.4, the literature review focused on identifying the 

mechanisms that can lead to human error. These cognitive mechanisms were organized and 

sorted into the cognitive framework structure shown in this appendix and the supporting tables 

in Appendix A. 

This appendix should be used in conjunction with the tables in Appendix A. The diagrams in this 

appendix are a graphical depiction and summary of the information in the tables in Appendix A, 

and the tables provide explanation and support for the analyst when using the cognitive 

framework diagrams.

The generic cognitive framework is shown in Figure B-1 below. The macrocognitive functions 

are shown in purple. The proximate causes of the failure of a function are in blue. The 

mechanisms behind the proximate causes of failure are shown in turquoise, and the PIFs that 

influence those mechanisms are shown in orange. The entire framework spans numerous 

pages; for ease of use, they are broken down by macrocognitive function and proximate cause.

Figure B-1. Generic cognitive framework.
B-2

Section B-1 contains the cognitive framework trees for Detecting and Noticing. The diagrams 

are divided by proximate cause: 

B-1.1 Cue/information not perceived

B-1.2 Cue/information not attended to

B-1.2 Cue/information misperceived.

Section B-2 contains the cognitive framework trees for Understanding and Sensemaking. The 

diagrams are divided by proximate cause:

B-2.1 Incorrect data

B-2.2 Incorrect integration of data, frames, or data with a frame

B-2.3 Incorrect frame.

Section B-3 contains the cognitive framework trees for Decisionmaking. The diagrams are 

divided by proximate cause:

B-3.1 Incorrect goals or priorities set

B-3.2 Incorrect internal pattern matching

B-3.3 Incorrect mental simulation.

Section B-4 contains the cognitive framework trees for Action. The diagrams are divided by 

proximate cause:

B-4.1 Failure to take required action (Action not attempted)

B-4.2 Execute desired action incorrectly.

Section B-5 contains the cognitive framework trees for Teamwork. The diagrams are divided by 

proximate cause:

B-5.1 Failure of team communication

B-5.2 Error in leadership/supervision.

Note that in some of the trees, the mechanisms are not in the same order as they are in the 

tables in Appendix A; this is due to the need to make the tree fit on one page and has no 

bearing on the content.
B-3

B-1. DETECTING AND NOTICING FRAMEWORK
B-4

B-1.1 Cue/Information Not Perceived
B-5

B-1.2 Cue/Information Not Attended To
B-6

B-1.3 Cue/Information Misperceived
B-7

B-2. Understanding and Sensemaking Framework
B-8

B-2.1 Incorrect Data
B-9

B-2.2 Incorrect Integration
B-10

B-2.3 Incorrect Frame
B-11

B-3. Decisionmaking Framework
B-12

B-3.1 Incorrect Goals or Priorities Set
B-13

B-3.2 Incorrect Pattern Matching
B-14

B-3.3 Incorrect Mental Simulation or Evaluation of Options
B-15

B-4. Action Framework
B-16

B-4.1 Failure to Execute Desired Action
B-17

B-4.2 Execute Desired Action Incorrectly
B-18

B-5. Teamwork Framework
B-19

B-5.1 Failure of Team Communication
B-20

B-5.2 Error in Leadership/Supervision




NUREG-2114 Cognitive Basis for Human Reliability Analysis