-------------------- -- -- ---------------- ------ -- -- -------- -- -----------
                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       

                                                                       
  -------------------- -- -- ---------------- ------ -- -- -------- -- -----------

**Table of contents**

[3 Automation 3-1](#automation)

[3.1 General 3-1](#__RefHeading___Toc27187341)

[3.2 Design and evaluation 3-6](#__RefHeading___Toc27187342)

[3.3 System response and feedback 3-7](#__RefHeading___Toc27187343)

[3.4 Interface 3-8](#__RefHeading___Toc27187344)

[3.5 User acceptance and trust 3-9](#__RefHeading___Toc27187345)

[3.6 Modes 3-10](#__RefHeading___Toc27187346)

[3.7 Monitoring 3-12](#__RefHeading___Toc27187347)

[3.8 Fault management 3-16](#__RefHeading___Toc27187348)

[3.9 False alarms 3-18](#__RefHeading___Toc27187349)

[3.10 Training 3-19](#__RefHeading___Toc27187350)

[3.11 Function allocation/levels of automation
3-21](#__RefHeading___Toc27187351)

[3.12 Information automation 3-23](#__RefHeading___Toc27187352)

[3.13 Adaptive automation 3-26](#__RefHeading___Toc27187353)

[3.14 Decision aids 3-28](#__RefHeading___Toc27187354)

[3.15 Control automation 3-33](#__RefHeading___Toc27187355)

[Glossary 3-34](#__RefHeading___Toc27187356)

[References 3-35](#__RefHeading___Toc27187357)

[Index 3-43](#__RefHeading___Toc27187358)

**Table of exhibits**

[Exhibit 3.11 Levels of automation, from high to
low.............................................\...3-22](#__RefHeading___Toc29967845)

#  {#section .unnumbered}

#  {#section-1 .unnumbered}

# 

# 

#  

#   {#section-5 .unnumbered}

# 

# 

#  Page

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

#  

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

#  

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

#  

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

#  

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

#  Automation

**Definition.** **Automation** is the independent accomplishment of a
function by a device or system that was formerly carried out by a human.
\[Source: National Research Council (NRC), 1998; Parasuraman & Riley,
1997\]

## 3.1 General 

-   **3.1.1 Minimum automation human factors requirements.** An
    automated system should

a.  provide sufficient information to keep the user informed of its
    operating mode, intent, function, and output;

b.  inform the user of automation failure or degradation;

c.  inform the user if potentially unsafe modes are manually selected;

d.  not interfere with manual task performance; and

e.  allow for manual override. \[Source: Veridian (AHCI), 1998;
    Billings, 1997\]

-   **3.1.2** **Place user** **in command.** Automated systems shall
    prevent the removal of the user from the command role. \[Source:
    Billings, 1997\]

> **Discussion.** The reasoning behind this rule is twofold. First, it
> is ultimately the user who is responsible for the task. Second,
> automation is subject to failure. Therefore, it is the user, not the
> automation who must be in control of the system with the automation
> playing a subservient role. \[Source: Billings, 1997\]

-   **3.1.3 Automate only to improve performance.** Functions shall be
    automated only if they improve system performance without reducing
    human involvement, situation awareness, or human performance in
    carrying out the intended task. \[Source: Billings, 1991\]

    **Discussion.** The introduction of automation is often intended to
    reduce workload and augment performance; however, this is not always
    the result. Automation can lead to distraction from the primary
    task, increased workload, boredom, or complacency. Automation can
    also have psychosocial impacts, influencing job satisfaction or self
    worth. \[Source: Bowers, Deaton, Oser, Prince & Kolb, 1995; Danaher,
    1980; Edwards, 1976; Parasuraman, Molloy, Mouloua, & Hilburn, 1996;
    Wiener, 1989; Wiener & Curry, 1980\]

```{=html}
<!-- -->
```
-   **3.1.4** **Automate with good reason.** Automation should be used
    to support the user(s) where appropriate (human-centered
    automation), not implemented simply because the technology is
    available (technology-centered automation). \[Source: Billings,
    1997\]

```{=html}
<!-- -->
```
-   **3.1.5** **Enable** **users to carry out tasks.** Automation shall
    help or enable the users to carry out their responsibilities and
    tasks safely, efficiently, and effectively. \[Source: Billings,
    1991\]

    **Definitions.** Carrying out a task **effectively** means producing
    the desired result. Carrying out a task **efficiently** means that
    the desired result is produced with a minimum of waste (usually in
    relation to time).

-   **3.1.6 Provide a clear relationship with user tasks.** The
    relationships between display, control, decision aid, and
    information structure and user tasks and functions shall be clear to
    the user. \[Source: Nuclear Regulatory Commission (NUREG-0700),
    1996; Nuclear Regulatory Commission (NUREG/CR-6105), 1994\]

    **Discussion.** The user needs to be able to see clearly how the
    display or decision aid, and so on, facilitates the completion of
    the necessary task.

-   **3.1.7 Ensure active user involvement in operation.** Users shall
    be given an active role through relevant and meaningful tasks in the
    operation of a system regardless of the level of automation being
    employed. \[Source: AHCI, 1998; Billings, 1991\]

    **Discussion.** User awareness of system state cannot be sustained
    passively. Active involvement is essential for operators to exercise
    their responsibilities and be able to respond to emergencies.
    Reducing active involvement may be detrimental to the user's
    understanding of important information, may lead to longer response
    times in case of emergencies, or, in the long term, may lead to loss
    of relevant knowledge or skills. \[Source: Galster, Duley,
    Masalonis, & Parasuraman, 2001; Garland & Hopkin, 1994; Hopkin,
    1988; Sarter & Woods, 1992 (as found in Scerbo, 1996); Wickens, 1992
    (as found in Scerbo, 1996)\]

```{=html}
<!-- -->
```
-   **3.1.8 Make procedures suitable to user expertise.** Procedures
    employed in automation should be appropriate to the user's level of
    expertise with the system. \[Source: Defense Information Systems
    Agency (DISA), 1996\]

    **Example.** Shortcuts such as function keys can be provided for the
    more experienced users, whereas novice users can still use standard
    procedures.

-   **3.1.9** **Implement based on goals for system.** How automation is
    implemented should be determined by the explicit goals of the
    system, not by comparison between automated and manual systems.
    \[Source: Wiener & Curry, 1980\]

**Discussion.** When automation is implemented, explicit goals of the
system need to be kept in mind, thus, an automated system does not need
to perform a task the same way as it was performed manually to be
effective.

-   **3.1.10 Avoid increasing demands for cognitive resources.**
    Automation should not increase the demands for cognitive resources
    (thinking or conscious mental processes). \[Source: Bainbridge,
    1983; Parasuraman & Riley, 1997; Wiener & Curry, 1980; Woods, 1996\]

**Discussion.** Automation that increases the demand for cognitive
resources is poorly designed. Expert users in complex, dynamic systems
have been observed to cope with poorly designed automation by using only
a subset of the available functionality, especially during periods of
high workload. \[Source: Woods, 1996\]

-   **3.1.11 Avoid extreme workload levels.** Extreme levels of workload
    (low or high) due to automation use should be avoided. \[Source:
    Hilburn, Jorna, Byrne, & Parasuraman, 1996; NRC, 1993; Warm, Dember,
    & Hancock, 1996; Wiener, 1988\]

    **Discussion.** Extreme levels of workload can be caused by poorly
    designed automation. Poorly designed automation can cause extreme
    workload levels by increasing workloads when they are already high
    (for example, for pilots, during the high workload flight phases of
    take-off and landing) and decreasing workloads that are already low
    (for example, providing a pilot with the ability to engage autopilot
    during the low workload "cruise" phase of a flight). Automation is
    often introduced to reduce workload. However, reduction of workload
    may not always be advantageous, for example, if workload is already
    low. \[Source: Hilburn et al., 1996; Parasuraman & Mouloua, 1996\]

```{=html}
<!-- -->
```
-   **3.1.12 Prevent distraction from operations.** User interaction
    with automation shall not require the user to take significant
    amounts of attention away from the primary task. \[Source: Danaher,
    1980\]

**Discussion.** When automation requires the user or one member of the
user team to devote a significant amount of attention to adjusting or
monitoring the automation, this removes the user away from
minute-to-minute operations, thereby taking the user out of the loop.
This can be especially dangerous if an abnormal situation occurs that
needs to be remedied quickly. \[Source: Danaher, 1980\]

-   **3.1.13 Avoid interruption at inappropriate times.** Automation
    should not interrupt at inappropriate times such as during periods
    of high workload or during critical moments in a process. \[Source:
    Woods, 1996\]

**Discussion.** An interruption during high workload or at a critical
moment can cause a delay in the user's ability to respond to a
malfunction, leading to a potential failure. If the user is attending to
a malfunction in an automated task and is interrupted, the interruption
depletes the user's mental resources causing him to be less capable of
averting the potential failure. For example, in the cockpit, certain
automation functions might be stopped from interrupting during the
takeoff and landing portions of flight.

-   **3.1.14 Make tasks easier to perform.** An automated task should be
    less difficult to perform than the manual task it replaces.
    \[Source: AHCI, 1998\]

-   **3.1.15 Guide the use of automation.** Standard operating
    procedures and company policies should guide users in the
    appropriate use of automation, although the user should be
    ultimately responsible to make the decision to use or not use the
    automation. \[Source: Billings, 1997; Parasuraman & Riley, 1997\]

```{=html}
<!-- -->
```
-   **3.1.16 Provide easy data access.** Data that are needed by the
    user shall be easily accessible. \[Source: NUREG/CR-6105, 1994;
    NUREG-0700, 1996\]

**Discussion.** User requirements can serve as a guide of whether the
data are available at all times, accessible at the users' discretion, or
not at all if the user does not need information.

-   **3.1.17 Prompt for data entry format.** The automated system should
    prompt users as to the correct data entry format. \[Source:
    Billings, 1996\]

**Example.** If the automated system requires that the data be entered
in all capital letters, it should specifically tell the user to enter
the data in capital letters.

-   **3.1.18 Make it error resistant and error tolerant.** Automation
    should be error resistant and error tolerant. \[Source: Billings,
    1991\]

**Discussion.** To make a system **error resistant** is to make it
difficult for a user to make an error. Simplicity in design and the
provision of clear information are tools to improve error resistance.
**Error tolerance** is the ability to mitigate the effects of human
errors that are committed. Error tolerance can be improved by adding
monitoring capabilities to the automation. Electronic checklists also
have the potential to improve error resistance by providing reminders of
items that need to be completed. \[Source: Billings, 1991\]

-   **3.1.19 Make system behavior predictable.** Automated systems shall
    behave predictably so that the user knows the purpose of the
    automation and how the operation will be affected by that
    automation. \[Source: Billings, 1991, 1996\]

**Discussion.** The predictability of an automated system allows the
user to know what to expect when the automation is functioning
correctly. This makes it easier for the user to recognize when the
system is not functioning. \[Source: Billings, 1996\]

-   **3.1.20 Ensure safe operations are within human capacity.** Systems
    shall not be so reliant on automation or on human skills degraded by
    automation use that human users can no longer safely recover from
    emergencies or operate the system manually if the automation fails.
    \[Source: Billings, 1996; NRC, 1998\]

**Discussion.** A balance is needed between the efficiency created by
automation, the need for the operator to be able to recover from
emergencies, and control the system manually in case the automation
fails.

-   **3.1.21 Provide means of user override.** The automation should not
    be able to veto user actions leaving the user without means to
    override or violate the rules that govern the automation unless
    there is not enough time for the user to make a decision. \[Source:
    Garland & Hopkin, 1994; Inagaki, 1999\]

    **Discussion.** Problems with automation can occur when the
    automated options do not apply to a situation and the user is
    restricted to the options provided by the automation.

```{=html}
<!-- -->
```
-   **3.1.22 Provide interaction consistency.** The way that automation
    systems interact with their users shall reflect a high degree of
    consistency within and between systems. \[Source: NUREG-0700, 1996\]

**Discussion.** There are many possible types of interaction, such as
menu selection, direct manipulation, and form-filling. (See Chapter 8 on
computer-human interfaces for more information on interaction). An
example of inconsistent interaction would be having one system require
filling in forms as the interaction method, whereas another system
requires menu-driven interaction.

-   **3.1.23 Make systems easy to understand and use.** Automated
    systems and associated integrated information displays should be
    intuitive, easy to understand, and easy to use. \[Source: Billings,
    1991; Sarter & Woods, 1994; Woods, 1996\]

    **Discussion.** System operations that are easily interpretable or
    understandable by the user can facilitate the detection of improper
    operation and the diagnosis of malfunctions. \[Source: Wiener &
    Curry, 1980\]

-   **3.1.24 Make systems simple to learn.** Automation should be simple
    for the users to learn. \[Source: Billings, 1991; Wiener & Curry,
    1980\]

-   **3.1.25 Provide means to check input and setup data.** Automated
    systems should provide a way to check automation setup and to check
    information used as input for the automated system. \[Source: Wiener
    & Curry, 1980; Wickens, 2000\]

**Discussion.** Automation failures are often due to setup error.
Although the automated system itself could check some of the setup,
independent error-checking equipment or procedures may be needed. The
user needs to be able to distinguish whether a failure occurred due to
the automation setup or due to an inaccuracy in the input information.
An automation failure could have been caused by a malfunction of an
algorithm or by the input of inaccurate data. For example, if the
automated system relies on primary radar and secondary radar as inputs
and uses an algorithm to predict conflicts, a failure could arise from
faulty data from either the primary or secondary radar or from the
algorithm that combines this information. \[Source: Wiener & Curry,
1980; Wickens, 2000\]

## 

## 3.2 Design and evaluation

-   **3.2.1 Involve users in design.** Users should be involved in the
    design of an automated tool. \[Source: Amalberti, 1999; Billings,
    1997; Parasuraman, Sheridan, & Wickens, 2000\]

**Discussion.** Input from the user is essential in defining information
requirements.

-   **3.2.2 Design based on human-centered goals and functions.** Design
    of automation should begin by choosing the human-centered criteria
    (goals) of the system and then defining the functions that the
    system will perform. \[Source: Wiener & Curry, 1980\]

**Discussion.** Defining the goals and functions of an automated system
may require the use of task analysis.

-   **3.2.3 Consider effect on coordination.** When new automation is
    introduced, the designers shall consider the possibility of negative
    effects on team coordination. \[Source: Wiener, 1989\]

**Discussion.** Automation may deplete team interaction and cooperation
unless all parties are provided with information that allows them to be
actively involved in the task. Automation can cause physical difficulty
in seeing what the other team member is doing, reduce the ability to
cross monitor, change traditional roles and responsibilities, and change
the manner in which team members attempt to help one another. \[Source:
Danaher, 1980; Rudisill, 1994\]

-   **3.2.4 Assess overall impact.** The overall impact of automation
    shall be thoroughly examined before implementation to ensure that
    changes do not result in additional complexities, loss of
    situational awareness, or possibilities for error. \[Source: Woods,
    1996\]

**Discussion.** Automation of some user tasks may result in the user
processing less information or processing information at less depth. A
diminished understanding and appreciation for the overall situation may
result. \[Source: Garland & Hopkin, 1994\]

-   **3.2.5 Validate system design.** Contextually valid
    human-in-the-loop experiments and simulations should be conducted to
    validate and refine automated system design. \[Source: NRC, 1998\]

```{=html}
<!-- -->
```
-   **3.2.6 Evaluate interactions with other functions.** Possible
    interactions with other tools, system functions, and user tasks
    shall be evaluated when new automation is designed. \[Source: NRC,
    1998\]

-   **3.2.7 Test as a whole.** New automation components shall be tested
    with the complete system, including other automated components of
    the system, to ensure they function together as an effective whole.
    \[Source: NRC, 1998\]

-   **3.2.8 Test normal and failure modes.** Automated systems shall be
    tested under normal modes of operation and under failure modes of
    the automation. \[Source: NRC, 1998; Wickens, 2000\]

-   **3.2.9 Test before implementation.** Automated systems shall be
    tested in a realistic operational environment with representative
    users before implementation to ensure that operator performance is
    not compromised and workload is not increased. \[Source: Drury,
    1998\]

3.3 System response and feedback

-   **3.3.1 Visualize consequences of decisions.** The user should be
    able to visualize the consequences of a decision, whether made by
    the user or the automated system. \[Source: Billings, 1996\]

-   **3.3.2 Provide brief and unambiguous command response.** Automated
    system responses to user commands should be brief and unambiguous.
    \[Source: Billings, 1997\]

-   **3.3.3 Keep users aware of function.** The automated system should
    keep the user aware on a continuing basis of the function (or
    malfunction) of each automated system and the results of that
    function (or malfunction). \[Source: Billings, 1996\]

-   **3.3.4 Provide effective feedback.** Automation should provide the
    user with effective feedback on its actions and the purpose of these
    actions. \[Source: Woods, 1996\]

**Discussion.** When feedback is poor, automation is considered silent.
Silent automation may result in coordination and system failures. Users
may be surprised by the behavior of silent automation. \[Source: Woods,
1996\]

## 3.4 Interface

-   **3.4.1 Keep it simple.** The automation interfaces should represent
    the simplest design consistent with functions and tasks of the
    users. \[Source: NUREG-0700, 1996\]

    **Discussion.** Simplicity for the user is achieved by attaining
    compatibility between the design and human perceptual, physical,
    cognitive, and dynamic motor responsiveness capabilities. (See
    Chapter 8 on computer-human interfaces for more information on
    interface design.) \[Source: NUREG-0700, 1996\]

```{=html}
<!-- -->
```
-   **3.4.2 Provide interface consistency.** Human interfaces in
    automation programs and systems shall have a high degree of
    consistency. \[Source: NUREG-0700, 1996\]

**Discussion.** Consistency can be obtained by presenting information in
predictable locations and keeping elements of screens such as headers,
fields, and labels consistent in appearance and relative location
throughout a system or application. (See Chapter 8 on computer-human
interfaces for more information on interface design.) \[Source:
Shneiderman, 1998\]

-   **3.4.3 Be consistent with user expectations.** Automated systems
    and interfaces should be consistent with the expectations and
    understandings of users. \[Source: Billings, 1991, 1996\]

```{=html}
<!-- -->
```
-   **3.4.4 Make interface structure logical.** Automation interfaces
    shall reflect an obvious logic based on user task needs and
    capabilities. \[Source: NUREG/CR-6105, 1994; NUREG-0700, 1996\]

-   **3.4.5 Make location status obvious.** Interfaces and navigation
    aids shall make it easy for users to know where they are in the data
    space. \[Source: NUREG/CR-6105, 1994; NUREG-0700, 1996\]

```{=html}
<!-- -->
```
-   **3.4.6 Use spatial representations where possible.** Where
    possible, spatial representations of information should be used
    instead of verbal or textual displays in high workload situations.
    \[Source: Barnes, 1981\]

**Discussion.** Although humans are often better able to attend to
spatial representations, it is not always easy or even possible to
create spatial representations of information. \[Source: Barnes, 1981\]

-   **3.4.7 Present dynamic information in real time.** Dynamic
    information (information that changes over time) should be presented
    in real time and on demand to ensure accurate and timely
    decision-making. \[Source: Morris, Rouse, & Ward, 1985\]

## 3.5 User acceptance and trust

-   **3.5.1 Increasing user trust in automation.** To increase user
    trust in automation, automation performance should be

a.  reliable and predictable with minimal errors,

b.  robust (able to perform under a variety of circumstances),

c.  familiar (use terms and procedures familiar to the user), and

d.  useful. \[Source: Lee & Moray, 1992; Lerch & Prietula, 1989;
    Masalonis & Parasuraman, 1999; Muir, 1987 (as found in Riley, 1996);
    NRC, 1998\]

> **Discussion.** Trust in automation tends to be relatively stable.
> However, changes in trust may occur over time. User trust in
> automation can increase with reliable and predictable performance.
> Decreases in trust may occur as a result of some critical error or
> automation failure. It is more difficult for users to regain trust in
> automation after a failure than to develop an initial trust. Higher
> trust in automation is not always better because automation errors may
> be overlooked due to complacency. Decreases in trust typically occur
> suddenly, but increases happen slowly and steadily. The consequences
> of an automation failure (for example, the magnitude of an error)
> impact the decline in trust. \[Source: Lee & Moray, 1992; Lerch &
> Prietula, 1989; Masalonis & Parasuraman, 1999; Riley, 1996; NRC 1998\]

-   **3.5.2 Provide training for users to develop trust in automation
    reliability.** Training should be provided to enable the user to
    calibrate their trust in the automated system. \[Source: Cohen,
    Parasuraman, & Freeman, 1998\]

**Discussion.** Training will allow the user to develop an adequate
model of how reliable or unreliable the automation is under specific
conditions.

-   **3.5.3 Ensure automation availability.** The automated system
    should be available to the user as needed. \[Source: Morris, Rouse,
    & Ward, 1985\]

```{=html}
<!-- -->
```
-   **3.5.4 Prevent interference with user tasks.** The automated system
    shall not interfere with task performance. \[Source: Andes, 1987\]

> **Discussion.** A user will be less likely to accept an automated
> system that interferes with their ability to perform tasks. \[Source:
> Andes, 1987\]

-   **3.5.5 Provide accurate and reliable information.** Automation
    shall provide accurate and reliable information. \[Source: Andes,
    1987\]

> **Discussion.** When users believe automation to be highly reliable,
> they place greater trust in it. However, there is a trade-off involved
> with a constant high level of automation reliability and
> predictability. Constant high levels of automation reliability and
> predictability may be more likely to promote complacency and may cause
> users to monitor automation with less vigilance. \[Source: Dzindolet,
> Pierce, Beck, & Dawe, 1999; Parasuraman, Molloy, & Singh, 1993, (as
> found in Masalonis & Parasuraman, 1999); Wiener, 1981\]

-   **3.5.6 Minimize changes due to automation.** Changes in cognitive
    processing, ways of thinking, and methods and skills used for new
    automation should be minimized. \[Source: Garland & Hopkin, 1994\]

> **Discussion.** Automation that requires different kinds of cognitive
> processing, ways of thinking, and discarding of traditional methods
> and skills may cause the system to be both less efficient and less
> acceptable to the users. This could include automatic conversion of
> data into a usable format. \[Source: Garland & Hopkin, 1994\]

-   **3.5.7 Promote understanding of automation function.** To promote
    user acceptance of automation, users should be taught how an
    automated system functions. \[Source: Cohen, Parasuraman, & Freeman,
    1998; Dzindolet et al., 1999; Lehner, Mullin, & Cohen, 1989\]

> **Discussion.** The better the user understands the automation, the
> more likely the user is to trust the automation appropriately.
> Designers need to alter the false belief that automation is perfect
> and ensure that the users understand when the automation is likely to
> become unreliable. \[Source: Dzindolet et al., 1999\]

## 3.6 Modes

-   **3.6.1 Clearly identify modes and functions.** When control,
    display, or automation functions change in different modes of
    operation, mode and function identification and status should be
    clear. \[Source: Billings, 1991; Sarter & Woods, 1995\]

    **Discussion.** Lack of effective feedback on the state of
    automation (including which mode is active) can lead to mode errors.
    \[Source: Sarter & Woods, 1995\]

-   **3.6.2 Identify alternatives in rarely used modes.** Seldom-used
    modes and functions should be clearly identified. \[Source:
    Billings, 1991\]

    **Example.** As automated systems become more complex with many
    modes and functions, the cognitive burden caused by the need for
    mode awareness increases. Seldom-used modes and functions will pose
    the largest burden on the user because of a lack of familiarity.
    Enabling the user to immediately recognize the purpose of modes and
    functions, such as labeling the engine failure function "ENG OUT,"
    can lessen this burden. \[Source: Billings, 1997\]

-   **3.6.3 Make frequently used modes easy to get to.** Frequently used
    modes should be more accessible than infrequently used modes.
    \[Source: AHCI, 1998\]

-   **3.6.4 Number of modes.** The number of different modes for a given
    system should be minimized. \[Source: AHCI, 1998\]

    **Discussion.** Multiple modes will provide a means of flexibility
    but will introduce more opportunities for error. Furthermore,
    automation that has multiple modes of operation can be difficult to
    learn and can produce increases in workload. Users must understand
    and remember how and when to use each mode, and they must remember
    which mode is currently active. \[Source: Scerbo, 1996; Woods,
    1996\]

-   **3.6.5 Allow switching between modes.** The user should be able to
    easily switch between modes. \[Source: AHCI, 1998\]

-   **3.6.6 Provide consistent features and functions.** Features and
    functions that are common between display modes should be
    consistent. \[Source: AHCI, 1998\]

> **Discussion.** In the original Standard Terminal Automation
> Replacement System (STARS), the Full Service Level (FSL) and the
> Emergency Service Level (ESL) had independent and inconsistent
> interfaces requiring users to learn two different interfaces: mouse
> interaction styles and status-coding schemes. This can lead to
> additional training requirements and workload. The human factors team
> recommended that the two subsystems have identical coding strategies,
> identical access and execution of system commands, consistent data
> display formatting, and consistent monitoring and reporting of
> resources. \[Source: Standard Terminal Automation Replacement System
> Human Factors Team, 1997, 1998\]

-   **3.6.7 Alert user to potentially hazardous interactions.** The
    automated system should alert the user to the implications of
    interactions between modes, especially when they are potentially
    hazardous. \[Source: Billings, 1996\]

-   **3.6.8 Alert users of unsafe modes.** The automated system should
    either prevent the use of potentially unsafe modes or alert the user
    that a particular mode may be hazardous. \[Source: Billings, 1996\]

## 3.7 Monitoring

-   **3.7.1 Allow users to monitor automated systems.** The system shall
    be designed so that users are able to monitor the automated systems
    and the functionality of its hardware and software, including the
    display of status and trend information, as needed. \[Source:
    Billings, 1991\]

**Discussion.** One way that this can be accomplished is by providing
the user with access to raw data that the automation processes.

-   **3.7.2 Display changing data as graphic.** Changing data that must
    be monitored by the users should be displayed in a graphic format.
    \[Source: Smith & Mosier, 1986\]

```{=html}
<!-- -->
```
-   **3.7.3 Make users active in control and monitoring.** Automation
    should be designed so that users are involved in active control and
    monitoring rather than just passive monitors. \[Source: Hilburn,
    Jorna, & Parasuraman, 1995; Wickens & Kessel, 1979\]

**Discussion.** Automation failures may be easier to detect when users
are involved in both active control and monitoring, than when they are
just passive monitors. \[Source: Hilburn, et al., 1995; Wickens &
Kessel, 1979\]

-   **3.7.4 Allocate cognitive resources for monitoring.** System
    designers should allow adequate cognitive resources for monitoring
    by ensuring that task load does not become excessive. \[Source:
    Wiener & Curry, 1980\]

    **Discussion.** Users of automated systems may experience higher
    levels of mental workload than manual controllers due to monitoring,
    diagnosis, and planning, with significant cognitive demand resulting
    from relatively "simple" vigilance tasks. \[Source: Deaton &
    Parasuraman, 1993; Sheridan, 1970; Warm et al., 1996\]

-   **3.7.5 Limit monitoring time.** Users should not be required to
    perform purely monitoring tasks for longer than 20 minutes at a
    time. \[Source: Parasuraman et al., 1993; Warm et al., 1996\]

    **Discussion.** Users may become complacent in monitoring automated
    systems if they have other tasks to complete simultaneously. Such
    decrements in user monitoring of automated systems have been
    observed to occur in the laboratory in as little as 20 minutes.
    \[Source: Parasuraman et al., 1993; Warm et al., 1996\]

-   **3.7.6 Integrate displays.** When users must monitor multiple
    displays, important events should occur in the same display in order
    to promote effective monitoring performance. \[Source: Warm et al.,
    1996\]

-   **3.7.7 Minimize spatial uncertainty.** Important events should
    occur in the same location on a display in order to promote
    effective monitoring performance. \[Source: Warm et al., 1996\]

    **Discussion.** Users will be able to detect a particular event more
    easily if they know where that event will occur (i.e., spatial
    certainty). Spatial uncertainty has been shown to increase perceived
    workload and decrease performance efficiency. If users do not know
    where on a display an event will occur then they must engage in
    visual scanning to look for the event. \[Source: Adams & Boulter,
    1964; Warm et al., 1996)\]

-   **3.7.8 Provide indication of monitoring.** Automated systems that
    are without incident for long periods of time should provide some
    type of indication that the automation is still monitoring the
    system. \[Source: AHCI, 1998\]

-   **3.7.9 Warn of potential user errors.** Automated systems should be
    able to monitor user interactions and to warn of user errors.
    \[Source: Billings, 1991\]

    **Discussion.** To monitor user interactions and to warn of user
    errors, automated systems may need to be able to receive input
    information on user intentions.

-   **3.7.10 Monitor critical functions.** Critical automation functions
    should be independently monitored by the user. \[Source: Billings,
    1996\]

**Definition.** A **critical** **function** is a function that can cause
system failure when a malfunction is not attended to immediately.

**Discussion.** When a function is critical, combining the monitoring of
that critical function with other, possibly less critical functions may
lead to delays in response. When a critical function is independently
monitored, a user can respond to a malfunction very quickly (within one
second). If a user is attending to another task when there is a
malfunction, there will be a delay in the user's response (several
seconds). In this period of delayed response, the malfunction can cause
the system to fail. For this reason, critical functions require constant
attention. Critical automation functions do assist in the completion of
critical tasks, however they do not assist in freeing the user to attend
to other tasks. \[Source: Parasuraman et al., 1996\]

-   **3.7.11 Ensure adequate understanding.** Users should be given an
    adequate understanding (mental model) of how the automated system
    works in order to monitor effectively. \[Source: Carroll & Olsen,
    1988 (as found in Scerbo, 1996); Wickens, 1992 (as found in Scerbo,
    1996); Wickens & Flach, 1988; Woods, 1994 (as found in Scerbo,
    1996); Woods, 1996\]

**Discussion.** Users must possess accurate mental models of automated
systems in order to monitor effectively, comprehend current situations,
plan their actions, predict future system states, remember past
instructions, and diagnose system failures. One way to establish
adequate mental models is through training. \[Source: Scerbo, 1996;
Wickens, 1992 (as found in Scerbo, 1996); Wickens & Flach, 1988; Woods,
1994 (as found in Scerbo, 1996); Woods, 1996\]

-   **3.7.12 Provide intermittent manual control.** Intermittent periods
    of manual control should be used during extended periods of task
    automation to improve monitoring of the automation. (See adaptive
    automation-Section 3.13.) \[Source: Morrison, Cohen, & Gluckman,
    1993; Parasuraman et al., 1993\]

**Discussion.** Complacency is a major concern with automation.
Intermittent periods of manual control have been advocated as a means of
minimizing complacency. Automation may also result in the decrement of
cognitive abilities such as instrument scan and navigation/positional
\[situation\] awareness and the loss of manual skills, making
transitions from automated to conventional systems difficult. Because
automation can decrease basic manual skills, these skills should be used
and maintained. Intermittent periods of manual control during which
automation is suspended periodically can promote optimal user
performance, and allow better recovery from failure, regardless of the
type of task that is automated. \[Source: Endsley & Kiris, 1995;
Morrison et al., 1993; Rudisill, 1994; Wickens, 1992 (as found in
Scerbo, 1996)\]

-   **3.7.13 Minimize noise.** Environmental noise should be minimized
    to ensure optimal vigilance. \[Source: Warm et al., 1996\]

> **Discussion**. Vigilance will be reduced when high levels of
> intermittent noise are present in the environment, especially if the
> information processing task demands are high. Noise is defined as
> sounds that are loud, disagreeable or unwanted. Music, however, may
> act as a stimulant and offset decrements in arousal due to fatigue and
> prolonged performance. \[Source: Davies, Lang & Shackleton, 1973;
> Hancock, 1984 (as found in Warm et al., 1996); Matthews, Davies,
> Westerman, & Stammers, 2000\]

-   **3.7.14 Consider circadian rhythm effects on performance.** System
    designers should consider the effects of circadian rhythms on user
    vigilance and monitoring performance. \[Source: Colquhoun, 1977 (as
    found in Warm et al., 1996)\]

> **Discussion.** It will be most difficult for users to maintain
> monitoring performance during the early morning (8:00 a.m.) when body
> temperature is low. Performance will peak late in the day (between
> 5:00 p.m. and 9:00 p.m.) as body temperature rises. Monitoring
> performance will then decline again as body temperature drops.
> Maintaining monitoring performance can also be difficult for users
> working irregular work schedules. Working consecutive night shifts,
> prolonged work shifts, or starting hours that are too early can cause
> users to experience a desynchronization of their circadian rhythm
> caused by the accumulation of sleep deficit and fatigue. \[Source:
> Costa, 1999; Warm et al., 1996\]

-   **3.7.15 Consider potential vigilance decrements.** The effects on
    vigilance due to the use of automation should be considered before
    introducing new automation. \[Source: Warm et al., 1996\]

**Discussion.** A **vigilance decrement**, that is, a continuously
decreasing ability to maintain attention over time while monitoring, may
occur with the use of automation.

> Vigilance decrements do not occur because monitoring tasks are under
> stimulating. Rather, they require a large amount of cognitive
> resources and are often frustrating. Vigilance decrements have been
> observed to occur for both expert and novice users in high fidelity
> simulations and real-world operations. \[Source: Baker, 1962;
> Colquhoun, 1967, 1977; Mackworth, 1948, 1961; Schmidke, 1976 (as found
> in Warm et al., 1996); Warm et al., 1996\]
>
> How hard the user must work in order to maintain vigilance can be
> determined by at least two factors. First, workload is affected by the
> ease with which relevant signals can be detected. Signals that have
> low salience are more difficult to detect than signals high in
> salience. Visual fatigue will also require more effort to be expended
> in order to detect a signal. Second, musculo-skeletal fatigue
> associated with maintaining a fixed posture will increase the workload
> needed to perform optimal monitoring. \[Source: Dember, Warm, Nelson,
> Simons, Hancock, & Gluckman, 1993; Warm et al., 1996\]

## 3.8 Fault management

**Fault management** relates to how the user notices and recovers from
system failures. Such failures may or may not be detected by automation.
Fault management has been defined to include the four distinct tasks of
detection, diagnosis, prognosis, and compensation. \[Source: Rogers,
Schutte, & Latorella, 1996\]

-   **3.8.1 Ensure safety should automation fail.** Automated systems
    shall allow for manual control and preservation of safe operations
    should the automation of one or more components of the system, on
    which the automation depends, fail. \[Source: NRC, 1998\]

**Discussion.** The resumption of manual control needs to be within the
capacity of the user, without relying on manual skills that may be
degraded by the use of automation. \[Source: NRC, 1998\]

-   **3.8.2 Make failures apparent.** Automation failures shall be made
    unambiguously obvious to the user. \[Source: AHCI, 1998; Billings,
    1991\]

    **Discussion.** Stress, preoccupation, and distraction may reduce
    the user's ability to detect faults. \[Source: Rogers et al., 1996\]

```{=html}
<!-- -->
```
-   **3.8.3 Provide adequate early warning notification.** Early warning
    notification of pending automation failure or performance decrements
    should use estimates of the time needed for the user to adjust to
    task load changes due to automation failure. \[Source: Morrison,
    Gluckman, & Deaton, 1990\]

    **Discussion.** In situations where automation failure would require
    user intervention, it is useful for the user to be warned that he or
    she will need to take manual control before the automated system
    fails. Ideally, this warning needs to come in adequate time to allow
    the user to adjust to the new task load. There may, however, be
    cases where it is not possible to provide advance notification of
    pending failure or where the estimate of time needed for the user to
    take control is unknown. \[Source: Morrison et al., 1990\]

```{=html}
<!-- -->
```
-   **3.8.4 Inform user of potential failure.** The user shall be
    informed of automation performance decrements, potential failures,
    and malfunctions. \[Source: Billings, 1996\]

    **Discussion.** It can increase workload for the user to continually
    monitor the automation for failure. Advance knowledge about
    potential failures can also help the user prepare to take manual
    control.

```{=html}
<!-- -->
```
-   **3.8.5 Automate diagnostic aids.** Fault isolation, inspection, and
    checkout tasks shall be automated to the extent practical. \[Source:
    National Air Space Administration (NASA-STD-3000A), 1989\]

-   **3.8.6 Incorporate automatic self-checking components.** All
    essential electronic computer and peripheral components that are
    part of a system shall incorporate an automatic self-check
    diagnostic test of software and hardware, both at power up and at
    the request of the operator, to ensure they are functioning
    properly. \[Source: Department of Defense (MIL-STD-1472F), 1999\]

-   **3.8.7 Provide capability for on-demand system check.** On-demand
    system checkout shall be available. \[Source: NASA-STD-3000A, 1989\]

-   **3.8.8 Make sensor status verifiable.** The status of sensors on
    replacement units shall be verifiable with respect to accuracy and
    proper operation. \[Source: NASA-STD-3000A, 1989\]

-   **3.8.9 Permit status verification without disassembly.** When
    feasible, equipment shall permit verification of operational status
    prior to installation without the need for disassembly. \[Source:
    NASA-STD-3000A, 1989\]

-   **3.8.10 Permit fault detection without disassembly.** Equipment
    shall permit fault detection and isolation without removing
    components, through the use of built-in test, integrated
    diagnostics, or standard test equipment. \[Source: Department of
    Defense (MIL-STD-1800A), 1990; NASA-STD-3000A, 1989\]

-   **3.8.11 Facilitate rapid fault detection.** Equipment design shall
    facilitate rapid fault detection and isolation of defective items to
    permit their prompt removal and replacement. \[Source:
    MIL-STD-1472F, 1999; NASA-STD-3000A, 1989\]

-   **3.8.12 Identify failures without ambiguity.** Fault detection and
    isolation shall identify without ambiguity which component has
    failed. \[Source: MIL-STD-1800A, 1990; NASA-STD-3000A, 1989\]

-   **3.8.13 Provide portable diagnostic tools.** When built-in test
    equipment is not available, diagnostic tools or portable equipment
    shall be provided to aid in fault isolation. \[Source:
    NASA-STD-3000A, 1989\]

```{=html}
<!-- -->
```
-   **3.8.14 Identify first alarm event.** Automated warning systems
    should provide a means for identifying the first event in a series
    of alarm events. \[Source: NUREG-0700, 1996\]

**Discussion.** When a series of interrelated alarms occur, information
identifying which component first exceeded the set threshold can be
valuable in determining the initiating cause of a problem. \[Source:
NUREG-0700, 1996\]

-   **3.8.15 Provide sufficient diagnostic information.** The user
    should be provided with sufficient information and controls to
    diagnose automated warning system operation. \[Source: Wiener &
    Curry, 1980\]

**Discussion.** In order for the user to diagnose the automated system,
diagnostics information needs to be self-explanatory and in plain
English. The diagnostic information must provide the user with the
information they need without requiring the user to seek additional
references, or a help function, to understand the problem and the
recommended solution.

## 3.9 False alarms

-   **3.9.1 False alarm rates.** False alarm rates should not be so
    frequent as to cause the user to mistrust the automated system.
    \[Source: NUREG/CR-6105, 1994; Parasuraman, Hancock, & Olofinboba,
    1997; Wiener & Curry, 1980\]

**Discussion.** The trade-off between alerting the user to off-normal
conditions and the creation of nuisance alarms needs to be considered
when determining appropriate alarm set points. A system that is designed
to minimize misses, at all costs, is likely to have frequent false
alarms. However, automated systems that have frequent false alarms are
unlikely to be trusted or even tolerated. When there is low probability
that the alarm is a true alarm (the "cry-wolf" phenomenon), users tend
to ignore, mistrust or turn off alarms. Setting the false alarm
threshold requires careful evaluation of the trade offs between missed
signals and false alarms including not only the decision thresholds at
which the system is set, but also the probabilities of the condition to
be detected. \[Source: NRC, 1997\]

-   **3.9.2 Inform users of the probability of a true alarm.** Users
    should be informed of the inevitable occurrence of automation false
    alarms particularly when base rates are low. \[Source: NRC, 1998\]

**Discussion.** When the probability of an event is low, the odds of a
true alarm can be quite low for even a very sensitive warning system,
causing inevitable false alarms. \[Source: NRC, 1998; Parasuraman et
al., 1997\]

## 3.10 Training 

-   **3.10.1 Introducing new automation.** New automation should be
    introduced with advanced briefing and subsequent training
    procedures. \[Source: Billings, 1997; NRC, 1998; Parasuraman &
    Riley, 1997\]

    **Discussion.** The introduction of new automation may introduce
    changes in traditional roles and responsibilities, a redistribution
    of authority for tasks or changes to the nature of the cognitive
    demands imposed on the human operator. \[Source: Bowers et al.,
    1995; Wiener, 1989\]

-   **3.10.2 Prepare users for changes.** Before automation is
    introduced, users should be informed of associated changes and
    increases in the work effort, as well as the benefits associated
    with the automation. \[Source: DISA, 1996; Scerbo, 1996\]

    **Discussion.** The roles and responsibilities of the users,
    cognitive demands, and operational procedures may change as a result
    of introducing automation. \[Source: Bowers, Deaton, Oser, Prince, &
    Kolb, 1995\]

-   **3.10.3 Train users to understand automated functions.** Initial
    training in the use of automation should be sufficient for the users
    to fully understand how the automation functions within the
    particular system, as well as how to use the automation. \[Source:
    Billings, 1997\]

    **Discussion.** Lack of knowledge and understanding of how
    automation works can make it difficult for users to assess potential
    problems and may result in improper use of automation. \[Source:
    Rudisill, 1995\]

-   **3.10.4 Train users to backup automation.** Users should be
    provided with backup training in performing any tasks replaced by
    automation or in operating any backup systems replaced by
    automation. \[Source: DISA, 1996\]

-   **3.10.5 Train to recognize inappropriate use of automation.** Users
    should be trained to recognize inappropriate uses of an automated
    tool including automation bias (the use of automation in a heuristic
    manner as opposed to actively seeking and processing information).
    \[Source: DISA, 1996; Dzindolet, Pierce, Beck, & Dawe, 1999; Mosier
    & Skitka, 1999\]

    **Discussion.** There are different categories of inappropriate
    automation use, including automation bias, ignoring or turning off
    the automation, and improper implementation of automation.

    Users may rely on automated decision aids in a heuristic manner
    (referred to as **automation bias**). Using heuristics is to apply
    simple decision-making rules to make inferences or to draw
    conclusions simply and quickly. Heuristics are useful principles
    having wide application, but may not be strictly accurate. Usually a
    heuristic strategy is optimal, however, under certain conditions
    heuristics will be inappropriate and errors or misuse may occur.
    Automation bias leads to errors of omission (failure to notice
    system anomalies when automation fails) and errors of commission
    (acceptance of automated decisions without cross-checking or in
    presence of contradictory information). Training will help prevent
    automation bias and help the user learn to examine multiple sources
    of information before making a decision. Early training on
    automation bias may reduce commission errors for users new to
    automation, but may be less likely to reduce omission errors or
    errors made by expert users.

    Inappropriate use of automation may be influenced by various
    individual factors such as self-confidence in completing the task,
    trust in the automation, differential effects of fatigue, and how
    all of these factors combined weigh into the decision making
    process. Inappropriate use of automation can be due to misuse
    (automation bias, complacency), disuse (ignoring or turning off
    automation) or abuse (improper implementation of automation).
    \[Source: Dzindolet et al., 1999; Lee & Moray, 1992; Mosier &
    Skitka, 1996; Mosier, Skitka, Dunbar, Burdick, McDonnell, &
    Rosenblatt, 1998; Muir, 1987 (as found in Scerbo, 1996); Parasuraman
    & Riley, 1997; Riley, 1996\]

-   **3.10.6 Train users when to question automation.** Users should be
    trained to recognize and understand the conditions under which
    automation may be unreliable, and to learn the conditions where it
    performs well (when or when not to question the automation).
    \[Source: Cohen et al., 1998; Dzindolet et al., 1999\]

    **Discussion.** Users must learn not to categorically accept the
    recommendation of a decision aid. Understanding the automation's
    weaknesses allows users to better judge how much they should trust
    the automation without becoming overconfident in its performance.
    This recognition process may impose an additional workload on the
    user. \[Source: Dzindolet et al., 1999\]

-   **3.10.7 Avoid over-reliance on automation.** Users should be
    trained not to become overly reliant on automation. \[Source:
    Mosier, Skitka, Heers, & Burdick, 1997; Parasuraman & Riley, 1997\]

**Discussion.** When users rely on automation too much they become
susceptible to automation-induced complacency. Monitoring failures are
likely to occur when users become overly reliant on automation.
\[Source: Mosier, Skitka, & Korte, 1994; Parasuraman et al., 1993\]

-   **3.10.8 Train for risk assessment and reduction.** Users should be
    trained on risk assessment and actions needed for risk reduction.
    \[Source: Mosier & Skitka, 1999\]

```{=html}
<!-- -->
```
-   **3.10.9 Train for failure recovery transitions.** Users shall be
    trained on transitioning from automated to conventional systems.
    \[Source: Rudisill, 1994\]

**Discussion.** If automation were to fail, users need to be skilled at
both recognizing the failure and taking manual control.

-   **3.10.10 Stress interaction skills.** Training programs should
    stress user-automation interaction skills and cognitive/problem
    solving skills rather than psychomotor skills. \[Source: Sarter &
    Woods, 1994\]

    **Discussion.** Problems in automation may not be inherent in the
    technology itself. Problems can arise due to limitations in the
    integration of the user and automation. The user and automation
    should be integrated by developing a joint, distributed cognitive
    system by means of training and design. \[Source: Sarter & Woods,
    1994\]

-   **3.10.11 Train for changes due to automation.** When automation
    requires different kinds of cognitive processing, ways of thinking,
    and discarding of traditional methods and skills, then training
    should be designed to address problems related to these changes.
    \[Source: Garland & Hopkin, 1994\]

-   **3.10.12 Train to identify normal output.** Users should be trained
    on what constitutes the normal automation output so that the user
    can easily determine whether the system is functioning properly.
    \[Source: Morris, et al., 1985\]

## 3.11 Function allocation/levels of automation

There are many possible levels of automation (see Exhibit 3.11)
including: automation that automatically executes tasks, automation that
performs tasks when pre-specified conditions are met, and automation
that suggest a course of action or facilitates a decision. \[Source:
Billings, 1997; NRC, 1998; Parasuraman et al., 2000\]

> []{#__RefHeading___Toc29967845 .anchor}Exhibit 3.11 Levels of
> automation, from high to low. \[Source: NRC, 1998; Sheridan, 1996\]

  -----------------------------------------------------------------------
  The system acts autonomously without human intervention

  The system informs the user after executing the action only if the
  system decides it is necessary

  The system informs the user after executing the action only upon user
  request

  The system executes an action and then informs the user

  The system allows the user a limited time to veto before executing an
  action

  The system executes an action upon user approval

  The system suggests one alternative

  The system narrows the selection down to a few

  The system offers a complete set of action alternatives

  The system offers no assistance
  -----------------------------------------------------------------------

-   **3.11.1 Evaluate function allocation alternatives.** Alternative
    function allocations including fully manual, partially automated,
    fully automated, and adaptive allocation should be evaluated for
    feasibility and effectiveness. \[Source: Wiener & Curry, 1980\]

-   **3.11.2 Evaluate through simulation.** Alternative schemes for the
    allocation of functions should be examined in the context of the
    whole system through the use of high fidelity simulations. \[Source:
    Wiener & Curry, 1980\]

    **Discussion.** Because there may be multiple potential schemes in
    the allocation of functions, simulating these schemes in the context
    of the whole system will allow them to be evaluated properly. A
    scheme that seems to be the most appropriate in regards to
    accomplishing a specific task may not be the best choice in relation
    to the functioning of the entire automated system.

-   **3.11.3 Only automate functions performed well by machines.** Only
    functions that are performed well by machines should be automated,
    not functions that are performed better by humans. \[Source: Drury,
    1998\]

-   **3.11.4 Automate full behavioral modules.** Behavioral modules in
    their entirety should either be automated or preserved as manual
    subtasks, not fractionally (partially) automated. \[Source: Vortac,
    Barile, Albright, Truitt, Manning, & Bain, 1996\]

    **Discussion.** A behavioral module is a unitized set of actions
    that can be performed in an over-learned, automatic fashion with
    very little effort. When a set of cognitive or behavioral actions is
    frequently performed together they will eventually form a module.
    Automation that replaces only a portion of a module will produce no
    advantage in performance and may inhibit performance. \[Source:
    Vortac et al., 1996\]

-   **3.11.5 Give tasks requiring flexibility to user.** Tasks that are
    performed in an unpredictable environment requiring flexibility and
    adaptability should be allocated to the user. \[Source: AHCI, 1998\]

-   **3.11.6 Make roles and responsibilities clear.** The automated
    system should make it clear whether the user or computer is supposed
    to perform a particular task at a specific time. \[Source:
    Parasuraman & Riley, 1997\]

-   **3.11.7 Provide means for changing roles and responsibilities.**
    The automated system should provide a means for changing the
    allocation of roles and responsibilities. \[Source: Parasuraman &
    Riley, 1997\]

-   **3.11.8 Automation of high-risk actions or decisions.** For system
    tasks associated with greater uncertainty and risk, automation
    should not proceed beyond the level of suggesting a preferred
    decision/action alternative. \[Source: NRC, 1998\]

**Discussion.** High levels of automation can be used for tasks
involving relatively little uncertainty and risk. \[Source: NRC, 1998\]

## 3.12 Information automation

**Definition. Information automation** includes information acquisition
and integration. This type of automation would include filtering,
distributing or transforming data, providing confidence estimates and
integrity checks, and enabling user requests.

-   **3.12.1 Indicate if data are incomplete, missing, uncertain, or
    invalid.** The automated system should provide a means to indicate
    to the user that data are incomplete, missing, unreliable, or
    invalid or that the system is relying on backup data. \[Source:
    AHCI, 1998\]

-   **3.12.2 Provide automatic update.** When the displayed data are
    changed as a result of external events, the user should be provided
    with the option of having an automatic update of changed
    information. \[Source: AHCI, 1998\]

-   **3.12.3 Provide multiple output formats.** System designers should
    provide information in multiple formats (for example, text,
    graphics, voice, and video) to allow better communication and
    reduction of workload. \[Source: Scerbo, 1996\]

    **Discussion.** Communication will be improved by allowing
    information to be presented in the most understandable format.
    Eliminating the need to translate information into a specific format
    will reduce workload. \[Source: Scerbo, 1996\]

-   **3.12.4 Show accurate status.** Information presented to the user
    should accurately reflect system and environment status in a manner
    so that the user rapidly recognizes, easily understands, and easily
    projects system outcomes in relation to system and user goals.
    \[Source: Endsley & Kiris, 1995; NUREG-0700, 1996\]

-   **3.12.5 Minimize errors.** Error-prone conditions should be
    minimized by maintaining user awareness, providing adequate
    training, developing standard operating procedures, and fostering
    crew coordination. \[Source: Sheehan, 1995\]

> **Discussion.** Errors due to automation may arise from data entry
> errors, monitoring failures, system workarounds, and mode
> misapplication. Error-prone conditions in automated systems may result
> from lack of mode awareness, lack of situation awareness, lack of
> systems awareness, increased heads down time, over-dependence on
> automation, and interrupted crew coordination. Automation-related
> errors usually occur in conjunction with other factors such as haste,
> inattention, fatigue, or distraction. \[Source: Sheehan, 1995\]

-   **3.12.6 Information displays.** Information displays shall support
    and reinforce status and situation awareness at all times. \[Source:
    Billings, 1991, 1996\]

    **Discussion.** A primary objective of information automation is to
    maintain and enhance situation awareness. However, too much
    information presented simultaneously may become cluttered and make
    visual search difficult, interfering with status, decision-making,
    or control. It is important for the user to be able to easily locate
    needed information. \[Source: Billings, 1991\]

    The user's ability to detect a signal while monitoring varies
    inversely with the rate at which neutral background events are
    repeated. \[Source: Lanzetta, Dember, Warm, & Berch, 1987;
    Parasuraman, 1979 (as found in Warm et al., 1996)\]

```{=html}
<!-- -->
```
-   **3.12.7 Situation displays.** Event data should be combined with a
    map background when the geographic location of changing events needs
    to be shown. \[Source: Smith & Mosier, 1986\]

```{=html}
<!-- -->
```
-   **3.12.8 Present information consistent with task priorities.** Both
    the content of the information made available through automation and
    the ways in which it is presented shall be consistent with the task
    priorities. \[Source: Billings, 1996\]

```{=html}
<!-- -->
```
-   **3.12.9 Cueing important information.** When information must be
    updated quickly, the most important information should be cued to
    ensure it will be the first to be processed by the user. \[Source:
    Wickens, 2000\]

    **Discussion.** It is important that the cues be correct, as there
    may be significant costs of invalid cueing. \[Source: Wickens,
    2000\]

-   **3.12.10 Queue messages automatically.** Incoming messages should
    be queued automatically by the system so they do not disrupt current
    information handling tasks. \[Source: Smith & Mosier, 1986\]

-   **3.12.11 Highlight changed data.** Data changes that occur
    following automatic display update should be temporarily
    highlighted. \[Source: Smith & Mosier, 1986\]

-   **3.12.12 Store and prioritize lists of information.** Long lists of
    information, tasks, and so on, should be stored and prioritized by
    the automated aid to minimize the number of decision alternatives
    and reduce the visual processing load of human operators. \[Source:
    Barnes, 1981\]

-   **3.12.13 Integrate display elements only if performance is
    enhanced.** Display elements should only be integrated if it will
    enhance status interpretation, decision-making, situation awareness,
    or other aspects of task performance. \[Source: Billings, 1991\]

-   **3.12.14 Integrated displays.** Integrated displays should combine
    various information automated system elements into a single
    representation. \[Source: Billings, 1996; Parasuraman et al., 2000\]

    **Discussion.** Feedback information that is widely distributed
    among various indicators can result in insufficient monitoring of
    automation and/or mode confusion. In such cases, monitoring adequacy
    is limited by inefficient scanning patterns and information that is
    difficult to integrate. \[Source: Mosier & Skitka, 1999\]

-   **3.12.15 Automatically arrange information depending on status.**
    System information should be automatically reorganized into
    integrated or non-integrated arrangements depending on the current
    system status. \[Source: Forester, 1987; Parasuraman, et al., 1996\]

**Discussion.** Integrated information arrangement allows the user to
assess the overall status of the system. Integrating display components
into aggregated arrangements may reduce the attention demands of fault
detection. Non-integrated arrangement of components draws user attention
to system errors or other relevant information. Presenting the
information in a format relevant to the state of the system can
facilitate the ability of the user to quickly and easily assess the
system status. \[Source: Forester, 1987; Parasuraman et al., 1996\]

-   **3.12.16 Make cues equally prominent.** Automated and non-automated
    cues should be made equally prominent to enable users to collect
    confirming/disconfirming evidence before deciding on appropriate
    action. \[Source: Mosier & Skitka, 1999\]

    **Discussion.** Automation bias, the tendency to use automation in a
    heuristic manner, may be suppressed if other, non-automated sources
    of information are presented with salience equal to that of the
    automated information. \[Source: Mosier & Skitka, 1999\]

## 3.13 Adaptive automation

**Definition. Adaptive automation** is the real time allocation of tasks
to the user or automated system in a flexible manner, changing the
automation to meet current situational demands. Adaptive automation may
benefit user performance by allowing the user to remain in active
control of the system instead of becoming a passive observer. Active
control may prevent performance decrements associated with long-term
monitoring, loss of situation awareness and manual skill degradation.
\[Source: Morrison et al., 1990; NRC, 1998; Scerbo, 1996; Scerbo &
Mouloua, 1999\]

**Discussion.** Laboratory experiments have shown that short periods of
automation use (for example, 10-minute cycles of manual and automated
control) do not result in performance decrements. This suggests that
intermittent periods of manual control may help to maintain performance
in the presence of automation. \[Source: Gluckman, Carmody, Morrison,
Hitchcock, & Warm, 1993 (as found in Scerbo, 1996); Parasuraman,
Hilburn, Molloy, & Singh, 1991\]

-   **3.13.1 Help during high workload.** Automation should be designed
    to adapt by providing the most help during times of highest user
    workload, and somewhat less help during times of lowest workload.
    \[Source: Billings, 1996; Parasuraman, Mouloua & Hilburn, 1998\]

    **Discussion**. Research has shown that adaptive automation may
    reduce mental workload most effectively during periods of high
    taskload. \[Source: Hilburn et al., 1996\]

-   **3.13.2 When not to implement adaptive automation.** Adaptive
    automation should not be implemented unexpectedly or at a time when
    the user may not desire the aiding. \[Source: Scerbo, 1996\]

> **Discussion.** The timing of adaptation may have critical impact on
> user acceptance of automation. Studies show that users prefer to be in
> control of the system. However, there are times that automation may
> need to be initiated by the system, particularly when changes in
> workload occur rapidly or are unexpected by the user. \[Source:
> Harris, Goernert, Hancock, & Arthur, 1994 (as found in Scerbo, 1996)\]

-   **3.13.3 When to implement adaptive automation.** Adaptive
    automation should be implemented at the point at which the user
    ignores a critical amount of information. \[Source: Sen, 1984\]

**Discussion.** Fatigue (or other factors) may prevent users from
recognizing the best time to utilize automation and performance
decrements may consequently occur. One indication that the user is being
overloaded is an increase in the amount of information he must ignore in
order to make a timely decision. Thus, the designer can use a threshold
critical amount of ignored information as an indicator that the user is
overloaded and implement adaptive automation at that point (to help
reduce workload). What constitutes a critical amount of information can
vary depending on the particular task and may best be determined on a
system-by-system basis. \[Source: Harris, Hancock, & Arthur, 1993 (as
found in Scerbo, 1996); Sen, 1984\]

-   **3.13.4 Adapt to skill of the user.** Adaptive automation should be
    used to increase the performance of users with different skill
    levels. \[Source: Norico & Stanley, 1989\]

**Discussion.** By adapting to the skill of the user, adaptive
automation can increase the proficiency of the novice user and prevent
frustration that might otherwise occur with complex systems.

-   **3.13.5 Make adaptive automation at least as skilled as user.**
    Adaptive automation should be at least as skilled as the user, if
    not greater, to promote optimal user performance. \[Source: Woods,
    1996\]

-   **3.13.6 Modeling of human behavior.** Modeling of human behavior
    for aid-initiated intervention should at least include: task
    execution goal states, environment representation (graphical),
    situation assessment information and planning, and commitment logic.
    \[Source: Andes & Hunt, 1989\]

    **Discussion.** When modeling user behavior, it ought to be noted
    that users vary greatly in the way they employ automation. \[Source:
    Lee, 1992; Lee & Moray, 1992 (as found in Riley, 1996)\]

-   **3.13.7 Interface adaptation.** When dynamic adaptation of the
    interface is used, it should be attained by utilizing information
    provided to the system through user interactions within a specific
    context. \[Source: Norico & Stanley, 1989\]

**Discussion.** Dynamic adaptation of the interface may promote operator
acceptance of automation.

-   **3.13.8 Menu adaptation.** When dynamic adaptation of menus is
    used, the resultant menus should offer only the options that are
    relevant to the current environment. \[Source: Barnes, 1985\]

**Discussion.** Dynamic adaptation of the menus occurs when menus are
altered to reflect the needs of the current environment. This approach
may reduce user workload. \[Source: Barnes, 1985\]

-   **3.13.9 Use direct manipulation interfaces.** Direct manipulation
    interfaces should be used to minimize the impact of a transition to
    manual control. \[Source: Morrison et al., 1993\]

**Discussion.** An example of **direct manipulation** is a graphical
user interface (GUI). In direct manipulation, the user controls the
interaction with the computer by acting directly on objects on the
display screen. An object may be an icon, menu option, symbol, button,
or dialog box. (See Chapter 8 on computer-human interfaces for more
information on direct manipulation.) \[Source: Shneiderman, 1998\]

##  3.14 Decision aids 

**Definition.** **Decision aids** (sometimes referred to as decision
support systems) are automated systems that provide support to human
decision-making processes either unsolicited or by user request.
Decision aids can narrow the decision alternatives to a few or suggest a
preferred decision based on available data. \[Source: Wiener, 1988\]

-   **3.14.1 When to use.** Decision aids should be used

a.  for managing system complexity;

b.  for assisting users in coping with information overload;

c.  for focusing the user's attention;

d.  for assisting the user in accomplishing time-consuming activities
    more quickly;

e.  when limited data results in uncertainty;

f.  for overcoming human limitations that are associated with
    uncertainty, the emotional components of decision-making,
    finite-memory capacity, and systematic and cognitive biases; and

g.  for assisting the user in retrieving, retaining, representing or
    manipulating large amounts of information, combining multiple cues
    or criteria, allocating resources, managing detailed information,
    performing computations, and selecting and deciding among
    alternatives. \[Source: AHCI, 1998; DISA, 1996\]

-   **3.14.2 When to avoid.** Decision aids should not be used

a.  when solutions are obvious;

b.  when one alternative clearly dominates all other options;

c.  when there is insufficient time to act upon a decision;

d.  when the user is not authorized to make decisions; or

e.  for cognitive tasks in which humans excel, including generalization
    and adapting to novel situations. \[Source: AHCI, 1998\]

-   **3.14.3 Let users determine decision aid use.** Users should be
    able to determine when and how the decision aid should be used.
    \[Source: Parasuraman & Riley, 1997\]

-   **3.14.4 Use terms and criteria appropriate to users.** Decision
    aids should use terminology and criteria appropriate to the target
    user group. \[Source: DISA, 1996\]

-   **3.14.5 Reduce number of response options.** Decision aids should
    reduce the number of response options. \[Source: Barnes, 1985\]

    **Discussion.** The number of options that the user must consider is
    expected to decrease when a decision aid is used. Reducing the
    response options focuses the user's attention onto the most viable
    options.

-   **3.14.6 Assist user decisions.** Decision aids should assist,
    rather than replace, human decision makers by providing data for
    making judgments rather than commands that the user must execute.
    \[Source: AHCI, 1998; DISA, 1996; Parasuraman & Riley, 1997\]

-   **3.14.7 Make support consistent with mental models.** The support
    provided by decision aids should be consistent with user cognitive
    strategies and expectations (mental models). \[Source: NUREG-0700,
    1996\]

    **Definition.** A **mental model** is an individual's understanding
    of the processes underlying system operation. \[Source: NRC, 1998;
    Parasuraman et al., 1996\]

-   **3.14.8 Do not cancel ongoing user tasks.** Use of decision aids
    should not require ongoing user tasks to be cancelled. \[Source:
    NUREG-0700, 1996\]

-   **3.14.9 Minimize query of user.** Decision aids should minimize
    query of the users for information. \[Source: NUREG-0700, 1996\]

-   **3.14.10 Minimize data entry.** Decision aids should minimize user
    data entry requirements. \[Source: DISA, 1996\]

-   **3.14.11 Provide ability for planning strategy or guiding
    process.** Decision aids should be capable of planning a strategy to
    address a problem or guide a complex process. \[Source: NUREG-0700,
    1996\]

-   **3.14.12 Accept user direction.** Decision aids should accept
    direction from the users on which problem solving strategy to employ
    when alternative strategies are available. \[Source: NUREG-0700,
    1996\]

-   **3.14.13 Prioritize alternatives.** When more than one alternative
    is available, the decision aid should provide the alternatives in a
    recommended prioritization scheme based on mission and task
    analysis. \[Source: AHCI, 1998\]

-   **3.14.14 Alert user when unable to process.** Decision aids should
    alert the user when a problem or situation is beyond its capability.
    \[Source: NUREG-0700, 1996\]

-   **3.14.15 Be flexible in type and sequence of input accepted.**
    Decision aids should be flexible in the types and sequencing of user
    inputs accepted. \[Source: NUREG-0700, 1996\]

-   **3.14.16 Estimate uncertainty and rationale.** Decision aids should
    estimate and indicate the certainty of analysis and provide the
    rationale for the estimate. \[Source: NUREG-0700, 1996\]

-   **3.14.17 Make derived or processed data accessible.** When
    information used by a decision aid is derived or processed, the data
    from which it is derived should be either visible or accessible for
    verification. \[Source: Billings, 1996\]

**Discussion.** Data that are not critical for operation can be made
available only upon request.

-   **3.14.18 Provide hard copy of decision aid use.** The user should
    be able to obtain hard copy print outs of data including screen
    displays, rules and facts, data employed, hypotheses tested, and
    summary information. \[Source: NUREG-0700, 1996\]

-   **3.14.19 Allow access to procedural information.** Decision aids
    should give the user access to procedural information used by the
    aid. \[Source: Morris, Rouse & Ward, 1985; NUREG-0700, 1996\]

**Discussion.** **Procedural information** is information about the
rules or algorithms used by the decision aid. Knowledge of procedural
information fosters user acceptance of the aid because the user is able
to understand how the aid functions. As the user becomes more familiar
with a given situation, he or she requires less procedural information.
\[Source: Morris, Rouse & Ward, 1985\]

-   **3.14.20 Provide user controlled level of explanation detail.**
    When the system provides explanations to the user, it should supply
    a short explanation initially, with the ability to make available
    more detail at the user's request, including access to process
    information or an explanation for the rules, knowledge-basis, and
    solutions used by the decision aid. \[Source: DISA, 1996;
    NUREG-0700, 1996\]

**Discussion.** **Process information** is the information about how the
aid accomplishes a task. This information is required by users to decide
whether to use the aid in unfamiliar situations and for identifying the
nature and extent of malfunctions. \[Source: Morris et al., 1985\]

-   **3.14.21 Provide clear explanations to user.** When the system
    provides explanations to the user, the explanation should use terms
    familiar to the user and maintain consistency with the immediate
    task. \[Source: DISA, 1996\]

-   **3.14.22 Present information with appropriate detail.** Decision
    aids should present information at the level of detail that is
    appropriate to the immediate task, with no more information than is
    essential. \[Source: AHCI, 1998\]

-   **3.14.23 Avoid repeated information.** Decision aids should avoid
    repeating information that is already available. \[Source: AHCI,
    1998\]

-   **3.14.24 Integrate decision aids.** Decision aids should be fully
    integrated and consistent with the rest of the computer-human
    interface. \[Source: NUREG-0700, 1996\]

-   **3.14.25 Alert to newly available information.** Decision aids
    should alert the user to changes in the status of important system
    information such as when critical information becomes available
    during decision aid utilization. \[Source: NUREG-0700, 1996\]

**Discussion.** Critical information in this standard refers to
information that may have a significant impact on task completion.

-   **3.14.26 Alert to meaningful events or patterns.** Decision aids
    should automatically notify the user of meaningful patterns or
    events such as when it predicts a future problem. \[Source: AHCI,
    1998\]

-   **3.13.27 Predict based on historical data.** Decision aids should
    be able to predict future data based on historical data and current
    conditions. \[Source: AHCI, 1998\]

-   **3.14.28 Provide ability to represent relationships graphically.**
    Decision aids should be able to graphically represent system
    relationships, its rules network, and reasoning process. \[Source:
    NUREG-0700, 1996\]

-   **3.14.29 Identify simulation mode.** When decision aids have a
    simulation mode, entering the simulation mode should require an
    explicit command and result in a distinguishable change in output.
    \[Source: NUREG-0700, 1996\]

```{=html}
<!-- -->
```
-   **3.14.30 Provide knowledge of intent.** Each element in an
    intelligent human-machine system shall have knowledge of the intent
    of the other elements. \[Source: Billings, 1996; NRC, 1998;
    Parasuraman et al., 2000\]

**Discussion.** Monitoring of the system by the user and the user by the
system can only be effective if each knows what the other one is trying
to accomplish. \[Source: Billings, 1996\]

-   **3.14.31 Adapt with situational demands.** When adaptive decision
    aiding is used, the level of decision aiding should change with the
    situational demands in order to optimize performance (See Section
    3.13 on adaptive automation). \[Source: Rouse, 1988\]

**Discussion.** The criticality of a given task can change dramatically
depending on the current situation. \[Source: Derrick, 1988\]

-   **3.14.32 Adaptive decision aiding implementation.** Adaptive
    decision aiding should be applied when resource loading,
    performance, error frequency, and deviations from intent exceed
    threshold levels (See Section 3.13 on adaptive automation).
    \[Source: Andes, 1987\]

**Discussion.** Resource loading, performance, errors, and deviations
from intent can be used as indicators to determine when the user might
need the help of the automated decision aid. The threshold levels of
these indicators, specifying the optimal time to implement decision
aiding may need to be determined on a system-by-system basis, possibly
through simulation.

-   **3.14.33 Provide planning assistance.** Adaptive decision aiding
    interfaces should allow the user to receive direct assistance in
    planning how to carry out the intended task. \[Source: Tyler & Treu,
    1989\]

-   **3.14.34 Allow user to initiate automation implementation.** The
    user should be able to initiate automated aids even if
    system-initiated automation is the norm. \[Source: Billings, 1997;
    Harris, Hancock, Arthur, & Caird, 1995\]

**Discussion.** User acceptance of automation centers on whether the
user feels in control of the system. \[Source: Rouse, 1988\]

## 3.15 Control automation

**Definition.** **Control automation** is when the system executes
actions or control tasks with some level of autonomy.

-   **3.15.1 Make automated tasks easily understood.** When automated
    control actions are performed, the automated tasks should be easily
    understood by users and similar to user control actions. \[Source:
    Billings, 1991\]

-   **3.15.2 Limit control automation authority.** Control automation
    should not be able to jeopardize safety or make a difficult
    situation worse. \[Source: AHCI, 1998\]

-   **3.15.3 Provide appropriate range of control options.** Automated
    systems should provide the user with an appropriate range of control
    options that are flexible enough to accommodate the full range of
    operating conditions for which it was certified. \[Source: AHCI,
    1998; Parasuraman & Riley, 1997; Sarter & Woods, 1995\]

    **Discussion.** Highly flexible automated systems can be useful when
    the user knows how to implement the various options across a wide
    spectrum of operational situations. However, the multiple options
    that are associated with highly flexible systems also require
    additional cognitive resources in order for the user to remember
    which mode is active. \[Source: Woods, 1996\]

```{=html}
<!-- -->
```
-   **3.15.4 Provide immediate feedback.** To promote successful
    situation awareness of the automated system, the user shall be given
    immediate feedback to command and control orders. \[Source: Morris &
    Zee, 1988\]

```{=html}
<!-- -->
```
-   **3.15.5 Allow for different user styles.** Control automation
    should be flexible enough to allow for different user styles and
    responses without imposing new tasks on users or affecting
    automation performance. \[Source: Wiener & Curry, 1980; Woods,
    1996\]

```{=html}
<!-- -->
```
-   **3.15.6 Make available override and backup alternatives.** Override
    and backup control alternatives shall be available for automation
    controls that are critical to the integrity of the system or when
    lives depend on the system. \[Source: Billings, 1991\]

-   **3.15.7 Make backup information easy to get.** Information for
    backup or override capability shall be readily accessible. \[Source:
    Billings, 1991\]

```{=html}
<!-- -->
```
-   **3.15.8 Allow overriding out-of-tolerance conditions.** When a user
    might need to operate in out-of-tolerance conditions, then a
    deliberate overriding action should be possible. \[Source: Billings,
    1991\]

**Discussion.** There may be cases, particularly in an emergency
situation, when the user needs to operate in out-of-tolerance
conditions. \[Source: Billings, 1996\]

[]{#__RefHeading___Toc27187356 .anchor}Glossary

**Adaptive automation -** The real time allocation of tasks to the user
or automated system in a flexible manner, changing the automation to
meet current situational demands. Adaptive automation may benefit user
performance by allowing the user to remain in active control of the
system instead of becoming a passive observer.

**Automation** - A device or system that independently carries out a
task that was formerly carried out by a human.

**Automation bias** - When users rely on automated decision aids in a
heuristic manner.

**Control automation** - Control automation is when an automated system
executes actions or control tasks with some level of autonomy.

**Critical** **function** - A critical function is a function that can
cause system failure when a malfunction is not attended to immediately.

**Decision aids** - Decision aids (sometimes referred to as decision
support systems) are automated systems that provide support to human
decision-making processes either unsolicited or by user request.
Decision aids can narrow the decision alternatives to a few or suggest a
preferred decision based on available data.

**Direct manipulation** - Direct manipulation is when the user controls
the interaction with the computer by acting directly on objects on the
display screen. An object may be an icon, menu option, symbol, button,
or dialog box. An example of direct manipulation is a GUI.

**Effectively** - Carrying out a task effectively means producing the
desired result.

**Efficiently** - Carrying out a task efficiently means that the desired
result is produced with a minimum of waste (usually in relation to
time).

**Information automation** - Includes information acquisition and
integration. This type of automation would include filtering,
distributing or transforming data, providing confidence estimates and
integrity checks, and enabling user requests

**Mental model** - A mental model is an individual's understanding of
the processes underlying system operation.

[]{#__RefHeading___Toc27187357 .anchor}References

Adams, J. A., & Boulter, L. R. (1964). Spatial and temporal uncertainty
as determinants of vigilance performance. *Journal of Experimental
Psychology,* 52, 204-208.

Amalberti, R. (1999). *Automation in aviation: A human perspective.*
Mahwah, NJ: Lawrence Erlbaum Associates.

Andes, R. C. (1987). Adaptive aiding in complex systems: An
implementation. *Proceedings of the 1987 IEEE Conference on Systems,
Man, and Cybernetics.* New York: Institute of Electrical and Electronics
Engineers.

Andes, R. C. & Hunt, R. M. (1989). *Adaptive aiding for human-computer
control: Final report and future directions of research* (Tech. Report.
086084-3240-51). Dayton, OH: AAMRL Laboratory.

Bainbridge, L. (1983). Ironies of automation. *Automatica,* 19*,*
775-770*.*

Baker, C. H. (1962). *Man and radar displays.* New York: Macmillan.

Barnes, M. J. (1981). *Human information processing guidelines for
decision-aiding displays* (Tech. Report NWC-TM-4605). Chine Lake, CA:
Naval Weapons Center.

Barnes, M. J. (1985). An information-processing approach to decision
aiding. *Proceedings of the IEEE International Conference on Systems,
Man, and Cybernetics (*pp. 636-640), New York: Institute of Electrical
and Electronics Engineers.

Billings, C. E. (1991). *Human --centered aircraft automation: A concept
and guidelines*. Moffett Field, CA: National Aeronautics and Space
Administration, Ames Research Center.

Billings, C. E. (1996). *Human --centered aviation automation:
Principles and guidelines*. Moffett Field, CA: National Aeronautics and
Space Administration, Ames Research Center.

Billings, C. E. (1997). *Aviation automation: The search for a
human-centered approach.* Mahwah, NJ: Lawrence Erlbaum Associates.

Bowers, C., Deaton, J., Oser, R., Prince, C., & Kolb, M. (1995). Impact
of automation on aircrew communication and decision-making performance.
*The International Journal of Aviation Psychology, 5*, 145-167.

Cohen, M. S., Parasuraman, R., & Freeman, J. T. (1998). *Trust in
decision aids: A model and its training implications* (Technical Report
USAATCOM TR 97-D-4). Arlington, VA: Cognitive Technologies, Inc.

Colquhoun, W. P. (1967). Sonar target detection as a decision process.
*Journal of Applied Psychology, 51*, 187-190.

Costa, G. (1999). Fatigue and biological rhythms. In D. J. Garland, J.A.
Wise & V. D. Hopkin, (Eds.) *Handbook of Aviation Human Factors.* (pp.
235-255). Mahwah, New Jersey, Lawrence Erlbaum Associates.

Danaher, J. W. (1980). Human error in ATC system operation. *Human
Factors*, *22,* 535-545.

Davies, D. R., Lang, L., & Shackleton, V. J. (1973). The effects of
music and task difficulty on performance at a visual vigilance task.
*The British Journal of Psychology*, 64, 383-389.

Deaton, J. E., & Parasuraman, R. (1993). Sensory and cognitive
vigilance: Age, event rate, and subjective workload. *Human
Performance,* 4, 71-97.

Defense Information Systems Agency. (1996). *Department of Defense human
computer interface style guide. Department of Defense technical
architecture framework for information management Vol. 8, (Version
3.0).* (DISA). Washington, DC: Defense Information Systems Agency, Joint
Interoperability and Engineering Organization.

Dember, W. N., Warm, J. S., Nelson, W. T., Simons, K. G., Hancock, P.
A., & Gluckman, J. P. (1993). The rate of gain of perceived workload in
sustained attention. *Proceedings of the Human Factors Society 37^th^
Annual Meeting*, Santa Monica, CA: Human Factors and Ergonomics Society,
1388-1392.

Department of Defense. (1990). *Human engineering performance
requirements for systems* (MIL-STD-1800A). Philadelphia, PA: Navy
Publishing and Printing Office.

Department of Defense. (1999). *Design criteria standard -- human
engineering* (MIL-STD-1472F)*.* Philadelphia, PA: Navy Publishing and
Printing Office.

Derrick, W. L. (1988). Dimensions of operator workload. *Human factors,
30*(1), 95-11C.

Drury, C. G. (1998). Human factors in aviation maintenance and
inspection. In *Human factors guide for aviation maintenance, Chapter 9,
Automation.* Retrieved from World Wide Web http://hfskyway.faa.gov.

Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (1999).
Misuse and disuse of automated aids. *Proceedings of the Human Factors
and Ergonomics Society 43^rd^ Annual Meeting.* Santa Monica, CA: Human
Factors Society, 339-343.

Edwards, E. (1976). Some aspects of automation in civil transport
aircraft. In T.B. Sheridan & G. Johannsen (Eds.), *Monitoring behavior
and supervisory control.* New York: Plenum.

Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop performance
problem and level of control in automation. *Human Factors,* 37,
381-394.

Forester, J. A. (1987). An assessment of variable format information
presentation. *Information Management and Decision Making in Advanced
Airborne Weapon Systems.* AGARD Conference, Toronto, Ont., Canada,
9/1-13.

Galster, S., Duley, J. A., Masalonis, A., & Parasuraman, R. (2001). Air
traffic controller performance and workload under mature free flight:
Conflict detection and resolution of aircraft self-separation.
*International Journal of Aviation Psychology*, 11, 71-93.

Garland, D. J., & Hopkin, V. D. (1994). Controlling automation in future
air traffic control: The impact on situational awareness. In R.D.
Gilson, D.J. Garland, & J.M. Koonce (Eds.), *Situational awareness in
complex systems: Proceedings of a CAHFA Conference* (pp. 179-197).
Daytona Beach, FL: Embry Riddle Aeronautical University Press.

Harris, W. C., Hancock, P. A., Arthur, E. J., & Caird, J. K. (1995).
Performance, workload, and fatigue changes associated with automation.
*The International Journal of Aviation Psychology,* 5, 169-185.

Hilburn, B., Jorna, P. G. A. M., Byrne, E. A., & Parasuraman, R. (1996).
*The effect of adaptive Air Traffic Control (ATC) decision aiding on
controller mental workload*. National Aerospace Laboratory Technical
Publication (NLR TP 96216 L). The Netherlands.

Hilburn, B., Jorna, P. G. A. M., & Parasuraman, R. (1995). The effect of
advanced ATC automation on mental workload and monitoring performance:
An empirical investigation in Dutch airspace. *Proceedings of the 8^th^
International Symposium on Aviation Psychology.* Columbus, OH: The Ohio
State University.

Hopkin, V. D. (1988). Air Traffic Control*.* In E.L. Wiener and D.C.
Nagel (Eds.), *Human factors in aviation* (pp. 639-663). San Diego:
Academic Press.

Inagaki, T. (1999). Situation-adaptive autonomy for time-critical
takeoff decisions. *International Journal of Modeling and Simulation,
19*(4).

Lanzetta, T. M., Dember, W. N., Warm, J. S., & Berch, D. B. (1987).
Effects of task type and stimulus homogeneity on the event rate function
in sustained attention. *Human Factors, 29*, 625-633.

Lee, J. (1992). *Trust, self-confidence, and operators' adaptation to
automation.* Unpublished doctoral thesis, University of Illinois,
Champaign.

Lee, J., & Moray, N. (1992). Trust, control strategies and allocation of
function in human-machine systems. *Ergonomics, 35*, 1243-1270.

Lehner, P. E., Mullin, T. M., & Cohen, M. S. (1989). Adaptive decision
aids: Using fallible algorithms to support decision making. *Proceedings
of the IEEE International Conference of Systems, Man, and Cybernetics,*
893-894.

Lerch, F., & Prietula, M. (1989). How do we trust machine advice? In G.
Salvendy & M. Smith (Eds.), *Designing and using human-computer
interfaces and knowledge-based systems* (pp. 410-419). Amsterdam:
Elsevier Science.

Mackworth, N. H. (1948). The breakdown of vigilance during prolonged
visual search. *Quarterly Journal of Experimental Psychology, 1,* 6-21.

Mackworth, N. H. (1961). Researches on the measurement of human
performance*.* In H.W. Sinaiko (Ed.), *Selected papers on human factors
in the design and use of control systems* (pp. 174-331). (Reprinted from
Medical research council Special Report Series 268, London, H. M.
Stationary Office, 1950).

Masalonis, A. J., & Parasuraman, R. (1999). Trust as a construct for
evaluation of automated aids: Past and future theory and research*.*
*Proceedings of the Human Factors and Ergonomics Society 43^rd^ Annual
Meeting,* Santa Monica, CA: Human Factors Society, 184-188.

Matthews, G., Davies, D. R., Westerman, S. J., & Stammers, R. B. (2000).
*Human Performance: Cognition, stress, and individual differences.*
Hove, East Sussex, UK: Psychology Press.

Milŏsević, S. (1974). Effect of time and space uncertainty on a
vigilance task. *Perception & Psychophysics, 15*, 331-334.

Morris, N. M., Rouse, W. B., & Ward, S. L. (1985). Information
Requirements for effective human decision making in dynamic task
allocation. *Proceedings of the 1985 IEEE Conference on Systems, Man,
and Cybernetics, IEEE, New York,* 720-724.

Morris, N. M., & Zee, T. A. (1988). *Adaptive aiding for human-computer
control: Evaluation of an enhanced task environment* (Final Report for
Project 086084-3240-51). Norcross, GA: Search Technology.

Morrison, J. G., Cohen, D., & Gluckman, J. P. (1993*).* Prospective
principles and guidelines for the design of adaptively automated
crewstations. In J.G. Morrison (Ed.), *The Adaptive Function Allocation
for Intelligent Cockpits (AFAIC) program: Interim research and
guidelines for the application of adaptive automation* (Technical Report
No. NAWCADWAR-93931-60). Warminster, PA: Naval Air Warfare Center,
Aircraft Division.

Morrison, J. G., Gluckman, J. P., & Deaton, J. E. (1990). *Adaptive
function allocation for intelligent cockpits. Cockpit automation study
1: Baseline study* (Tech. Report NADC-91028-60). Warminster, PA: NADC.

Mosier, K. L., & Skitka, L. J. (1996*).* Human decision makers and
automated decision aids: Made for each other? In R. Parasuraman & M.
Mouloua (Eds.), *Automation and human performance: Theory and
applications.* Mahwah, NJ: Lawrence Erlbaum Associates.

Mosier, K. L., & Skitka, L. J. (1999). Automation use and automation
bias. *Proceedings of the Human Factors and Ergonomics Society 43^rd^
Annual Meeting*, Santa Monica, CA: Human Factors Society, 344-348.

Mosier, K. L., Skitka, L. J., Dunbar, M., Burdick, M., McDonnell, L., &
Rosenblatt, B. (1998). Automation bias and errors: Are teams better than
individuals? *Proceedings of the 42^nd^ Annual Meeting of the Human
Factors and Ergonomics Society.* Santa Monica, CA: Human Factors
Society, 201-205.

Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. D. (1997).
Patterns in the use of cockpit automation. In M. Mouloua & J. Koonce
(Eds.), *Human-automation interaction: Research and practice.* (pp.
167-173). Hillsdale, NJ: Lawrence Erlbaum Assoc., Inc.

Mosier, K.L., Skitka, L. J., & Korte, K. J. (1994*).* Cognitive and
social psychological issues in flight crew/automation interaction. In M.
Mouloua and R. Parasuraman (Eds.), *Human performance in automated
systems: Current research and trends.* (pp. 191-197). Hillsdale, NJ:
Lawrence Erlbaum Associates, Inc

National Aeronautics and Space Administration (1989). *Man-systems
integration standards* (NASA-STD-3000A). Houston, TX: National
Aeronautics and Space Administration.

National Research Council. (1993*).* *Workload transition: Implications
for individual and team performance.* C.D. Wickens, (Eds.). Washington,
DC: National Academy Press.

National Research Council. (1997*).* *Flight to the future: Human
factors in air traffic control.* C. D. Wickens, A.S. Mavor, R.
Parasuraman, & J.P. McGee (Eds.). Washington, DC: National Academy
Press.

National Research Council. (1998). *The future of air traffic control:
Human operators and automation.* C. D. Wickens, A. S. Mavor, R.
Parasuraman, & J. P. McGee (Eds.). Washington, DC: National Academy
Press.

Norico, A. F., & Stanley, J. (1989). Adaptive human-computer interfaces:
A literature survey and perspective. *IEEE Transactions on Systems, Man,
and Cybernetics, 19*(2), 399-408.

Nuclear Regulatory Commission. (1994). *Human factors engineering
guidance for the review of advanced alarm systems* (NUREG/CR-6105*).*
Washington, DC: United States Nuclear Regulatory Commission.

Nuclear Regulatory Commission. (1996). *Human-system interface design
review guideline* (NUREG-0700 Rev. 1 Vol. 1). Washington, DC: United
States Nuclear Regulatory Commission.

Parasuraman, R., Hancock, P. A., & Olofinboba, O. (1997). Alarm
effectiveness in driver-dash centered collision-warning systems.
*Ergonomics, 39*, 390-399.

Parasuraman, R., Hilburn, B., Molloy, R., & Singh, I. (1991)*. Adaptive
automation and human performance III. Effects of practice on the
benefits and costs of automation shifts* (Technical Report CSL-N91-2).
Washington, DC: The Catholic University of America, Cognitive Science
Laboratory.

Parasuraman, R., Molloy, R., Mouloua, M., & Hilburn, B. (1996).
Monitoring of automated systems. In R. Parasuraman and M. Mouloua
(Eds.), *Automation and human performance: Theory and applications* (pp.
91-115). Mahwah, NJ: Lawrence Erlbaum Associates.

Parasuraman, R., Molloy, R., & Singh, I. L. (1993*).* Performance
consequences of automation-induced "complacency." *The International
Journal of Aviation Psychology, 3*, 1-23.

Parasuraman, R., & Mouloua, M. (Eds.) (1996). *Automation and human
performance*: *Theory and applications*. New Jersey: Lawrence Erlbaum.
(Refereed, International).

Parasuraman, R., Mouloua, M., & Hilburn, B. (1998). A*daptive aiding and
adaptive task allocation enhance human-machine interaction.* In Scerbo,
M., and Mouloua, M. (Eds.). *Automation technology and human
performance: current research and future trends.* (Refereed,
International).

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse,
disuse, abuse*.* *Human Factors*, *39*(2), 230-253.

Parasuraman, R., Sheridan, T.B., & Wickens, C.D. (2000). A model for
types and levels of human interaction with automation. *IEEE
Transactions on Systems, Man, and Cybernetics, 30*, 286-297.

Riley, V. (1996). Operator reliance on automation*:* Theory and data. In
R. Parasuraman and M. Mouloua (Eds.), *Automation and human performance:
Theory and applications* (pp. 19-35). Mahwah, NJ: Lawrence Erlbaum
Associates, Inc.

Rogers, W. H., Schutte, P. C., & Latorella, K. A. (1996). Fault
management in aviation systems. In R. Parasuraman and M. Mouloua (Eds.),
*Automation and human performance: Theory and applications* (pp.
281-317). Mahwah, NJ: Lawrence Erlbaum Associates.

Rouse, W. B. (1988). Adaptive aiding for human/computer control. *Human
Factors, 30*(4), 431-443.

Rudisill, M. (1994). Flight crew experience with automation technologies
on commercial transport flight decks. In M. Mouloua, & R. Parasuraman
(Eds.), Human performance in automated systems: Current research and
trends. *Proceedings of the First Automation Technology and Human
Performance Conference* (pp. 203-211). Hillsdale, NJ: Lawrence-Erlbaum
Associates, Inc.

Rudisill, M. (1995). Line pilots' attitudes about and experience with
flight deck automation: Results of an international survey and proposed
guidelines. In R.S. Jensen & L.A. Rakovan (Eds.), *Proceedings of the
Eighth International Symposium on Aviation Psychology*, 288-293.

Sarter, N. B., & Woods, D. D. (1994). Pilot interaction with cockpit
automation II: An experimental study of pilots' model and awareness of
the flight management system*.* *The International Journal of Aviation
Psychology, 4*, 1-28.

Sarter, N. B., & Woods, D. D. (1995). How in the world did we ever get
into that mode? Mode error and awareness in supervisory control. *Human
Factors, 37*(1), 5-19.

Scerbo, M. W. (1996). Theoretical perspectives on adaptive automation.
In R. Parasuraman and M. Mouloua (Eds.), *Automation and human
performance: Theory and applications* (pp. 37-63). Mahwah, NJ: Lawrence
Erlbaum Associates.

Scerbo, M. W., & Mouloua, M. (1999). *Automation technology and human
performance: Current research and trends.* Mahwah, NJ: Lawrence Erlbaum
Associates.

Sen, P. (1984). Adaptive channels and human decision-making. *IEEE
Transactions on systems, man and cybernetics, 14*(1), 120-130.

Sheehan, J. (1995). *The tyranny of automation.* Professional Aviation
Briefing. Retrieved April 3, 2000 from the World Wide Web:
http://www.faa.gov/avr/NEWS/Previous/autom.htm.

Sheridan, T. B. (1970). On how often the supervisor should sample. *IEEE
Transactions on Systems Science and Cybernetics, SSC-6*, 140-145.

Sheridan, T. B. (1996). Speculations on future relations between humans
and automation*.* In *Automation and human performance: Theory and
applications*. R. Parasuraman and M. Mouloua, (Eds.). Mahwah, NJ:
Lawrence Erlbaum Associates.

[]{.mark}

Shneiderman, B. (1998). *Designing the user interface: Strategies for
effective human computer interaction* (3rd edition). Reading, MA:
Addison Wesley.

Smith, S. L., & Mosier, J. N. (1986). Guidelines for designing user
interface software (ESD-TR-86-278). Hanscom AFB, MA: Electronic Systems
Division.

Standard Terminal Automation Replacement System Human Factors Team.
(1997). *Standard terminal automation replacement system human factors
review supporting documents*. Unpublished manuscript.

Standard Terminal Automation Replacement System Human Factors Team.
(1998). Report of the *Computer-human interface re-evaluation of the
standard terminal automation replacement system monitor and control
workstation*. Unpublished manuscript.

Tyler, S. W., & Treu, S. (1989). An interface architecture to provide
adaptive task-specific context for the user. *International Journal of
Man-Machine Studies, 30,* 303-327.

Veridian (1998). *Aviation human computer interface style guide (Report
64201-97U/61223)* (AHCI). Dayton, OH: Author.

Vortac, O. U., Barile, A. L., Albright, C. A., Truitt, T. R., Manning,
C. A., & Bain, D. (1996). Automation of flight data in air traffic
control. In D. Herrmann, M. Johnson, C. McEvoy, C. Hertzog, & P. Hertel
(Eds.), *Basic and applied memory: Research on practical aspects of
memory, 2*. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Warm, J. S., Dember, W. N., & Hancock, P. A. (1996). *Vigilance and
workload in automated systems.* In R. Parasuraman and M. Mouloua (Eds.),
Automation and human performance: Theory and applications (pp. 183-200).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Wickens, C. D. (2000). *Imperfect and unreliable automation and its
implications for attention allocation, information access and
situational awareness.* (Technical report ARL-00-10/NASA-00-2).
Urbana-Champaign, IL: Aviation Research Lab Institute of Aviation.

Wickens, C. D., & Flach, J. M. (1988). Information processing*.* In E.
L. Wiener and D. C. Nagel (Eds.), *Human factors in aviation* (pp.
111-155). San Diego,CA: Academic Press.

Wickens, C. D., & Kessel, C. (1979). The effects of participatory mode
and task workload on the detection of dynamic system failure. *IEEE
Transactions on Systems, Man, and Cybernetics, 9*, 24-34.

Wiener, E. L. (1981). Complacency: Is the term useful for air safety?
*Proceedings of the 26^th^ Corporate Aviation Safety Seminar*. Denver,
CO: Flight Safety Foundation, Inc., 116-125

Wiener, E.L. (1988). Cockpit automation*.* In E. Wiener & D.C. Nagel
(Eds.), *Human Factors in Aviation*, San Diego, CA: Academic Press.

Wiener, E. L. (1989). *Human factors of advanced technology ("glass
cockpit") transport aircraft* (Technical Report 117528). Moffett Field,
CA: National Aeronautics and Space Administration, Ames Research Center.

Wiener, E.L. & Curry, R. E. (1980). Flight-deck automation promises and
problems. *Ergonomics, 23*, 995-1011.

Woods, D. D. (1996). Decomposing automation: Apparent simplicity, real
complexity*.* In R. Parasuraman and M. Mouloua (Eds.), *Automation and
human performance: Theory and applications* (pp. 3-17). Mahwah, NJ:
Lawrence Erlbaum Associates.

[]{#__RefHeading___Toc27187358 .anchor}Index

A

Acceptance 3-10

Adaptive automation 3-27

Automation

implementation 3-4

Availability 3-10

C

Control automation 3-34

D

Decision aids 3-29

Design 3-7

Direct manipulation 3-29

Dynamic information 3-10

E

Error resistance 3-5

Error tolerance 3-5

F

False alarms 3-19

Fault management 3-17

Feedback 3-8, 3-9, 3-11, 3-34

Function allocation 3-22

I

Information automation 3-24

Integration 3-26

Interface 3-9

direct manipulation 3-29

location status 3-9

simplicity 3-9

Interface consistency 3-9

L

Levels of automation 3-22

M

Minimum automation elements 3-2

Modes 3-11

Monitoring 3-13

S

Spatial representation 3-9

System response 3-8

T

Training 3-20

Trust 3-10

U

User expertise 3-3

User in command 3-2

User involvement 3-7

V

Validation 3-8

W

Workload levels 3-4