A Meta-Analysis of Factors Influencing the Development of 

Trust in Automation: Implications for Human-Robot 

Interaction

by Kristin E. Schaefer, Deborah R. Billings, James L. Szalma,

Jeffrey K. Adams, Tracy L. Sanders, Jessie Y. C. Chen,

and Peter A. Hancock

ARL-TR-6984 July 2014

Approved for public release, distribution is unlimited.
NOTICES

Disclaimers

The findings in this report are not to be construed as an official Department of the Army position unless 

so designated by other authorized documents.

Citation of manufacturer’s or trade names does not constitute an official endorsement or approval of the 

use thereof.

Destroy this report when it is no longer needed. Do not return it to the originator.
Army Research Laboratory

Aberdeen Proving Ground, MD 21005-5425

ARL-TR-6984 July 2014

A Meta-Analysis of Factors Influencing the Development of 

Trust in Automation: Implications for Human-Robot 

Interaction

Kristin E. Schaefer

University of Central Florida, Institute for Simulation and Training

Oak Ridge Associated Universities Postdoctoral Fellow, ARL

Deborah R. Billings

University of Central Florida, Institute for Simulation and Training

James L. Szalma, Jeffrey K. Adams, and Tracy L. Sanders 

University of Central Florida, Department of Psychology

Jessie Y. C. Chen

Human Research and Engineering Directorate, ARL

Peter A. Hancock

University of Central Florida, Department of Psychology

University of Central Florida, Institute for Simulation and Training

Approved for public release, distribution is unlimited.
ii

REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering 

and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information,

including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson 

Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to 

comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

July 2014

2. REPORT TYPE

Final

3. DATES COVERED (From - To)

September 2012–February 2014

4. TITLE AND SUBTITLE

A Meta-Analysis of Factors Influencing the Development of Trust in Automation: 

Implications for Human-Robot Interaction

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

Kristin E. Schaefer, Deborah R. Billings, James L. Szalma,Jeffrey K. Adams,

Tracy L. Sanders, Jessie Y. C. Chen, and Peter A. Hancock

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

U.S. Army Research Laboratory

ATTN: RDRL-HRS-E

Aberdeen Proving Ground, MD 21005-5425

8. PERFORMING ORGANIZATION

 REPORT NUMBER

ARL-TR-6984

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release, distribution is unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT

Trust has become increasingly important because of the increasing need for synergistic human-machine teaming. Here, we expand on our previous 

meta-analytic work in the field of human-robot interaction (HRI) to include all of human-automation interaction. We used meta-analytic procedures to 

assess trust. A total of 343 articles were reviewed, of which 30 studies, providing 164 effect sizes, and 16 studies, providing 63 correlational effect 

sizes, met the selection criteria for analysis. The overall experimental effect on trust was g�= +0.48, and the correlational effect was r̅= +0.34, each of 

which represented a medium to strong effect. Moderator effects were examined for the human-related (g�= +0.49; r̅= +0.16) and automation-related 

(g̅= +0.53; r̅= +0.41) factors. However, moderator effects specific to the environmental factors were not calculated because of a lack of presently 

qualified studies. Submoderating factors were examined and reported for human-related (i.e., age, cognitive factors, emotive factors) and automationrelated (i.e., features and capabilities) effects. Analyses were also conducted for type of automated aid: cognitive, control, and perceptual automation 

aids. Automated cognitive aids provide recommendations to users about the current and potential future states of systems. Automated control aids 

replace varying levels of human (operator, user) action. Perceptual aids are used to assist the operator or user by providing warnings or to assist with 

pattern recognition. All three types of aids—cognitive (g̅= +0.41; r̅= +0.39), control (g̅= +0.51; r̅= +0.12), and perceptual (g̅= +0.62; r̅= +0.37)—

had a moderate effect on trust development. These findings provide a quantitative picture of human-, automation-, and environment-related factors 

influencing trust in automation. Findings from this work were used to develop design and training guidelines that are also applicable to HRI. Future 

research needs were identified based on the results of the foregoing analyses.

15. SUBJECT TERMS

automation, human-machine interaction, meta-analysis, robot, trust

16. SECURITY CLASSIFICATION OF: 

17. LIMITATION

OF ABSTRACT

UU

18. NUMBER

OF PAGES

70

19a. NAME OF RESPONSIBLE PERSON

Kristin E. Schaefer

a. REPORT

Unclassified

b. ABSTRACT

Unclassified

c. THIS PAGE

Unclassified

19b. TELEPHONE NUMBER (Include area code)

410-278-5972

Standard Form 298 (Rev. 8/98)

Prescribed by ANSI Std. Z39.18
iii

Contents

List of Figures v

List of Tables v

Acknowledgments vi

1. Introduction 1

1.1 Foundation.......................................................................................................................2

1.2 Definitions.......................................................................................................................3

1.3 Report Organization ........................................................................................................4

2. Task-Specific Trust in Automation 5

2.1 Trust in Cognitive Aids...................................................................................................5

2.2 Trust in Control Aids.......................................................................................................7

2.3 Trust in Perceptual Aids..................................................................................................8

3. Development of a Trust-in-Automation Model 8

3.1 Theoretical Model of Human-Automation Trust ............................................................9

3.2 Factors Associated With the Human.............................................................................10

3.2.1 Operator Traits and States.................................................................................10

3.2.2 Cognitive Factors ..............................................................................................11

3.2.3 Emotive Factors.................................................................................................13

3.3 Factors Associated With the Automation......................................................................14

3.3.1 Feature-Based Characteristics...........................................................................14

3.3.2 Automation Capability ......................................................................................16

3.4 Factors Associated With the Environment....................................................................19

4. Analytical Method 19

4.1 Sample of Studies..........................................................................................................19

4.2 Criteria for Study Inclusion...........................................................................................19

4.3 Coding of Studies..........................................................................................................20

4.4 Effect Size .....................................................................................................................21

4.5 Variance Estimates........................................................................................................21
iv

5. Results 21

5.1 Preference for Human or Automation Assistance.........................................................21

5.2 Task-related Automation...............................................................................................22

5.3 Human-Related Factors.................................................................................................23

5.4 Automation-Related Factors..........................................................................................24

6. Discussion 25

6.1 Training Suggestions.....................................................................................................26

6.2 Design Recommendations.............................................................................................26

7. Conclusions 28

8. References 29

Appendix A. Definitions 43

Appendix B. Comprehensive Automation Trust Definition Set 45

Appendix C. References of Experimental and Correlational Articles 51

Appendix D. Organization Results 57

Distribution List 61
v

List of Figures

Figure 1. Human-Robot trust (Hancock et al., 2011). .....................................................................2

Figure 2. Antecedents of trust from definitions of trust in automation. ..........................................9

Figure 3. Theoretical model of human-automation trust. ..............................................................10

Figure 4. Revised human-robot descriptive trust model................................................................25

Figure 5. Levels of automation (Parasuraman et al., 2000)...........................................................27

List of Tables

Table 1. Task-dependent antecedents of trust in cognitive automation...........................................6

Table 2. Task-dependent antecedents of trust in control automation. .............................................7

Table 3. Meta-analytic results for perception of trust in a human or automation for a task..........22

Table 4. Meta-analytic results for trust in task-based automation.................................................23

Table 5. Meta-analytic results for human-related factors on trust in automation..........................24

Table 6. Meta-analytic results for automation-related factors on trust in automation...................24
vi

Acknowledgments

Research was sponsored by the U.S. Army Research Laboratory (ARL) and was accomplished 

under Cooperative Agreement Number W911NF-10-2-0016. The views and conclusions 

contained in this document are those of the authors and should not be interpreted as representing 

the official policies, either expressed or implied, of ARL or the U.S. Government. The U.S. 

Government is authorized to reproduce and distribute reprints for Government purposes 

notwithstanding any copyright notation herein. The authors would like to thank our reviewers 

Dr. Raja Parasuraman, Dr. Ewart de Visser, and Andrea Krausman for their detailed feedback 

and recommendations.
1

1. Introduction

The purpose of this report is to review the current evidence relating to trust in human-automation 

interaction and to compare these findings with our prior research on the more specific area of 

trust in human-robot interaction (HRI) (Hancock et al., 2011). Since automation has been shown 

to be a key dimension of a robot (including unmanned systems), identifying the factors that can 

impact trust between humans and automation may significantly increase our understanding of 

trust and its applications to HRI.

Previous meta-analytic findings on human-robot trust helped to formulate and formalize a triadic 

model of trust, which categorized the factors influencing trust development as human-related,

robot-related, or environment-related characteristics (figure 1) (Hancock et al., 2011). Hancock 

and his colleagues (2011) established the great importance of robot performance consistency 

(e.g., reliability) and allied attributes in the development of trust in HRI. Environmental 

characteristics (e.g., team collaboration, task type) were found to influence trust in HRI but only 

to a lesser degree than the reliability factor. Further, this work revealed that human-related 

factors did not appear to play a large role in the development of trust; however, limited empirical 

research was available regarding human-related dimensions and the influences on trust in robots

at the present time. The resulting model of HRI trust must thus be interpreted with caution, as the 

descriptive model may change as the dearth of current experimentation was addressed.

Additionally, the research used to construct the trust model was limited only to empirical studies 

documenting human interaction with embodied robots, which is a specific subfield of 

automation. Thus, our prior meta-analysis did not include an assessment of all other types of 

automation (e.g., decision support systems). Consequently, an examination of the trust in the 

larger body of automation literature may increase our understanding of human-robot trust. The 

following represents this effort using a quantified meta-analytic technique.
2

Figure 1. Human-Robot trust (Hancock et al., 2011).

1.1 Foundation

Technical developments have made it possible to automate tasks that have traditionally been 

carried out by humans. Automated systems can be used in environments that are high risk for 

humans (e.g., hazardous, life-threatening environments), in tasks that humans do not want to 

perform (e.g., boring and repetitious tasks), and in tasks where a machine is more accurate or 

reliable than a human (e.g., mathematical computations, robotics manufacturing, and autopilot

activity). Parasuraman and colleagues (2000) have provided a review of system functions that 

can and should be automated based on their cost of action consequences on human performance. 

As such, automation is now available in multiple domains, from the home (e.g., refrigerator) to 

the local environment (e.g., automated teller machine) to the battlefield (e.g., unmanned 

vehicles). For example, within robotics, unmanned aerial systems are designed to be used in 

situations where machines provided better sustained performance than a human, when there are 

differing political and human costs in high-risk environments, and when there is an increased 

likelihood for mission success (Office of the Secretary of Defense, 2005). Further, in the final 

Task Force Report on the role of autonomy in Department of Defense systems, the Defense 

Science Board (DBS, 2012) suggested that unmanned systems are making a significant impact 

on warfare worldwide. This report stated that the value of unmanned systems is not the 

replacement of a human but human-system collaboration (i.e., the capability to extend and 

complement human capability without degradation because of fatigue or lack of attention);

however, the major challenge is the lack of trust that the system will perform as intended. 
3

In addition, automation literature has explored the impacts of trust on performance outcomes and 

trust calibration strategies in greater depth than traditional human-robot trust literature. For 

example, trust in automation is directly tied to the outcome measurements of human-automation 

interaction, specifically automation use (see Beck et al., 2007; Moray et al., 2000; Muir and

Moray, 1996; Parasuraman et al., 1993). These include human reliance on automation and 

human compliance with automation (see also Barnett, 2007; Chen and Terrence, 2009; Dzindolet

et al., 2003; Lee and Moray, 1992; Lee and Moray, 1994; Lee and See, 2004). In addition, trust 

development is related to performance outcomes that lead to safety decisions (see Maltz and

Meyer, 2001; Scallen and Hancock, 2001; Schobel, 2009; Schobel and Manzey, 2011; Seppelt 

and Lee, 2007; Yamada and Kuchar, 2006). Appropriate trust calibration is critical for 

appropriate reliance on automation for all types of automated aids. According to Lee and Moray 

(1994), trust calibration pertains to the trust level of the human user and its correspondence to the 

capabilities of the automated aid. As such, too much trust may lead to overreliance,

complacency, and mistrust, while too little trust can lead to disuse. Properly calibrated trust 

creates a situation where the human user is engaged with the automation and participates at the 

appropriate level in the process (Muir, 1994). Calibration of trust is also an important issue for 

computer users in the e-commerce domain. Bonsall and Joint (1991) suggested that users have 

difficulties placing the “right level” of trust in computer advice systems. Computer users also 

have difficulties heeding advice from a computer aid during the decision-making process 

(Hoffman et al., 1999; Ratnasingham, 1998).

1.2 Definitions

For the purpose of this work, it is important to understand the relationship between the terms 

autonomy, automation, and robot. Literature and human perception have often blurred the 

definitional lines between these terms. For example, automation and autonomy, as well as 

automation and robot, are often used interchangeably. Here, autonomy is defined as “a capability 

(or set of capabilities) that enables a particular action of a system to be automatic or, within 

programmed boundaries, ‘self-governing’” (DSB, 2012). Automation, in the broadest sense of 

the word, is a “machine execution of functions” (Parasuraman et al., 2000, p. 286). With such a 

broad definition of automation, it is no wonder there is confusion or overlap among the 

definitions of autonomy, automation, and robot. In addition, a large portion of automation-based 

definitions and theoretical contributions build on the work by Victor Riley (1989) that discussed 

automation states in terms of combinations of values along the dimensions of machine 

intelligence (i.e., degree of data processing) and autonomy (i.e., degree of authority to act on the 

“World”). For example, Parasuraman and Riley (1997) incorporated the human-machine 

comparison in which automation was defined as “the execution by a machine agent (usually a 

computer) of a function that was previously carried out by a human” (p. 231). In a more detailed 

view of the functions of the technology, Lee and See (2004) define automation as the

“technology that actively selects data, transforms information, makes decisions, or controls 

processes” (p. 50).
4

A robot is often defined and discussed in terms of its functional and autonomous capabilities. 

This is illustrated by Mahoney’s (1997) definition of a robot as “programmable automation to 

augment human manipulation” (p. 3). More recently, Yagoda (2011) defined a robot as the 

interaction of intelligence and autonomy, thus building on Riley’s (1989) automation research 

and further linking the two definitions. However, additional work has also shown the importance 

of physical form or embodiment to the classification of a machine as a robot. For example,

Kurfess (2005) suggested that a robot is the manifestation of human characteristics, such as 

complex actions and anthropomorphic features. Schaefer (2013) also provided a more complete 

review of the variations between robot definitions as well as additional experimentation on a 

human’s perception of the classification of a machine as a robot. 

In addition, Scholtz (2003) has suggested that the type of interaction, the number of systems a 

user may interact with, the environment, and the physical nature of the robot (e.g., complex,

dynamic control systems, exhibition of autonomy and cognition) help differentiate HRI from 

more general human-computer or human-machine interaction. Steinfield et al. (2006) suggested 

that HRI can and should be discussed in terms of five types of tasks: navigation, perception (i.e.,

perceive and understand the remote environment for application), management (i.e., coordination 

of the actions of humans and robots), manipulation (i.e., interaction with the environment), and 

social tasks (i.e., perform tasks that require “social interaction”). Therefore, by taking into 

account the task-based nature of HRI, it may be more accurate to classify a robot as having 

varying or multiple autonomous capabilities within a single embodiment or system. 

1.3 Report Organization

Section 2 identifies three types of task-specific automation (i.e., cognitive, control, and 

perceptual aids) and the importance to trust development within each type of automation aid. 

Section 3 outlines a descriptive model of human-automation trust and provides literary support 

for the model structure in terms of the human-related, automation-related, and environmentalrelated trust factors. Section 4 outlines the analytical methodology for the meta-analytic analysis 

provided in section 5. Section 6 reconnects the human-automation trust results with the potential 

links within HRI. Training suggestions, as well as design recommendations, are provided. This 

report is summated with a conclusion (section 7) addressing the importance of this work to the 

military domain. 
5

2. Task-Specific Trust in Automation

Automation literature provides a number of insights into types of functions that may be 

automated across a variety of tasks. More importantly, task- and function-specific aspects of 

automation play an important role in the development of trust in these systems. Task-related 

automation is hereby broken down into three major areas of automation aids: cognitive, control,

and perceptual aids. This accords with the framework discussed in Parasuraman et al. (2000) that 

identified types of automation as acquisition, analysis, decision, and action aids while taking into 

account that the task-related nature of automation elements is related to HRI.

2.1 Trust in Cognitive Aids

Automated cognitive aids provide recommendations to users about the current and potential 

future states of systems. These automated decision aids or expert systems can make accurate 

identifications or detections during a screening or inspection, monitoring, or route planning 

tasks. This type of aid has been used in many areas, including e-commerce, medication 

management domains, and many others (table 1).

In addition to these factors, Corritore et al. (2003) suggest that computer interface design criteria 

(e.g., ease of use, good visual design elements, information content, and reputation) can provide 

cues to the degree to which an aid will be perceived as trustworthy. One concern about cognitive

aids is that operators may trust and rely excessively on the automated aid. This can lead to 

reduced monitoring and, in turn, increase the number of errors (e.g., decision-making task for 

pilots, Sarter and Schroeder, 2001). Similar findings were also shown by Bagheri and Jamieson 

(2004) during a monitoring task: increased trust in the automation led to lower operator error 

detection.
6

Table 1. Task-dependent antecedents of trust in cognitive automation.

Task Antecedent 

of Trust Finding

Target Detection 

(Chen and Terrence, 2009;

Wang et al., 2009)

Reliability of 

automation

A more reliable aid led to more trust; however, this effect can be 

moderated by individual differences factors, such as operator 

attentional control.

Inspection Task 

(Madhavan and

Wiegmann, 2007)

Expertise of 

Automation Users trust expert automation more than novice automation.

Human vs.

automation

Users have more trust in a novice automated aid than a novice 

human, but more trust in an expert human than an expert 

automated aid.

Task 

difficulty

Users trust an automated aid more when it is difficult to identify 

errors in the task.

Route Planning 

(Chen and Barnes, 2012;

Cummings et al., 2010;

De Vries and Midden,

2008)

Error rate of 

automation

When automation error rate is high, operators trust themselves 

more than the automation; conversely, when error rate is low, they 

trust automation more. However, these effects can be moderated 

by individual differences factors, such as operator attentional 

control.

Risk Participants who chose less risky paths trusted the automation 

more.

Reputation Operators had slightly higher trust when the automation had a

positive reputation.

e-commerce 

(Aberg and Shahmehri,

2000)

Human vs.

automation Users preferred human assistance to automation.

Other Decision Aids

(Ezer et al., 2008;

Ho et al., 2005)

Reliability of 

automation A more reliable aid led to more trust in the aid.

Age of user

Older adults trusted automation more than younger adults;

however, younger adults are more flexible than older adults in 

changing their reliance behavior based on the cost of verification 

(of the automation).
7

2.2 Trust in Control Aids

Automated control aids (e.g., autopilots, adaptive cruise control, and navigation aids) replace 

varying levels of human (operator or user) action. Operators tend to prefer automated control 

aids to traditional systems or procedures (Ardern-Jones et al., 2009). Trust is generally high for 

control aids. Initial trust tends to be higher for the automated aid (e.g., function allocation tasks,

Ma and Kaber, 2007) and builds over time (Rajaonah, 2006). For a summary of factors 

impacting trust development of functional allocation tasks and vehicle control, see table 2.

Table 2. Task-dependent antecedents of trust in control automation.

Task Antecedent of Trust Finding

Function Allocation

(Moray et al., 2000; Stedmon 

et al., 2007)

Reliability Higher reliability led to higher trust.

Communication

mode

Operators had higher trust for automation that used 

human speech rather than synthetic speech.

Communication

accuracy

Operators had higher trust for accurate information 

than inaccurate information.

Vehicle Control: ACC, UAV

(Donmez et al., 2006; Hughes

et al., 2009; Spain and Bliss,

2008)

Age of user Older drivers trusted adaptive cruise control (ACC) 

more than middle age drivers.

Automation accuracy Drivers trusted more accurate systems.

Alarms Drivers trusted visual alarms more than auditory 

alarms for vehicle control.

Perceived usefulness 

of automation

As the level of usefulness of automation increased, so 

did trust.

Human vs.

automation Users trusted pilot more than autopilot.

Communication Higher reliability of information led to higher trust.

Workload Trust was higher when workload was low.

Trust also has a direct impact on outcome measures. For example, Manzey and his colleagues 

(2009) found that most surgeons trust in the proper functioning of the navigational control tasks 

in a surgery system and would choose to be operated on by the system. More operator trust 

during function allocation tasks leads to more use, better performance, and less operator time 

manually adjusting the automation (Muir and Moray, 1996). In addition, higher reliability of 

adaptive cruise control (ACC) systems leads to more driver trust and increased compliance with 

the automation (Spain and Bliss, 2008).
8

2.3 Trust in Perceptual Aids

Perceptual aids are used to assist the operator or user by providing warnings (e.g., fault detection,

automobile warning systems, and function allocation for signaling or communication) or to assist 

with pattern recognition (e.g., speech or target detection). For example, automobile warning 

systems are used for distraction mitigation, collision avoidance, and providing alarms (e.g.,

speed alerts). Gupta and colleagues (2002) assessed the effects of alarm type and alarm 

sensitivity (i.e., number of alarms) on drivers’ trust of perception aids. They found no significant 

difference in type of alarm; however, drivers did show a higher trust for alarms with a low 

sensitivity rate. Similarly to the other two types of automation aids, users or operators of 

perception aids have more trust in people than in the automation. For example, Bisantz and 

Seong (2001) found that drivers using automobile warning systems had higher negative feelings 

of trust toward the automation when they were told that the failures were attributed to the 

hardware or software instead of people. 

Trust in the warnings or alarms also has a direct effect on the outcome variables. One potential 

area for concern is the operator’s overreliance on and over compliance with a warning system. 

Bitan and Meyer (2007) found that the impacts of the sensitivity rate and the false alarm rate on 

trust also have a direct impact on the user’s reliance and compliance (respectively) on the 

warning system to the degree in which the operator will only perform an action when an alarm is 

present and fail to respond when the alarm is not present. 

3. Development of a Trust-in-Automation Model

While trust is important to specific automated tasks and outcomes (especially use and 

performance), the factors that make up these relationships are of critical importance to the 

development or degradation of trust in automation. However, despite the number of trust

definitions available, there is a lack of consensus among experts. Billings and colleagues (2012) 

reviewed 282 definitions of trust (i.e., 200 interpersonal trust definitions, 32 human-robot trust

definitions, and 50 trust-in-automation definitions). These automation trust definitions highlight

a number of antecedents that are important to trust development (see figure 2) and support the 

need for a meta-analytic approach to understanding trust in automation. 
9

Figure 2. Antecedents of trust from definitions of trust in automation.

Trust definitions within the automation domain range from a broad definition, “a human’s 

willingness to accept direction from an automated system” (Fan et al., 1998) to the specific, “the 

extent to which a user is confident in, and willing to act on the basis of recommendations,

actions, and decisions of an artificially intelligent agent” (Madsen and Gregor, 2000, p. 1). Refer 

to appendix A for a list of definitions associated with human-automation interaction, and 

appendix B for a complete list of trust-in-automation definitions. This diversity within 

definitions is seen in the other domains of trust as well, supporting the idea that there is no 

universal definition of trust. However, one of the most referenced definitions of trust in 

automation is “the attitude that an agent will help achieve an individual’s goals in a situation 

characterized by uncertainty and vulnerability” (Lee and See, 2004, p. 51). 

3.1 Theoretical Model of Human-Automation Trust

To identify potential antecedents of trust development in human-automation interaction, existing 

literature on trust in automation (both theoretical and empirical) was reviewed. The literature 

suggests that several factors play important roles in trust development. Some factors involve the 

human directly, while others focus on aspects of the automation. Other factors deal with various 

aspects of the operational environment. Therefore, the antecedents of trust are hereby divided by 

three main factors: the human, the automation, and the environment (see figure 3). The following 

section will analyze the empirical evidence of human-automation trust through meta-analytic 

procedures. The findings from this analysis will then be used to update the model of humanrobot trust. 
10

Human Operator Automation Environment

Human Traits Features Task-Related

+* Age * Degrees of automation +* Proximity

+Gender +* Appearance +Risk

+Ethnicity * Mode of communication

+Personality

Human States Capability

Fatigue +* Errors in automation

Stress +* Automation behavior(s)

Attentional control Quality/Accuracy * Cueing/Feedback/Alarms

Cognitive Factors

+* Understanding the automation

+* Ability to use automation

+* Expectancy of automation

Emotive Factors

+Attitudes toward automation

Confidence in automation

+Satisfaction with automation

+* Comfort with automation

Note: * represents experimental and + represents correlation findings.

Figure 3. Theoretical model of human-automation trust.

3.2 Factors Associated With the Human

As seen within human-technology interaction, the human component is often suggested to be of 

utmost importance; however, the factors associated with the human are often overlooked in 

empirical research (as seen by the limited number of cited work) or become a secondary focus 

behind the functional capabilities of the system. Here we discuss trust as a human-centric 

construct in that the human is the one trusting the technology. Therefore, the human user or 

operator must be considered when examining trust. Within the area of trust in automation, the

factors associated with the human can be addressed in terms of human traits and states as well as 

the cognitive and emotive factors. 

3.2.1 Operator Traits and States

There is evidence that stable human traits may be related to how an individual responds to 

automation, but it is likely that their relationships vary as a function of the characteristics of the 

automated task (Szalma and Taylor, 2011). For example, many demographic characteristics of 

the user or operator have been suggested to influence the human-automation trust relationship,

including age, gender, ethnicity, and personality. For example, age can impact the types of

systems that will be trusted (e.g., driver warning systems, Kircher and Thorslund, 2009) and 

influence the likelihood that the automation will be used (e.g., decision aids for medication 
11

management, Ho et al., 2005). Ho and colleagues also found that older adults were more likely to 

trust the automation than younger adults. However, upon further exploration, gender differences 

do not seem to influence trust development (Merritt and Ilgen, 2008; Stedmon et al., 2007). 

Some work suggests that ethnicity and personality traits are important to trust development 

(Merritt and Ilgen, 2008). However, they have not been thoroughly explored in the automation 

trust literature. These antecedents of trust are more commonly referenced in human-interpersonal 

trust literature and are therefore included in figure 3. 

Dynamic human states, such as stress and fatigue, during human-automation interaction are wellresearched areas (see Finomore et al., 2009; Neubauer et al., 2012; Reichenbach et al., 2011;

Warm et al., 2008). For the purpose of this work, stress is the “force that degrades performance 

capability” (Hancock and Warm, 1989). Fatigue is a “reduction in the capacity and desire to 

react . . . characterized by tiredness and an aversion to the continuation of goal-directed work” 

(Hancock et al., 2012). In addition, several studies also showed that operators with lower 

attentional control interacted differently with automated systems than those with higher 

attentional control (Chen and Terrence, 2009; Chen and Barnes, 2012; Thropp, 2006). 

Attentional control is defined as “one’s ability to focus and shift attention in a flexible manner”

(Chen and Barnes, 2012; Derryberry and Reed, 2002). Participants with lower attentional control 

tended to rely more heavily on automation, even when the reliability of those systems was low.

This research indirectly links human states to trust development; however, additional empirical 

study is needed in these areas. 

3.2.2 Cognitive Factors

The operator’s cognitive factors are hereby divided into three areas of research: (1) 

understanding how the automation works, (2) ability to use the automation, and (3) expectancy 

of the automation. While these are most often researched in the interpersonal trust domain,

Ashleigh and Stanton (2001) suggest that these areas of research are applicable to the technology 

domain as well. In addition, these divisions are supported by previous work in the field of trust in 

automation (see Lee and See, 2004; Madhavan and Wiegmann, 2007).

3.2.2.1 Understanding How the Automation Works

Understanding how to operate the automation is based in part on the ease of learning the system

and an individual’s previous experiences. Investigations into ease of learning have yielded mixed 

results. While one study (Wang et al., 2011) indicates that trust is negatively correlated with ease 

of learning, others have shown that a novice operator’s trust grows along a typical learning curve 

with training (Muir and Moray 1996), and it is critical to learning (Swadesh and Kasonf, 1995). 

Past experiences with technology, specifically automated systems, have a direct impact on trust 

development as well (Merritt and Ilgin, 2008). For example, Rajaonah (2006) suggests that trust 

is something that builds over time and use with a system. Work by Sullivan and colleagues 

(2008) supports the idea that trust builds over time and suggests that prior experiences directly 

influence how an operator will engage the automation. In the computer domain, research has 
12

shown that with sustained exposure to and use of information technologies, computer users 

develop trust in the accuracy of the data stored within a computer system despite the fact that there 

is upwards of 10% inaccuracies due to error (Klein et al., 1997; Parasuraman, 1987; Wiener 1985).

3.2.2.2 Ability to Use the Automation

Another area of the operator’s cognitive factors that affects the human-automation trust 

relationship is the operator’s perceived ability to use automation. This includes self-efficacy,

workload, and expertise as antecedents of trust. Self-efficacy is often defined as an individual’s 

perceived abilities to successfully carry out an identifiable task (Bandura, 1982; 1997). In the 

field of automation, self-efficacy affects not only whether or not a human will use automation,

but also how the human will perceive ease of use and usefulness (Igbaria and Iivari 1995). 

One purpose of automation is to reduce workload, and this has been shown to be successful in 

both mental workload (Wiegmann et al., 2010) and in the implementation of actions (Yang et al.,

2009). Increases in mental workload associated with the automated systems have been shown to 

lead to degradations of trust in automation (e.g., combat identification tasks, Wang et al., 2011). 

In their work on adaptive aiding of human-robot teaming, de Visser and Parasuraman (2011) 

found that human-robot teams benefited from imperfect static automation in high-task-load 

conditions, and adaptive automation helped alleviate the workload of the operator. 

Workload has also been directly tied to expertise and experience with automation. For example,

in a surgical task using image-guided navigation, inexperienced users reported increased time 

pressure and mental demands when using the systems (Manzey et al., 2009). Expertise may also 

be inhibitory, especially in highly reliable systems where a higher experience level may impair 

an operator’s ability to monitor for unanticipated system states (Bailey et al., 2006). As such,

increased expertise with a system can enhance trust development (Rajaonah, 2006).

3.2.2.3 Expectancy of the Automation

Prior to interacting with automation, human operators develop mental models or expectations of 

how the automation should behave. Therefore, this subcomponent of cognitive factors includes 

the following antecedents of trust: perceived commitment, usefulness, expectations, and benefits

of automation. Keil et al. (1995) found that the perceived usefulness (e.g., number of anticipated 

errors) of an automated system is often dictated by a function of the task or the fit of tool. The 

perceived benefit of the automation can influence whether or not the automation will be trusted 

(similar to software systems, Lee et al., 2007). If those expectations are high, the human operator 

may become overreliant on the automation (Mayer et al., 2006), which can lead to mistrust and 

misuse of the system. The system’s reputation may impact operator expectations. For example,

in an experiment involving a planning aid, De Vries and Midden (2008) found trust to be slightly 

higher when the aid was known to have a positive reputation. Findings also suggest that the 

number of people (i.e., majority or minority) supporting the reputation did not appear to 

influence trust development.
13

3.2.3 Emotive Factors

Emotive factors can be divided into four areas of research: attitudes toward automation,

confidence in automation, satisfaction, and comfort with automation. Similar to the previously

listed cognitive factors, emotive factors are most often researched in the interpersonal trust 

domain.

3.2.3.1 Attitudes Toward Automation

Attitudes toward the automation can impact overall system awareness states. Bailey and 

colleagues (2006) found that operators with positive attitudes toward automation may have too 

much trust in the system and focus less on the overall system state, which can lead to degraded 

performance. Ardern-Jones and colleagues (2009) found that errors in the system or difficulty 

accessing needed information within the system can lead to negative attitudes toward 

automation. Gao et al. (2006) found that operators with negative attitudes toward the automation 

(such as after a failure) were more likely to disuse the automation. Additionally, moods and 

emotions impact trust and liking of a system (e.g., happiness, Merritt et al., 2012) as well as 

reliance on automated systems.

3.2.3.2 Confidence in Automation

Confidence is often used as a synonym for trust. For example, in their evaluation of a navigation 

system (NaviBase) for Ears, Nose, and Throat surgery, Strauss and colleagues (2006) evaluated 

trust in the system by measuring user confidence. Other researchers suggest that confidence is 

one of the antecedents leading to trust development. Operators need to be confident in the 

functionality of the automation to successfully interact and use said automation. Gao et al. (2006) 

found that an operator will switch to manual control when confidence-based trust in automation 

decreases. Operators are also more likely to remain in manual control in the future, even when it 

is clear that the automation is more efficient than human performance. Therefore, we are 

including confidence as an antecedent of trust. 

There is also debate regarding whether self-confidence should be included in a humanautomation trust model. Many researchers suggest that there is an interaction between trust and 

self-confidence as it pertains to automation use (see Chen and Terrence, 2009; De Vries et al.,

2003; Lee and Moray, 1994; Neyedli et al., 2009; Wang et al., 2011). However, authors disagree 

on the interpretation of this interaction. Lee and Moray (1994) suggest that it is the relationship 

between trust and self-confidence that leads to automation use (i.e., trust exceeds selfconfidence) or manual override control (i.e., self-confidence overrides trust). Rani et al. (2000) 

suggest that operators who have both high trust and high self-confidence tend to prefer a higher 

level of automation. Since there is a lack of agreement regarding the relationship between trust 

and self-confidence, self-confidence will only be discussed here and will not be included in the 

meta-analysis results that follow.
14

3.2.3.3 Satisfaction With Automation

As is the case in many other arenas, satisfaction influences human perception of automation. 

This satisfaction is fostered by the quality of information and service that the automation 

provides (Lee et al., 2007). Generally, as satisfaction increases, so does trust (Domnez et al.,

2006). In one example, Wang and colleagues (2011) found trust to be positively correlated with 

satisfaction in a combat identification task. 

3.2.3.4 Comfort With Automation

Whether or not users are comfortable with an automated aid may influence their perception of 

the aid and, ultimately, whether or not they trust the aid. The level of comfort an individual has 

with automation may be dependent on familiarity and proximity (Nakada, 1997; van den Broek 

and Westerink, 2009), similarity of intent (Verberne et al., 2012), or even the degree of control 

the automated system has over levels of control, tactics, or strategy of the task (Ward, 2000). 

The majority of automated systems function within close proximity to their operators. Interaction 

typically occurs through a computer-based interface. To improve vehicle user interface design 

for optimum safety, Takayama and Nass (2008) examined the role of computer proximity (in-car 

versus wireless) on driving behavior. They suggest that in-car automated systems may be more 

effective for local information (e.g., climate, car maintenance), while distant computers 

connected by a wireless connection may be better for global information (e.g., navigational 

instructions). Additionally, systems showed that drivers felt more engaged with the in-car 

system, potentially leading to safer driving behavior.

3.3 Factors Associated With the Automation 

The primary trust development factors associated with the automation itself include features and 

capability of the automation. Here we discuss the features of the automation in terms of the 

degrees of automation (e.g., level of automation), appearance, and mode of communication. The 

capability of the automation is discussed in terms of errors, behavior, quality/accuracy, and 

communication through cueing and feedback.

3.3.1 Feature-Based Characteristics

Operators tend to have a fundamental bias toward trust in one’s own abilities or other human’s 

abilities rather than technology or automated systems (see Dzindolet et al., 2003; Gao et al.,

2006; Hughes et al., 2009). However, research by Ma and Kaber (2007) suggest that initial trust

can be higher for the automated control aid than a human performing the same function. In 

addition, Madhavan and Wiegmann (2007) found that trust is higher with novice automation 

(e.g., provides general assistance) than a novice human, while trust is higher with an expert 

human than expert automation (e.g., provides specific or detailed assistance). The following 

features-based antecedents of trust can assist in understanding trust development in automation. 
15

3.3.1.1 Degrees of Autonomy

This subcomponent of automation features includes levels of automation (LOA) and mode of 

automation (e.g., fixed, adjustable, and adaptive automation) as antecedents of trust. The LOA 

can range from high to low, with a high score indicative of an automated system where the 

computer or system is totally autonomous and the human has no control, and low score for a 

system where the human is totally in control and the computer offers no assistance (for a 

thorough review of the levels of automation, see Parasuraman et al., 2000). While Rani et al.

(2000) found no interaction between trust and the preferred LOA, more recent work suggests that 

LOA has a direct impact on trust development. For example, the combination of multiple types 

of automation to provide adaptive assistance is more trusted than individual lower LOAs (Cai 

and Lin, 2012; Cai et al., 2012). In addition, Merritt and colleagues (2012) suggest that when an 

individual implicitly perceives automation as “good,” the automated system is more trusted than 

if it is perceived as having “poor” or “ambiguous” automation. 

LOA also ties directly into research on the mode of automation (e.g., fixed, adjustable, and 

adaptive) through the degree to which the human interacts with the automation. Adjustable 

automation is a direct interaction with automation in which the user or operator has more direct 

control over the automation. Adaptive automation is an indirect interaction with the automation 

that takes into account the operator (e.g., emotional state, performance, and workload) and adapts 

the type and amount of automation accordingly (e.g., emotion-aware consumer products, van den 

Broek and Westerink, 2009). Human operators typically trust manual adjustable automation 

more than adaptive automation (e.g., function allocation tasks, Moray et al., 2000) because of the 

direct human interaction. However, Looije et al. (2010) suggested that when interacting with 

adaptive automation, humans prefer adaptive automation that can learn, recognize, and respond 

to personality differences in humans as well as exhibiting specific personality traits itself.

3.3.1.2 Appearance

If an automated aid or machine is perceived to be aesthetically compatible with its function, it is 

more likely to be used (Goetz et al., 2003). More specifically, Goetz and colleagues suggest that 

when the appearance of an automated aid matches the user’s expectations, the automated aid is

perceived more positively, and the user is more likely to appraise it positively. Further, general 

likeability has been shown to be influenced by appearance (Li and Yeh, 2010). Nass and Moon 

(2000) also suggest the importance of anthropomorphic appearance on automated systems. Pak 

and colleagues (2012) found that increasing the physical anthropomorphic appearance of an 

automated aid significantly impacted trust development through subjective assessment as well as 

objective/behavioral assessment.

3.3.1.3 Mode of Communication

Mode of communication refers to the human sensory system that the automation cues (e.g.,

visual, auditory, and tactile). Sarter (2006) reviewed guidelines focusing on multimodal 
16

information presentation and the research needed in the area, emphasizing the need to design the 

various parameters carefully to ensure their consistency with the users’ personality to maximize 

trust. For example, specific experimentation on auditory communication demonstrated that 

operators tend to trust systems with human speech more than synthetic speech and showed 

enhanced performance into the human speech condition (Stedmon et al., 2007). 

3.3.2 Automation Capability

The capability of automation is one of the most researched areas of automation trust literature. 

While capabilities are typically task specific, three major areas that can be discussed include: 

errors, behavior, and cueing or feedback. 

3.3.2.1 Errors

Errors are unavoidable in human-automation interaction; however, the goal of design is to reduce 

the number of errors. Errors are traditionally discussed in terms of both misses and false alarms 

(see Dixon and Wickens, 2006; Dixon et al., 2007); however, trust literature suggests that the 

reduction of all types of errors increases the reliability of the system and positively impacts trust 

development (De Vries et al., 2003). As such, the level of trust is mainly based on the user’s 

perception of the competence of the automated aid (Muir and Moray, 1996). Higher reliability 

has also been shown to lead to higher operator response frequency (i.e., an indirect measure of 

trust, Spain and Bliss, 2008) and higher reliance on the automation. 

In one study with conflicting findings, Uggirala and colleagues (2004) found that the number of 

errors presented by a system had no effect on trust, but the authors suggested that this was 

probably because the number of correct and incorrect responses was held constant for each 

treatment. Therefore, we typically perceive errors as negative influences on trust development in 

that when automation has too high of an error rate, operators will not trust the automation and 

disuse it. However, some errors may assist in trust calibration to reduce overreliance and overtrust in the system, which leads to reduced monitoring of the automation and negative 

performance.

The rate of errors is also important in the human-automation trust relationship. Yamada and 

Kuchar (2006) found that in drivers, miss rate was related to the degradation of trust in a system,

which led to slower response times. Conversely, it is the degree of difficulty of the task and not 

the type of error that appears to influence trust level. In a target identification task, Madhavan 

and Wiegmann (2007) found no significant difference between an easy miss group and an easy 

false alarm group; however, the difficult errors group trusted the automation more than the easy 

errors group. Similar findings by Master and colleagues (2005) showed significant change in 

trust between conservative and risky systems but no difference in trust based on error rates. This 

suggests that individuals are more willing to trust automation despite errors when the task is 

difficult.
17

In general, operators and users of automated systems have enhanced trust in the automation 

when information is accurate and reliable (Spain and Bliss, 2008; Stedmon et al., 2007). 

Degraded performance has also been shown when inaccurate information was presented (Sarter 

and Schroeder, 2001). When multiple automation aids are used simultaneously, the potential for 

conflicting information to be transmitted by different automated alerting systems can increase. In 

addition, Song and Kuchar (2003) found that this conflicting information can lead to distrust of 

one or all of the automated systems. Therefore, the accuracy of the communication becomes 

increasingly important to trust-related responses.

3.3.2.2 Behavior

Similar to human-interpersonal trust, the behavior of automation influences a user’s perceived 

trust in the automated aid. Importantly, trust in automation is established by consistent and 

reliable behavior on the part of the automation. Over time, this trust reduces the effort expended 

by the human user, because as trust is gained, the focus shifts away from observing specific 

behaviors toward assessing the global disposition of the system (McLeod et al., 2005). 

Therefore, this subcomponent of automation capability includes the following antecedents of 

trust: predictability, dependability, and specific types of behaviors. 

The quality of the automation includes automation expertise, etiquette, and transparency of 

communication and behavior. Research shows that users trust specialized (expert) automation 

more than they do general-purpose automated devices or automation they perceive to be novice 

(Koh and Sundar, 2010; Madhavan and Wiegmann, 2007). Etiquette of automation refers to a 

collection of behaviors that indicate that the system can be deemed trustworthy or displays 

behaviors that are satisfying in accordance to the customs of a culture (Parasuraman and Miller,

2004). Current literature suggests that etiquette is related to trust and can lead to a higher 

probability of intent errors (Dzindolet et al., 2006; Lee and Moray, 1992; Lee and See, 2004;

Lees and Lee, 2007; Merritt and Ilgen, 2008; Parasuraman and Miller, 2004). This research 

suggests that trust-related etiquette can increase the probability that an operator will disregard 

what they know about the utility of the automation when deciding to use or not use the 

automation. Simply put, enhancing automation with etiquette features increases trust in 

automation, despite previous knowledge or reputation.

Predictability and dependability of the automation are some of the most well-researched areas of 

trust in automation work. Predictability refers to how well the behaviors of the automation match 

the operator’s expectation of that behavior. Dependability refers to the consistency and 

effectiveness of a behavior. Systems with high reliability engender the trust of their operator 

across control, cognitive, and perceptual task types (Bagheri and Jamieson, 2004; Bailey et al.,

2006; Bliss and Acton, 2003; Cahour and Forzy, 2009; Cummings et al., 2010; Donmez et al.,

2006; Ho et al., 2005; Kazi et al., 2007; Moray et al., 2000). Similarly, predictability influences 

user trust. If experience with a machine provides predictable outcomes, then operators may start 

to trust the system (Cahour and Forzy, 2009; Muir, 1994). However, when operators experience 
18

unanticipated reactions from the automation (e.g., unpredictable outcomes), there is a rapid drop 

in trust that often leads to disuse or a disregard for future information provided by the aid 

(Wiegmann et al., 2010). Dependable systems, specifically the effectiveness of a warning 

system, are strongly related to the operator’s trust (Yamada and Kuchar, 2006). When examining 

the relationship of pilots and their automated aids, Tenney et al. (1998) found that pilots wanted 

the automation to be dependable and provide information that was simple and informative. These 

qualities were ranked more important than accountability (able to explain its actions),

subordination (easily countermanded), flexibility (range of modes and levels), adaptability 

(reprogrammable), error resistance (refuse erroneous inputs), or error tolerance (compensate for 

erroneous inputs) of automation behaviors. 

3.3.2.3 Cueing and Feedback

Many automated systems provide some level of cueing and/or feedback to the operator. The 

appropriateness of cues and feedback has been shown to have a direct connection to trust 

development. In a study examining the implications of installing ACC systems in vehicles, it was 

found that the development of the driver’s trust in the automated system may depend on the 

system’s ability to effectively communicate the status of the ACC to drivers to determine when 

intervention is necessary (Stanton et al., 2007). Bitan and Meyer (2007) suggest that the 

availability of information is important to trust development. For example, it was suggested that 

providing drivers with continuous information about the state of the automation may provide a 

necessary alternative to providing imminent crash warnings when it fails (Seppelt and Lee,

2007). Further, the amount of feedback sought from an automated system by the operators is 

directly related to the degree of trust they have that it will perform without failure (Muir and

Moray, 1996).

Different types of cueing may be used to provide the user with feedback. An experiment that 

investigated the impact of automation on young and older adults’ abilities to detect threat objects 

found that spatial cueing (e.g., highlighting a specific target) improved performance for both age 

groups; conversely, text cueing helped young adults but had no impact on older adults’

performance (Wiegmann et al., 2006). Sharples and colleagues (2007) identified the impact of 

different types of communication (i.e., text, radio, face-to-face) on trust development, in which 

true commands are trusted more than false commands. Alarms can be designed to provide very 

specific types of feedback to the operator. It is important that the user understand the necessity 

and benefit of the alarm. The effect of the type of alarm on trust development appears to be task 

dependent. For example, Gupta and colleagues (2002) found no significant difference in trust by 

type of alarm in a collision avoidance system, yet Donmez and colleagues (2006) found that 

drivers trust visual more than auditory alarms for distraction mitigation. The sensitivity of 

alarms—specifically, the frequency and duration—also affects trust development and use. Gupta 

and colleagues (2002) found that a reduced number of alarms led to more trust in a collision 

avoidance system. Wickens and colleagues (2009) support this finding by suggesting that 

operators perceive excessive alarms to contain false alerts and are thereby unnecessary. This 
19

leads to distrust in the automation and an operator disregard for true alerts. However, Lees and 

Lee (2007) found an interesting distinction between false alarms and unnecessary alarms. They 

found that while false alarms decreased trust and compliance levels, unnecessary alarms 

(instances where the automated device indicates a hazardous situations, but the human may not 

view them as hazardous) actually increase trust and compliance levels.

3.4 Factors Associated With the Environment

While factors specific to the human and the automation are important in trust development, there 

are also factors specific to the environment. These environment-based factors often require an 

interaction between the human and automation within a specific context. Trust is often a taskdependent construct (refer to section 2). The task domain and the environment that it is 

completed in can impact trust development. Risk and uncertainty are of essential importance in 

trust development. Here we discuss risk in terms of both the risk of the automation system as 

well as the risk from the environment. For example, Beckles et al. (2005) looked at the internal 

risk of data corruption from the automation of a computerized credential management system. 

They suggest that the system will not be trusted if the information is either compromised or 

revoked. Lyons et al. (2011) used factor analysis to examine trust in the context of IT suspicion 

and determined trust and distrust to be orthogonal constructs, and levels of trust are dependent 

upon cues in the environment. Borst et al. (2010) investigated automation-based pilot alerting 

systems for the external risk of the physical terrain. They suggest that trust is an important 

element in these types of automated systems because of the safety risk associated with this type 

of human-automation interaction.

4. Analytical Method

4.1 Sample of Studies

A review of empirical and nonempirical articles dealing specifically with human-automation 

trust was conducted using Web of Science and EBSCOhost databases with automation and trust

as the primary search terms. We also used Google and its derivative Google Scholar to perform 

searches for the primary search terms. After an initial listing of articles was obtained, the 

references for these papers were reviewed to determine whether any other related studies could 

be identified. This process resulted in 343 articles and conference proceedings published 

between 1960 and 2012. Both empirical studies and nonempirical reports were collected initially. 

4.2 Criteria for Study Inclusion

All 343 identified studies were inspected to ensure that they fulfilled the following four criteria 

for inclusion in the meta-analysis: 
20

1. Each study had to report an empirical examination of trust in which trust was a directly 

measured outcome of an experimental manipulation. Studies in which trust served as the 

experimental manipulation were excluded. 

2. The empirical examination of trust was directed toward automation. Thus, studies 

investigating interpersonal trust and human-robot trust were excluded. Empirical studies 

that focused on trust in automated computer-based systems were included in this analysis. 

3. Each included study was required to incorporate human participants who either viewed or 

participated directly in interactions with the automation through physical, virtual, or 

augmented means. 

4. Each study must include sufficient information to determine effect size estimates. Authors 

of potentially relevant statistics were contacted for additional statistical data needed to 

calculate effect size. 

Of the original 343 articles on human-automation trust that were collected, 287 were 

immediately omitted because of failure to meet the criteria for inclusion in the meta-analysis. 

First, 12 articles were written in another language other than English or could not be located. 

Second, 57 articles investigated trust in something other than automation (e.g., financial trusts,

interpersonal trust, robotics). Third, 218 articles did not directly measure trust and were excluded 

from the analysis. Of these discarded articles, 126 discussed theoretical underpinnings for trust 

and potential antecedents of trust in automation, 48 introduced factors involved in humanautomation interaction, and 44 discussed programming, automation design, and/or development. 

A total of 56 empirical articles were identified; however, 27 of these articles had insufficient 

information to calculate effect size. Authors were contacted via email and were given 5 weeks to 

respond with additional information. Of these studies, 14 were not included in the final analysis.

It is important to note that rejecting primary studies in a meta-analysis is a common occurrence 

and is necessary to ensure meaningful results when combining effect sizes across studies. 

The 42 papers meeting the inclusion criteria are identified in appendix C of this report by either 

an asterisk (*) or a cross (+) appearing in front of the first author’s name. The asterisk (*)

represents those studies included in the experimental analyses. The cross (+) represents those 

studies included in the correlational analyses (see also appendix D).

4.3 Coding of Studies

Available study characteristics from each experiment were coded, including, but not limited, to 

the following: dependent variable(s), independent variable(s), statistical analysis used, test 

values, degrees of freedom, means, standard deviations, alpha level, and statistical significance. 

If more than one variable was manipulated in a study, each independent variable was coded and 

analyzed separately. Several additional variables were also coded, including the following: (1) 

type of task, (2) participant population, (3) type of trust (e.g., trustworthiness, general trust), and 
21

(4) main moderator (human, automation, or environment), submoderator (refer to figure 4), and 

antecedent of trust (described in section 3). Participant population and type of trust were not 

included for analysis because of the lack of variation between studies. 

4.4 Effect Size

The studies included in effect size calculation contained both correlational and group design 

data, therefore the use of multiple meta-analytic methods (Pearson’s r and Hedges’ g) was 

necessary. The correlational effects represent an association between trust and the given factor. 

Hedges’s g indicates the standard difference between two means in standard deviation units. 

From these we can gather correlational and causal inferences between trust and any given factor. 

Through both types of meta-analytic effects, the more positive the effect represents a higher level 

of trust. Findings were interpreted using Hedges’ g and Pearson’s r established ranges for small 

(g ≤ .20, r ≤ .10), medium (g = .50, r =.25), and large (g ≥ .80, r ≥ .40) effect sizes.

4.5 Variance Estimates

Several variance estimates were calculated. First, variability of the effect sizes themselves (s

2

g) 

and variability due to sampling error (s

2

e) were estimated. Next, these two values were used to 

compute the residual variance (s

2

δ). A large (s

2

δ) is an indication that the effect sizes may be 

heterogeneous, and therefore one or more variables are likely to be moderating the magnitude of 

that particular effect. A final check for homogeneity of variance �

se

2

sg � 2� was calculated (proportion 

of total variance accounted for by sampling error). Hunter and Schmidt (2004) suggest that an 

outcome here of 0.75 or greater implies that the remaining variance is due to statistical artifacts,

and such cases are therefore interpreted as indicating homogeneity of variance among the effect 

sizes. However, large residual variance and small homogeneity of variance may also be due to a 

small number of studies, as is evident in some of the following results.

5. Results

The 42 papers meeting the inclusion criteria yielded a total of 166 experimental effect sizes and 

63 correlational effect sizes. Following organization of the studies, three main types of analyses 

were conducted: trusts preference for human or automation assistance (section 5.1), trust in type 

of task by cognitive, control, and perceptual aids (section 5.2), and moderator analyses (section 

5.3 and 5.4). Moderator analyses can be directly linked to figure 3 discussing the human-,

automation-, and environmental-related factors of trust. 

5.1 Preference for Human or Automation Assistance

There is debate in the literature as to how humans trust technology. While some literature has 

suggested that human users of technology tend to trust themselves or other humans more than 

technology, others suggest the exact opposite (see Dzindolet et al., 2001; Lyons and Stokes,
22

2012). Three empirical studies on how humans trust in automation, yielding nine statistics, were 

obtained (see De Vries et al., 2003; Ma and Kaber, 2007; Madhavan and Wiegmann, 2007). The 

Lyons and Stokes (2012) study was not retained, as it measured reliance, not specifically trust. 

Similarly, the study by Dzindolet and her colleagues (2001) was not retained, as it focused on 

disuse and misuse of automation, and not directly trust development. 

Results (g̅= –1.30) support the finding that operators of automation tend to trust themselves or 

other humans more than the technology (table 3). Although the results suggest large variability in 

the effect sizes themselves and sampling error, the proportion of total variance accounted for by 

sampling error (+.75) may be indicative of the small number of studies. There were no 

correlation-based analyses conducted for this factor. This finding is important to keep in mind 

when designing automated systems.

Table 3. Meta-analytic results for perception of trust in a human or automation for a task.

Category k g� sg

2

se

2 sδ

2

se

2

sg � 2 95% CI N

Lower Upper

Human vs. Automation 3 –1.30 .81 .61 .21 .75 –2.18 –.41 143

k = number of studies.

N = sample size.

s

2

g estimates the variability of the effect sizes themselves.

s

2

e estimates the variability due to sampling error. 

s

2

δ is an estimate of the residual variance.

(s2

e / s2g) is a calculation of homogeneity of variance.

5.2 Task-related Automation

While 30 experimental studies yielding 164 experimental effect sizes met the original selection 

criteria, this analysis only included studies that were specifically categorized by the task-related 

type of automation (i.e., cognitive, perceptual, or control aids). Therefore, 27 experimental 

studies, which yielded 155 experimental effect sizes, were included for analysis. These results,

shown in table 4, indicated there was a moderate global effect concerning trust and automation 

(g̅= +0.48). As the confidence interval here excluded zero, we can assume this is a substantive 

and consistent effect. The subdivision of this global effect into the three types of task-related

automation indicated that the cognitive (13 studies, 72 statistics), perceptual (6 studies, 14 

statistics), and control (8 studies, 68 statistics) aids all had a moderate effect on trust 

development. All findings appear to be consistent; however, the check for the homogeneity of 

variance for the cognitive aids suggests that the remaining variance may be due to statistical 

artifacts. This finding allowed us to complete all additional analysis as a complete sample 

without the need to consider task type as a moderating variable.
23

Table 4. Meta-analytic results for trust in task-based automation.

Category k g� sg

2

se

2 sδ

2 se

2

sg � 2 95% CI N

Lower Upper

Type of Automation 27 +.48 .38 .26 .12 .69 .28 .68 999

Cognitive Aids 13 +.41 .20 .20 .00 1.00 .14 .67 533

Perceptual Aids 6 +.62 .63 .30 .33 .48 .18 1.06 134

Control Aids 8 +.51 .54 .36 .18 .67 .09 .93 333

k = number of studies.

N = sample size. 

s

2

g estimates the variability of the effect sizes themselves.

s

2

e estimates the variability due to sampling error. 

s

2

δ is an estimate of the residual variance.

(s2

e / s2g) is a calculation of homogeneity of variance.

Correlation analyses, including 16 studies and 63 effect sizes, were also conducted. Ten studies 

(43 effect sizes) were included in cognitive aids, yielding a moderate to large effect (r̅= +0.39) 

on trust development. Perceptual aids yielded a moderate effect (r̅= +0.37) on trust 

development; however, this result was limited to one study. Five studies (19 effect sizes) were 

included in control aids, yielding a small effect on trust development (r̅= +0.12).

5.3 Human-Related Factors

Theoretical work across technological domains suggests the importance of the human element in 

trust development. Nine studies (54 statistics) were included for analysis, with three being 

specific to age, four specific to cognitive factors, and three specific to emotive factors. Overall,

the human element has a moderate effect (g̅= +0.49) on trust (table 5). When looking 

specifically at the submoderating effects of age, we found a moderate effect (g̅= +0.44) on trust 

development. The results of these three studies suggest that automation is trusted more by older 

adults than by younger adults. Only one study discussed the correlation between age and trust, so 

no additional analyses were conducted. The submoderating effect of cognitive factors suggests a 

small to medium effect (g̅= +0.39). Additional research should be conducted in this area to 

solidify the effects of cognitive traits on trust development. Three experimental studies on 

emotive factors of trust development yielded a moderate to large effect (g� = +0.72) on trust 

development. Six studies, yielding 11 effect sizes, were included for correlation-based analysis 

of emotive factors. This yielded a small to medium effect (r̅= +0.16), suggesting that trust and 

emotive factors are indeed related, albeit small.
24

Table 5. Meta-analytic results for human-related factors on trust in automation.

Category k g� sg

2

se

2 sδ

2 se

2

sg � 2 95% CI N

Lower Upper

Human 9 +.49 .08 .13 –.05 1.70 .25 .72 395

Traits (Age) 3 +.44 .15 .09 .06 .63 .09 .79 143

Cognitive Factors 4 +.39 .01 .15 –.14 15.39 .01 .77 162

Emotive Factors 3 +.72 .01 .18 –.17 21.18 .24 1.20 104

k = number of studies.

N = sample size. 

s

2

g estimates the variability of the effect sizes themselves.

s

2

e estimates the variability due to sampling error. 

s

2

δ is an estimate of the residual variance.

(s2

e / s2g) is a calculation of homogeneity of variance.

5.4 Automation-Related Factors

Automation-related factors were the most researched area of trust in automation with 22 studies

(100 statistics). Overall analysis of the automation-related factor suggests a moderate effect (g� =

+0.53) on trust development (table 6).

Table 6. Meta-analytic results for automation-related factors on trust in automation.

Category k g� sg

2

se

2 sδ

2 se

2

sg � 2 95% CI N

Lower Upper

Automation 22 +.53 .52 .31 .21 .59 .29 .78 766

Features 8 +.35 .17 .36 –.19 2.12 –.10 .79 270

Capability 16 +.70 .87 .30 .57 .35 .41 .99 517

Behavior 10 +.79 .50 .29 .22 .57 .44 1.15 437

Error 3 +.44 .36 .09 .28 .24 .11 .77 140

Feedback 6 +.45 1.93 .82 1.11 .43 –.28 1.18 140

k = number of studies.

N = sample size. 

s

2

g estimates the variability of the effect sizes themselves.

s

2

e estimates the variability due to sampling error. 

s

2

δ is an estimate of the residual variance.

(s2

e / s2g) is a calculation of homogeneity of variance.

A more detailed look at the submoderating factors of features and capabilities provides a more 

detailed understanding of the findings. Literature suggests a theoretical importance of automation 

features on trust development. However, the effects of correlation analysis based on only two 

studies (r̅= +0.16) and experimental analysis (g̅= +0.35) include zero in the confidence 

interval, suggesting an unstable effect. Analyses of automation capabilities showed a moderate 

correlational effect on trust (r̅= +0.39) and moderate to large experimental effect (g̅= +0.70) on 

trust development. The antecedents of the automation capabilities, behavior (g̅= +.79) and error 

(g̅= +0.44), were also analyzed. These results suggest that the capability of the automation is 

important to trust development in that the behaviors should be reliable and that lower error rate 

leads to more trust in the system. 
25

6. Discussion

Roboticists are more recently attempting to extend the classification of a robot beyond that of 

traditional automation to include the importance of cognition and intelligence (see Fong et al.,

2001; Yagoda, 2011), as well as the environment (see Groom, 2008; Lin et al., 2008). The types 

of interaction, the number of systems a user may interact with, the environment, and the physical 

nature of the robots (e.g., complex, dynamic control systems, exhibition of autonomy and 

cognition) further differentiate HRI from human-computer or human-machine interaction 

(Scholtz, 2003). However, the area of automation and trust in automation overlaps with humanrobot trust to some degree and can help us better understand human-robot trust and extend the 

human robot trust model first discussed in Hancock et al. (2011, see also figure 1). Figure 4

represents the updated descriptive model of human-robot trust that takes into account the 

findings from this human-automation trust meta-analysis.

Human Robot Environment

Demographic Traits Features Task-Related

Age Levels of Automation Risk/Uncertainty

Gender Appearance/Anthropomorphism Physical Environment

Ethnicity/Culture Type Task Type/Multitasking

Personality Robot Personality

Communication Mode

States Team Collaboration

Fatigue Capability In-group Membership

Stress Reliability/Errors Shared Mental Models

Attentional Control Behaviors(s)

Feedback/Cueing/Alarms

Cognitive Factors

Understanding

Ability to use/Attention

Expectancy

Emotive Factors

Attitudes

Confidence

Satisfaction

Comfort

Figure 4. Revised human-robot descriptive trust model.

A number of changes were made from the original descriptive trust model. The human-related 

factor was reorganized from ability-based characteristics and human characteristics subheadings 

to more specific subheadings (i.e., demographic traits, human states, cognitive factors, and 

emotive factors). The robot-related factor was reorganized from performance-based and 

attribute-based subheadings to features and capabilities. Findings from this work provided 

additional support for the importance of communication on trust development. Communication 
26

was also moved from the environmental-related factor to the robot-related factor and discussed 

in terms of communication mode (robot feature) and feedback (robot capability). Overall 

findings demonstrated that the human and automation-related factors had a moderate to large 

effect on trust development. Findings from this study were also used to identify potential training 

and design benefits specific to human-robot interaction.

6.1 Training Suggestions

1. Including some degree of error or failure during training may be important in trust 

calibration. While our analysis demonstrated that traditionally a low error rate has a 

moderate effect on trust development (g̅= +0.44), Bahner and colleagues (2008) suggested

that exposing users to failures during training can be used to reduce complacency and 

automation bias. An over-trusting operator is more likely to become complacent and follow 

the suggestions of the automation without cross-checking the validity against other 

available and accessible information. Automation bias is the operator’s tendency to trust 

the automation even when contradictory information is present or when information 

provided by the aid is known to be incorrect.

2. Scenario training may decrease the operators’ unwillingness to use automation. 

Automation has been shown to provide benefits to tasks that were traditionally 

accomplished by humans. Our meta-analytic findings suggest that despite these known 

benefits, operators or users may tend to trust other humans or themselves more than 

automation (g̅= –1.30). Scenario training may assist users or operators by allowing them to 

compare the utilities of automated and nonautomated control (Beck et al., 2007), which can 

reduce these effects. 

3. Training can provide additional understanding of the system. An increase in automation is 

predicted for the future, especially in the robotic domain. Sharples et al. (2007) suggest that 

novel technologies for flight deck air traffic control systems have great potential for 

increasing safety when the system is understood by the team interacting with it. Therefore,

operators and users need additional training with capabilities of the systems. Our analysis 

demonstrated that automation capabilities (g̅= +0.70), behaviors effect (g� = +.79), and 

communication feedback (g̅= +.45) between the operator and automation had a large effect 

on trust development. 

6.2 Design Recommendations

1. LOAs should be reviewed to determine specifications of the automation for a given task.

Often within human-automation interaction, the LOA dictates the type of communication 

that occurs. Communication in terms of accuracy, feedback, cueing, mode, and access to 

information was shown to be essential to trust development. Our findings suggested that 

the features of the automation including LOA, mode of communication (g�= +.35), as well 
27

as the capability of the automation in terms of communication feedback (g� = +.45), were 

each shown to have a small to moderate effect on trust development. 

The most referenced material on levels of automation is the Parasuraman et al. (2000) 

article on the 10 levels of automation (figure 5). Robots tend to incorporate multiple types 

of automation across cognitive, perceptual, and control aids. Therefore, these 10 levels may 

provide too much detail to successfully incorporate and test in the design. One possible 

solution is a reduction in levels to include human-in-the-loop (manual or frequent human 

interaction), human-on-the-loop (human monitors and has ability to intervene), and 

autonomous (no human intervention).

Figure 5. Levels of automation (Parasuraman et al., 2000).

2. Control displays should be appropriately designed to enhance communication. Many 

different sensors are utilized in robot design and use multiple modalities (visual, auditory,

tactile) to communicate with human operators or users. Reising and Sanderson (2002) 

suggested that control displays should be appropriately instrumented and designed 

according to the principles of ecological interface design, otherwise they might lead to 

misinterpretation. Our analysis suggested that there is a large effect of communication on 

trust development; however, more research needs to occur to determine the correct 

communication modality based on task needs and physical environment. In addition, more 

research needs to be conducted on the effects of the type of aid (cognitive, perceptual,

control) on human-automation trust. 

3. Design can have a direct impact on human safety. Semmer (2005) suggested that trust can 

impact how operators interact with automation and that designers should include the social 

and political aspects of design as well as the individualistic needs. Theoretical work in 

automation suggests the importance of risk and uncertainty in trust development; however,

empirical literature is lacking in this domain. The relationship between robot design and 

safety becomes an even more important area of research as robotic capabilities continue to 

advance through increased interdependency of multiple types of automation, as well as the 

societal need for robots to interact in a wide variety of physical environments. 

High 10. The automation decides everything, acts autonomously, ignoring the human.

9. The automation informs the human only if it, the robot, decides to

8. The automation informs the human only if asked

7. The automation executes automatically, then necessarily informs the human

6. The automation allows the human a restricted time to veto before automatic execution

5. The automation executes that suggestion if the human approves

4. The automation suggests one alternative

3. The automation narrows the selection down to a few options

2. The automation offers a complete set of decision/action alternatives

Low 1. The automation offers no assistance: the human must make all decisions and actions
28

7. Conclusions

Across the years and especially in the more recent decades, we have seen the general functions 

of the military become ever more subject to technological innovation. This includes all aspects of 

the military mission, but it is most especially evident in combat and conflict studies. All trends 

look toward a “distancing” from immediate interaction in which the agency of such distancing is 

technology. Not solely a mediating influence, advanced technologies promise to become a 

determining influence in which all decisions we advanced to are thus promulgated by 

“automation.” That those systems are currently simply design destinations at remote points in 

space and time are of little comfort to remaining combatants and to perception of the greater

public in general. Meanwhile, we are entering into a phase of hybrid control where action is 

effected by automation but mediated and monitored by remaining humans-in-the-loop. As this 

latter situation is an interactive one, we have to look to examine the effectiveness of that 

interaction and the factors that mediate its effect. Here, trust is a vital dimension. A system 

untrusted is a system unused or, worse, is a system misused. Yet, over-trust brings its own 

inherent problems. While our overall focus is on the Soldier-robot relationship, we directly 

acknowledge that this represents a subset of the overall Soldier-automation relationship. 
29

8. References

Aberg, J., Shahmehri, N. The Role of Human Web Assistants in E-commerce: An Analysis and a 

Usability Study. Internet Research 2000, 10 (2), 114–125. 

Adams, B. D.; Webb, R. D. G. Model of Trust Development in Small Teams (CR-2003-016);

Defense and Research Development, Human Systems Inc.: Toronto, Canada, 2003. 

Ardern-Jones, J.; Hughes, D. K.; Rowe, P. H.; Mottram, D. R.; Green, C. F. Attitudes and 

Opinions of Nursing and Medical Staff Regarding the Supply and Storage of Medicinal 

Products Before and After the Installation of a Drawer-Based Automated Stock-Control 

System. International Journal of Pharmacy Practice 2009, 17 (2), 95–99. 

Ashleigh, M. J.; Stanton, N. A. Trust: Key Elements in Human Supervisory Control Domains. 

Cognition, Technology & Work 2001, 3 (2), 92–100. 

Bagheri, N.; Jamieson, G. Considering Subjective Trust and Monitoring Behavior in Assessing 

Automation-Induced Complacency. Proceedings From the Human Performance, Situation 

Awareness and Automation Conference, Marietta, GA, 2004.

Bahner, J. E.; Hiper, A. D.; Manzey, D. Misuse of Automated Decision Aids: Complacency,

Automation Bias and the Impact of Training Experience. International Journal of HumanComputer Studies 2008, 66 (9), 688–699. 

Bailey N. R.; Scerbo M. W.; Freeman F. G.; Mikulka P. J.; Scott, L. A. Comparison of a BrainBased Adaptive System and a Manual Adaptable System for Invoking Automation. Human 

Factors: The Journal of the Human Factors and Ergonomics Society 2006, 48 (4), 693–709. 

Bandura, A. Self-Efficacy Mechanism in Human Agency. American Psychologist 1982, 37 (2),

122–147. 

Bandura, A. Self-Efficacy: The Exercise of Control; W. H. Freeman and Company: New York,

1997.

Barber, B. The Logic and Limits of Trust; Rutgers University Press: New Brunswick, NJ, 1983.

Barnett, J. S. How Training Affects Soldier Attitudes and Behaviors Toward Digital Command 

and Control Systems. Military Psychology 2007, 19 (1), 45–59. 

Beck H. P.; Dzindolet, M. T.; Pierce, L. G. Automation Usage Decisions: Controlling Intent and 

Appraisal Errors in a Target Detection Task. Human Factors 2007, 49 (30), 429–437.

Beckles, B.; Welch, V.; Basney, J. Mechanisms for Increasing the Usability of Grid Security. 

International Journal of Human-Computer Studies 2005, 63 (1/2), 74–101. 
30

Bentley, R.; Hughes, J. A.; Randall, D.; Shapiro, D. Z. Technological Support for Decision 

Making in a Safety Critical Environment. Safety Science 1995, 19 (2/3), 149–156. 

Billings, D. R.; Schaefer, K. E.; Llorens, N.; Hancock, P. A. What Is Trust? Defining the 

Construct Across Domains. Poster presented at the American Psychological Association 

Conference, Division 21, Orlando, FL, August 2012. 

Biros, D. P.; Daly, M.; Gunsch, G. The Influence of Task Load and Automation Trust on 

Deception Detection. Group Decision and Negotiation 2004, 13 (2), 173–189. 

Bisantz, A. M.; Seong, Y. Assessment of Operator Trust in and Utilization of Automated 

Decision-Aids Under Different Framing Conditions. International Journal of Industrial 

Ergonomics 2001, 28 (2), 85–97.

Bitan, Y.; Meyer, J. Self-Initiated and Respondent Actions in a Simulated Control Task. 

Ergonomics 2007, 50 (5), 763–788. 

Bliss, J. P.; Acton, S. A. Alarm Mistrust in Automobiles: How Collision Alarm Reliability 

Affects Driving. Applied Ergonomics 2003, 34 (6), 499–509. 

Boon, S.; Holmes, J. The Dynamics of Interpersonal Trust: Resolving Uncertainty in the Face of 

Risk. In Hindle, R., Groebel, J., Eds.; Cooperation and Prosocial Behavior; New York: 

Cambridge University Press, 1991; 167–182. 

Bonsall, P. W. Joint, M. Evidence on Driver’s Reaction To In-Vehicle Route Guidance Advice. 

Proceedings of the 24th ISATA International Symposium on Automotive Technology and 

Automation, Florence, Italy, 1991.

Borst, C.; Mulder, M.; van Paassen, M. M. Design and Simulator Evaluation of an Ecological 

Synthetic Vision Display. Journal of Guidance, Control, and Dynamics 2010, 33 (5), 

1577–1591.

Cahour, B.; Forzy, J. Does Projection Into Use Improve Trust and Exploration? An Example 

With a Cruise Control System. Safety Science 2009, 47 (9), 1260–1270. 

Cai, H.; Lin, Y. Coordinating Cognitive Assistance With Cognitive Engagement Control 

Approaches in Human-Machine Collaboration. IEEE Transactions on Systems, Man, and 

Cybernetics, Part A: Systems and Humans 2012, 42 (2), 286–294. 

Cai, H.; Lin, Y.; Cheng, B. Coordinating Multi-Level Cognitive Assistance and Incorporating

Dynamic Confidence Information in Driver-Vehicle Interfaces. Human Factors and 

Ergonomics in Manufacturing & Service Industries 2012, 22 (5), 437–449. 

Chen, J. Y. C.; Barnes, M. J. Supervisory Control of Multiple Robots: Effects of Imperfect 

Automation and Individual Differences. Human Factors 2012, 54 (2), 157–174.
31

Chen, J. Y. C.; Terrence, P. I. Effects of Imperfect Automation and Individual Differences on 

Concurrent Performance of Military and Robotics Tasks in a Simulated Multi-Tasking 

Environment. Ergonomics 2009, 52 (8), 907–920.

Cohen, M. S.; Parasuraman, R.; Freeman, J. T. Trust in Decision Aids: A Model and Its Training 

Implications. Proceedings From the 1998 Command and Control Research and Technology 

Symposium, Washington, DC, 1998.

Corritore, C.; Kracher, B.; Wiedenbeck, S. On-Line Trust: Concepts, Evolving Themes, a Model. 

International Journal of Human-Computer Studies 2003, 58 (6), 737–758. 

Costa, A. C.; Roe, R. A. Taillieu, T. Trust Within Teams: The Relation With Performance 

Effectiveness. European Journal of Work and Organizational Psychology 2001, 10 (3),

225–244.

Cummings, M. L.; Clare, A.; Hart, C. The Role of Human-Automation Consensus in Multiple 

Unmanned Vehicle Scheduling. Human Factors 2010, 52 (1), 17–27. 

Curral, S. C. The Role of Interpersonal Trust in Work Relationships. Doctoral dissertation,

Cornell University, Ithaca, New York, 1990.

Curral, S. C.; Inkpen, A. C. A Multilevel Approach to Trustin Joint Ventures. Journal of 

International Business Studies 2002, 33 (3), 479–95. 

de Visser, E.; Parasuraman, R. Adaptive Aiding of Human-Robot Teaming Effects of Imperfect 

Automation on Performance, Trust, and Workload. Journal of Cognitive Engineering and 

Decision Making 2011, 5 (2), 209–231. 

De Vries, P.; Midden, C. Effect of Indirect Information on System Trust and Control Allocation. 

Behavior & Information Technology 2008, 27 (1), 17–29.

De Vries, P.; Midden, C.; Bouwhuis, D. The Effects of Errors on System Trust, Self-Confidence,

and the Allocation of Control in Route Planning. International Journal of Human-Computer 

Studies 2003, 58 (6), 719–735.

Defense Science Board (DSB). The Role of Autonomy in Department of Defense Systems. Task 

Force Report. Office of the Under Secretary of Defense for Acquisition, Technology and 

Logistics: Washington, DC, 2012; http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf

(accessed 23 May 2014).

Derryberry, D.; Reed, M. A. Anxiety-Related Attentional Biases and Their Regulation by 

Attentional Control. Journal of Abnormal Psychology 2002, 111, 225–236.

Deutsch, M. The Resolution of Conflict: Constructive and Destructive Processes; Yale 

University Press: New Haven, CT, 1973.
32

Dixon, S. R.; Wickens, C. D. Automation Reliability in Unmanned Aerial Vehicle Control: A 

Reliance-Compliance Model of Automation Dependence in High Workload. Human Factors 

2006, 48 (3), 474–486. 

Dixon, S. R.; Wickens, C. D.; McCarley, J. S. On the Independence of Compliance and Reliance: 

Are Automation False Alarms Worse Than Misses? Human Factors 2007, 49 (4), 564–572. 

Donmez, B.; Boyle, L. N.; Lee, J. D.; McGehee, D. V. Drivers’ Attitudes Toward Imperfect 

Distraction Mitigation Strategies. Transportation Research Part F: Traffic Psychology and 

Behaviour 2006, 9 (6), 387–398.

Dzindolet, M. T.; Beck, H. P.; Pierce, L. G. Adaptive Automation: Building Flexibility Into 

Human-Machine Systems. Advances in Human Performance and Cognitive Engineering 

Research 2006, 6, 213–245. 

Dzindolet, M. T.; Peterson, S. A.; Pomranky, R. A.; Pierce, L. G.; Beck, H. P. The Role of Trust 

in Automation Reliance. International Journal of Human-Computer Studies 2003, 58 (6),

697–718. 

Dzindolet, M. T.; Pierce, L. G.; Beck, H. P.; Dawe, L. A.; Anderson, B. W. Predicting Misuse 

and Disuse of Combat Identification Systems. Military Psychology 2001, 13 (3), 147–164.

Ezer, N.; Fisk, A. D.; Rogers, W. A. Age-Related Differences in Reliance Behaviour 

Attributable to Costs Within A Human-Decision Aid System. Human Factors 2008, 50 (6),

853–863. 

Fan, T.; Hyams, D.; Kuchar, J. K. Preliminary Study of In-Flight Replanning Performed on the 

Flight Deck; National Center of Excellence for Aviation Operations Research (NEXTOR) 

Research Report RR-98-04: MIT, Cambridge, MA, 1998.

Feil-Seifer, D.; Matarić, M. J. Defining Socially Assistive Robotics. Proceedings of the 9th IEEE 

International Conference on Rehabilitation Robotics, Chicago, IL 2005; pp 465–668. 

Finomore, V.; Matthews, G.; Shaw, T.; Warm, J. Predicting Vigilance: A Fresh Look at an Old 

Problem. Ergonomics 2009, 52 (7), 791–808. 

Fong, T.; Thorpe, C.; Baur, C. Collaboration, Dialogue, and Human-Robot Interaction. Springer 

Tracts in Advanced Robotics 2001, 6, 255–266. 

Gao, J.; Lee, J. D.; Zhang, Y. A Dynamic Model of Interaction Between Reliance on Automation 

and Cooperation in Multi-Operator Multi-Automation Situations. International Journal of 

Industrial Ergonomics 2006, 36 (5), 511–526.
33

Goetz, J.; Kiesler, S.; Powers, A. Matching Robot Appearance and Behavior to Tasks To 

Improve Human-Robot Cooperation. Proceedings of the 12th IEEE International Workshop 

on Robot and Human Interactive Communication (RO-MAN 2003), Milbrae, CA, 2003; 

pp 55–60.

Groom, V. What’s the Best Role for Robot? Cybernetic Models of Existing and Proposed 

Human-Robot Interaction Structures. Proceedings of the 5th International Conference on 

Informatics in Control, Automation, and Robotics, 2008; pp 323–328.

Gupta, N.; Bisantz, A. M.; Singh, T. The Effects of Adverse Condition Warning System 

Characteristics on Driver Performance: An Investigation of Alarm Signal Type on Threshold 

Level. Behavior & Information Technology 2002, 21 (4), 235–248. 

Hancock, P. A. Warm a Dynamic Model of Stress and Sustained Attention. Human Factors 

1989, 31 (5), 519–537.

Hancock, P. A.; Desmond, P. A.; Matthews, G. Conceptualizing and Defining 

Fatigue. Matthews, G., Desmond, P. A., Neubauer, C., Hancock, P. A., (Eds.); In The 

Handbook of Operator Fatigue; Ashgate Publishing Company: Burlington, VT, 2012;

pp 63–73.

Hancock, P. A.; Billings, D. R.; Schaefer, K. E.; Chen, J. Y. C.; de Visser, E.; Parasuraman, R. A 

Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction. Human Factors 

2011, 53 (5), 517–527.

Ho, G.; Wheatley, D.; Scialfa, C. Age Differences in Trust and Reliance of a Medication 

Management System. Interacting With Computers 2005, 17 (6), 690–710. 

Hoffman, D. L.; Novak, T. P.; Peralta, M. Building Consumer Trust Online. Communications of 

the ACM 1999, 42 (4), 80–85. 

Hughes, J. S.; Rice, S.; Trafimow, D.; Clayton, K. The Automated Cockpit: A Comparison of 

Attitudes Towards Human and Automated Pilots. Transportation Research Part F: Traffic 

Psychology and Behaviour 2009, 12 (5), 428–439. 

Hunter, J. E.; Schmidt, F. L. Methods of Meta-Analysis: Correcting for Error and Bias in 

Research Findings; Sage: Thousand Oaks, CA, 2004.

Igbaria, M.; Iivari, J. The Effects of Self-Efficacy on Computer Usage. Omega 1995, 23 (6),

587–605. 

Jeannot, E.; Kelly, C.; Thompson, D. The Development of Situation Awareness Measures in 

ATM Systems; Eurocontrol: Brussels, Belgium, 2003.
34

Kazi, T. A.; Stanton, N. A.; Walker, G. H.; Young, M. S. Designer Driving: Drivers’ Conceptual 

Models and Level of Trust in Adaptive-Cruise-Control. International Journal of Vehicle 

Design 2007, 45 (3), 339–360.

Keil, M.; Beranek, P. M.; Konsynski, B. R. Usefulness and Ease of Use: Field Study Evidence 

Regarding Task Considerations. Decision Support Systems 1995, 13 (1), 75–91.

Kircher, K.; Thorslund, B. Effects of Road Surface Appearance and Low Friction Warning 

Systems on Driver Behaviour and Confidence the Warning System. Ergonomics 2009, 52

(2), 165–176. 

Klein, B. D.; Goodhue, D. L.; Davis, G. B. Can Humans Detect Errors in Data? Impact of Base 

Rates, Incentives, and Sis. MIS Quarterly 1997, 21 (2), 169–194.

Koh, Y. J.; Sundar, S. S. Effects of Specialization in Computer, Web Sites, and Web Agents on 

E-Commerce Trust. International Journal of Human-Computer Studies 2010, 68 (12),

899–912. 

Kurfess, T. R., Ed. Robotics and Automation Handbook; CRC Press: New York, 2005.

Lee, H.; Kim, J.; Kim, J. Determinants of Success for Applications Service Provider: An 

Empirical Test in Small Businesses. International Journal of Human-Computer Studies 

2007, 65 (9), 796–815. 

Lee, J.; Moray, N. Trust, Control Strategies and Allocation of Function in Human-Machine 

Systems. Ergonomics 1992, 35 (10), 1243–1270. 

Lee, J. D.; Moray N. Trust, Self-Confidence, and Operators’ Adaptation to Automation. 

International Journal of Human-Computer Studies 1994, 40 (1), 153–184. 

Lee, J. D.; See, K. A. Trust in Automation: Designing for Appropriate Reliance. Human Factors: 

The Journal of the Human Factors and Ergonomics Society 2004, 46 (1), 50–80. 

Lees, M. N.; Lee, J. D. The Influence of Distraction and Driving Context on Driver Response to 

Imperfect Collision Warning Systems. Ergonomics 2007, 50 (8), 1264–1286. 

Li, Y. M.; Yeh, Y. S. Increasing Trust in Mobile Commerce Through Design Aesthetics. 

Computers in Human Behavior 2010, 26 (4), 673–684.

Lin, P.; Bekey, G.; Abney, K. Autonomous Military Robotics: Risk, Ethics, and Design;

California Polytechnic State University: San Luis Obispo, CA, 2008. 

http://ethics.calpoly.edu/ONR_report.pdf (accessed June 2014).

Looije, R.; Neerincx, M. A.; Cnossen, F. Persuasive Robotic Assistant for Health SelfManagement of Older Adults: Design and Evaluation of Social Behaviors. International 

Journal of Human-Computer Studies 2010, 68 (6), 386–397. 
35

Luhmann, N. Trust and Power: Two Works by Niklas Luhmann; Wiley: Hoboken, NJ, 1979.

Lyons, J. B.; Stokes, C. K.; Eschleman, K. J.; Alarcon, G. M.; Barelka, A. J. Trustworthiness and 

IT Suspicion: An Evaluation of the Nomological Network. Human Factors 2011, 53 (3),

219–229.

Lyons, J. B.; Stokes, C. K. Human-Human Reliance in the Context of Automation. Human 

Factors: The Journal of the Human Factors and Ergonomics Society 2012, 54 (1), 112–121. 

Ma, R.; Kaber, D. B. Effects of In-Vehicle Navigation Assistance and Performance on Driver 

Trust and Vehicle Control. International Journal of Industrial Ergonomics 2007, 37 (8),

665–673.

Madhavan, P.; Wiegmann, D. A. Effects of Information Source, Pedigree, and Reliability on 

Operator Interaction With Decision Support Systems. Human Factors 2007, 49 (5),

773–785. 

Madsen, M. Gregor, S. Measuring Human-Computer Trust. Proceedings of 11th Australasian 

Conference on Information Systems, Gladstone, Australia, Central Queensland University, 

2000.

Mahoney, R. M. Robotic Products for Rehabilitation: Status and Strategy. Proceedings of 

ICORR ’97: International Conference on Rehabilitation Robotics, Bath, UK, 1997.

Maltz, M.; Meyer, J. Use of Warnings in an Attentionally Demanding Detection Task. Human 

Factors 2001, 43 (2), 217–226. 

Manzey, D.; Rottger, S.; Bahner-Heyne, J. E.; Schulze-Kissing, D.; Dietz, A.; Meixensberger, J.;

Strauss, G. Image-Guided Navigation: The Surgeon’s Perspective on Performance 

Consequences and Human Factors Issues. The International Journal of Medical Robotics 

and Computer Assisted Surgery 2009, 5 (3), 297–308.

Marsh, S.; Dibben, M. R. The Role of Trust in Information Science and Technology. Annual 

Review of Information Science and Technology 2003, 37 (1), 465–498. 

Master, R.; Bingham, J.; Jiang, X.; Gramopadhye, A. K.; Melloy, B. A Questionnaire for 

Measuring Trust in Hybrid Inspection Systems. Proceedings of the Industrial Engineering 

Research Conference, Dallas, TX, 2000.

Master, R.; Jiang, X.; Khasawneh, M. T.; Bowling, S. R.; Grimes, L.; Gramopadhye, A. K.;

Melloy, B. J. Measurement of Trust Over Time in Hybrid Inspection Systems. Human 

Factors and Ergonomics in Manufacturing & Service Industries 2005, 15 (2), 177–196. 

Mayer, R. C.; Davis, J. H.; Schoorman, F. D. An Integrative Model of Organizational Trust. 

Academy of Management Review 1995, 20 (3), 709–734.
36

Mayer, A.; Sanchez, J.; Fisk, A.; Rogers, W. Don’t Let Me Down: The Role of Operator 

Expectations in Human-Automation Interaction. Proceedings of the Human Factors and 

Ergonomics Society Annual Meeting 2006, 50 (21), 2345–2349. 

McLeod, R. W.; Walker, G. H.; Moray, N. Analysing and Modeling Train Driver Performance. 

Applied Ergonomics 2005, 36 (6), 671–680.

Merriam-Webster Web site. Trust. http://www.merriam-webster.com/dictionary/trust (accessed 

27 January 2013).

Merritt, S. M.; Heimbaugh, H.; LaChapell, J.; Lee, D. I Trust It, But I Don’t Know Why: Effects 

of Implicit Attitudes Toward Automation on Trust in an Automated System. Human Factors 

2012, 55 (3), 520–534. 

Merritt, S. M.; Ilgen, D. R. Not All Trust Is Created Equal: Dispositional and History-Based 

Trust in Human-Automation Interactions. Human Factors 2008, 50 (2), 194–210. 

Moorman, C.; Deshpande, R.; Zaltman, G. Factors Affecting Trust in Market-Research 

Relationships. Journal of Marketing 1993, 57 (1), 81–101. 

Moray, N.; Inagaki, T. Laboratory Studies of Trust Between Humans and Machines in 

Automated Systems. Transactions of the Institute of Measurement and Control 1999, 21

(4/5), 203–211. 

Moray, N.; Inagaki, T.; Itoh, M. Adaptive Automation, Trust, and Self-Confidence in Fault 

Management of Time-Critical Tasks. Journal of Experimental Psychology: Applied 2000, 6

(1), 44–58. 

Muir, B. M. Trust Between Humans and Machines, and the Design of Decision Aids. 

International Journal of Man-Machine Studies 1987, 27 (5/6), 527–539. 

Muir, B. M. Operators’ Trust in and Percentage of Time Spent Using the Automatic Controllers 

in a Supervisory Process Control Task. Doctoral Dissertation, University of Toronto,

Toronto, Canada, 1989.

Muir, B. M. Trust in Automation: Part 1. Theoretical Issues in the Study of Trust in Automated 

Systems. Ergonomics 1994, 37 (11), 1905–1922. 

Muir, B. M.; Moray, N. Trust in Automation. Part II. Experimental Studies of Trust and Human 

Intervention in a Process Control Simulation. Ergonomics 1996, 39 (3), 429–460. 

Nakada, K. Kansei Engineering Research on the Design of Construction Machinery. 

International Journal of Industrial Ergonomics 1997, 19 (2), 129–146. 

Nass, C.; Moon, Y. Machines and Mindlessness: Social Responses to Computers. Journal of 

Social Issues 2000, 56 (1), 81–103.
37

Neyedli, H. F.; Hollands, J. G.; Jamieson, G. A. Human Reliance On An Automated Combat ID 

System: Effects of Display Format. Proceedings of the Human Factors and Ergonomics 

Society Annual Meeting, 2009; 53 (4), pp 212-216. 

Neubauer, C.; Matthews, G.; Langheim, L.; Saxby, D. Fatigue and Voluntary Utilization of 

Automation in Simulated Driving. Human Factors 2012, 54 (5), 734–746.

Office of the Secretary of Defense. Unmanned Aircraft Systems Roadmap 2005–2030;

Washington, DC, 2005. http://handle.dtic.mil/100.2/ADA445081 (accessed 22 May 2014).

Pak, R.; Fink, N.; Price, M.; Bass, B.; Sturre, L. Decision Support Aids With Anthropomorphic 

Characteristics Influence Trust and Performance in Younger and Older Adults. Ergonomics,

2012, 55 (9), 1059–1072. 

Parasuraman, R. Human-Computer Monitoring. Human Factors 1987, 29 (6), 695–706. 

Parasuraman, R.; Miller, C. A. Trust and Etiquette in High-Criticality Automated Systems. 

Communications of the Association for Computing Machinery 2004, 47 (4), 51–55. 

Parasuraman, R.; Molloy, R.; Singh, I. L. Performance Consequences of Automation-Induced 

Complacency. The International Journal of Aviation Psychology 1993, 3 (1), 1–23. 

Parasuraman, R.; Riley, V. Humans and Automation: Use, Misuse, Disuse, Abuse. Human 

Factors 1997, 39 (2), 230–253. 

Parasuraman, R.; Sheridan, T. B.; Wickens, C. D. A Model for Types and Levels of Human 

Interaction With Automation. IEEE Transactions on Systems, Man and Cybernetics, Part A: 

Systems and Humans 2000, 30 (3), 286–297. 

Rajaonah, B. Study of Driver Trust During Cooperation With Adaptive Cruise Control.

Le Travail Humain 2006, 69 (2), 99–127. 

Rajaonah, B.; Anceaux, F.; Vienne, F. Study of Driver Trust During Cooperation With Adaptive 

Cruise Control. Le Travail Humain 2006, 69 (2), 99–127. 

Rani, M. R.; Sinclair, M. A.; Case, K. Human Mismatches and Preferences for Automation. 

International Journal of Production Research 2000, 38 (17), 4033–4039. 

Ratnasingham, P. The Importance of Trust in Electronic Commerce. Internet Research 1998, 8

(4), 313–321. 

Reichenbach, J.; Onnasch, L.; Manzey, D. Human Performance Consequences of Automated 

Decision Aids in States of Sleep Loss. Human Factors 2011, 53 (6), 717–728. 

Reising, D. V.; Sanderson, P. M. Work Domain Analysis and Sensors I: Principles and Simple 

Example. International Journal of Human-Computer Studies 2002, 56 (6), 569–596.
38

Rempel, J. K.; Holmes, J. G. How Do I Trust Thee? Psychology Today, February 1986, 20,

28–34.

Rempel, J. K.; Holmes, J. G.; Zanna, M. P. Trust in Close Relationships. Journal of Personality 

and Social Psychology 1985, 49 (1), 95–112.

Riley, V. A General Model of Mixed-Initiative Human-Machine Systems. Proceedings of the 

Human Factors and Ergonomics Society 1989, 33, 124–128.

Ross, J. Moderators of Trust and Reliance Across Multiple Decision Aids. Doctoral Dissertation,

University of Central Florida, Orlando, FL, 2008.

Rotter, J. B. A New Scale for the Measurement of Interpersonal Trust. Journal of Personality 

1967, 35 (4), 651–665. 

Sarter, N. B. Multimodal Information Presentation: Design Guidance and Research Challenges. 

International Journal of Industrial Ergonomics 2006, 36 (5), 439–445. 

Sarter, N. B.; Schroeder, B. Supporting Decision Making and Action Selection Under Time 

Pressure and Uncertainty: The Case of In-Flight Icing. Human Factors 2001, 43 (4),

573–583. 

Scallen, S. F.; Hancock, P. A. Implementing Adaptive Function Allocation. The International 

Journal of Aviation Psychology 2001, 11 (2), 197–221. 

Scanzoni, J. Social Exchange and Behavioral Interdependence. Burgess, R.L., Huston, T.L., 

Eds.; In Social Exchange in Developing Relationships; Academic Press: New York, 1979.

Schaefer, K. E. The Perception and Measurement of Human Robot Trust. Doctoral Dissertation, 

University of Central Florida, Orlando, FL, 2013. http://etd.fcla.edu/CF/CFE0004931 

/Schaefer_Kristin_E_201308_PhD.pdf (accessed June 2014).

Schobel, M. Trust in High-Reliability Organizations. Social Science Information 2009, 48 (2),

315–333. 

Schobel, M.; Manzey, D. Subjective Theories of Organizing and Learning From Events. Safety 

Science 2011, 49 (1), 47–54. 

Scholtz, J. Theory and Evaluation of Human Robot Interactions. Proceedings From HICSS ’03: 

36th Annual Hawaii International Conference on System Sciences, Hilton Waikoloa Village, 

Hawaii, 2003.

Semmer, N. K. Individualistic vs. Systemic, and Human vs. Technical Thinking. European 

Psychologist 2005, 10 (3), 237–243. 

Seppelt, B. D.; Lee, J. D. Making Adaptive Cruise Control (ACC) Limits Visible. International 

Journal of Human-Computer Studies 2007, 65 (3), 192–205. 
39

Sharples, S.; Stedmon, A.; Cox, G.; Nicholls, A.; Shuttleworth, T.; Wilson, J. Flight Deck and 

Air Traffic Control Collaboration Evaluation (FACE): Evaluating Aviation Communication 

in the Laboratory and Field. Applied Ergonomics 2007, 38 (4), 388–407.

Singh, I. L.; Molloy, R.; Parasuraman, R. Automation- Induced “Complacency”: Development of 

a Complacency-Potential Scale; Technical Report No. CSL-A-91-1; Cognitive Science 

Laboratory, Catholic University of America: Washington, DC, 1991.

Song, L.; Kuchar, J. K. Dissonance Between Multiple Alerting Systems Part I: Modeling and 

Analysis. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and 

Humans 2003, 33 (3), 366–375. 

Spain, R. D.; Bliss, J. P. The Effect of Sonification Display Pulse Rate and Reliability of 

Operator Trust and Perceived Workload During a Simulated Patient Monitoring Task. 

Ergonomics 2008, 51 (9), 1320–1337. 

Stanton, N. A.; Young, M. S.; Walker, G. H. The Psychology of Driving Automation: A 

Discussion With Professor Don Norman. International Journal of Vehicle Design 2007, 45

(30), 289–306. 

Stedmon, A. W.; Sharples, S.; Littlewood, R.; Cox, G.; Patel, H.; Wilson, J. R. Datalink in Air 

Traffic Management: Human Factors Issues in Communications. Applied Ergonomics 2007,

38 (4), 473–480. 

Steinfeld, A.; Fong, T.; Kaber, D.; Lewis, M.; Scholtz, J.; Schultz, A.; Goodrich, M. Common 

Metrics for Human-Robot Interaction. Proceedings of the 1st ACM/IEEE International 

Conference on Human Robot Interaction, Salt Lake City, UT, 2006; pp 33–40. 

Straus, G.; Koulechov, K.; Rottger, S.; Bahner, J.; Trantakis, C.; Hofer, M.; Luth, T. Evaluation 

of a Navigation System for ENT With Surgical Efficiency Criteria. The Laryngoscope 2006,

116 (4), 564–572. 

Sullivan, J. M.; Tsimhoni, O.; Bogard, S. Warning Reliability and Driver Performance in 

Naturalistic Driving. Human Factors 2008, 50 (5), 845–852. 

Swadesh, J. K.; Kasonf, C. J. Industrial Analytical Operations: Organizational Implications of 

Automation. Critical Reviews in Analytical Chemistry 1995, 25 (3), 195–202. 

Szalma, J. L.; Taylor, G. S. Individual Differences in Response to Automation: The Five Factor 

Model of Personality. Journal of Experimental Psychology: Applied 2011, 17 (2), 71–96. 

Takayama, L.; Nass, C. Driver Safety and Information From Afar: An Experimental Driving 

Simulator Study of Wireless vs. In-Car Information Services. International Journal of 

Human-Computer Studies 2008, 66 (3), 173–184.
40

Tenney, Y. J.; Rogers, W. H.; Pew, R. W. Pilot Opinions on Cockpit Automation Issues. The 

International Journal of Aviation Psychology 1998, 8 (2), 103–120. 

Thropp, J. E. Individual Differences Using Automation. Doctoral Dissertation, University of 

Central Florida, Orlando, FL, 2006.

Uggirala, A.; Gramopadhye, A. K.; Melloy, B. J.; Toler, J. E. Measurement of Trust in Complex 

and Dynamic Systems Using a Quantitative Approach. International Journal of Industrial 

Engineering 2004, 34 (3), 175–186.

van den Broek, E. L.; Westerink, J. H. Considerations for Emotion-Aware Consumer Products. 

Applied Ergonomics 2009, 40 (6), 1055–1064. 

Verberne, F. M. F.; Ham, J.; Midden, C. J. H. Trust in Smart Systems: Sharing Driving Goals 

and Giving Information to Increase Trustworthiness and Acceptability of Smart Systems in 

Cars. Human Factors 2012, 54 (5), 799–810. 

Wang, L.; Jamieson, G. A., Hollands, J. G. Trust and Reliance on an Automated Combat 

Identification System. Human Factors 2009, 51 (3), 281–291.

Wang, L.; Jamieson, G. A.; Hollands, J. G. The Effects of Design Features on Users’ Trust in 

and Reliance on a Combat Identification System. Proceedings of the Human Factors and 

Ergonomics Society 2011, 55 (1), 375–379.

Ward, J. N. Automation of Task Processes: An Example of Intelligent Transportation Systems. 

Human Factors and Ergonomics in Manufacturing and Service Industries 2000, 10 (4),

395–408. 

Warm, J. S.; Parasuraman, R.; Matthews, G. Vigilance Requires Hard Mental Work and Is 

Stressful. Human Factors 2008, 50 (3), 433–441. 

Wickens, C. D.; Rice, S.; Keller, D.; Hutchins, S.; Hughes, J.; Clayton, K. False Alerts in Air 

Traffic Control Conflict Alerting System: Is There a “Cry Wolf” Effect? Human Factors 

2009, 51 (4), 446–462. 

Wiegmann, D. A.; Eggman, A. A.; ElBardissi, A. W.; Parker, S. H.; Sundt, T. M. Improving 

Cardiac Surgical Care: A Work Systems Approach. Applied Ergonomics 2010, 41 (5),

701–712.

Wiegmann, D.; McCarley, J. S.; Kramer, A. F.; Wickens, C. D. Age and Automation Interact to 

Influence Performance of a Simulated Luggage Screening Task. Aviation, Space, and 

Environmental Medicine 2006, 77 (8), 825–831.

Wiener, E. L. Beyond the Sterile Cockpit. Human Factors 1985, 27 (1), 75–90. 
41

Yagoda, R. E. What! You Want Me To Trust a Robot? The Development of a Human Robot 

Interaction (HRI) Trust Scale. Master’s Thesis, North Carolina State University, Raleigh,

NC, 2011.

Yamada, K.; Kuchar, J. K. Preliminary Study of Behavioral and Safety Effects of Driver 

Dependence on a Warning System in a Driving Simulator. IEEE Transactions on Systems,

Man and Cybernetics, Part A: Systems and Humans 2006, 36 (30), 602–610.

Yang, C. W.; Yenn, T. C.; Lin, C. J. Assessing Team Workload Under Automation Based on a 

Subjective Performance Measure. Safety Science 2009, 48 (7), 914–920.
42

INTENTIONALLY LEFT BLANK.
43

Appendix A. Definitions
44

Attentional Control: One’s ability to focus and shift attention in a flexible manner (Chen and

Barnes, 2012; Derryberry and Reed, 2002).

Automation: (1) Machine execution of functions (Parasuraman et al., 2000); (2) “the execution 

by a machine agent (usually a computer) of a function that was previously carried out by a 

human” (Parasuraman and Riley, 1997, p. 231); (3) “the technology that actively selects data,

transforms information, makes decisions, or controls processes” (Lee and See, 2004, p. 50).

Cognitive Aids: Provide recommendations to users about the current and potential future states 

of systems.

Control Aids: Replace varying levels of human (operator/user) action.

Perceptual Aids: Used to assist the operator or user by providing warnings or to assist with 

pattern recognition.

Autonomy: (1) A capability (or set of capabilities) that enables a particular action of a system to 

be automatic or, within programmed boundaries, “self-governing” (Defense Science Board,

2012); (2) degree of authority to act on the “World” (Riley, 1989).

Etiquette (system): A collection of behaviors that indicate that the system can be deemed 

trustworthy, or displays behaviors that are satisfying in accordance to the customs of a culture 

(Parasuraman and Miller, 2004).

Fatigue: A reduction in the capacity and desire to react (Hancock et al., 2012).

Robot: (1) “Programmable automation to augment human manipulation” (Mahoney, 1997, p. 3); 

(2) an autonomous system that acts on its own decisions with input from a human operator but 

may not be completely controlled by a human (Feil-Seifer and Matarić, 2005); (3) the interaction 

of intelligence and autonomy of the technology (Yagoda, 2011); (4) a complex, dynamic system 

that shows a degree of autonomy and cognition as it operates in a real-world environment (Fong 

et al., 2001); (5) perceives the environment through sensor inputs that may or may not require 

direct human response and has the ability to move in order to manipulate an uncertain physical 

world by transmitting information at humans or directly acting on the environment (Groom,

2008; Lin et al., 2008).

Self-efficacy: An individual’s perceived abilities to successfully carry out an identifiable task 

(Bandura, 1982; 1997).

Stress: The force that degrades performance capability (Hancock and Warm, 1989).
45

Appendix B. Comprehensive Automation Trust Definition Set
46

Definition Citation Definition

Adams and Webb (2003) Trust is a psychological state involving positive confident expectations 

and willingness to act on the basis of these expectations. Issues of trust 

arise in contexts that involve risk, vulnerability, uncertainty and 

interdependence. Trust expectations are created primarily by the 

interaction of the perceived qualities of the trustee and contextual factors 

in play when trust decisions are made (p. 38).

Barber, B. (1983) Barber defines trust in terms of a taxonomy of three specific expectations 

that can be extended to the human/machine relationship, according to Muir 

(1994):

• our general expectation of the persistence of the natural physical order,

the natural biological order, and the moral social order,

• our specific expectation of technically competent role performance from 

those

involved with us in social relationships and systems,

• our specific expectation that partners in an interaction will carry out their 

fiduciary obligations and responsibilities, that is, their duty in certain 

situations to place others' interests before their own.

Bentley et al. (1995) Trust and familiarity is a process that is partially a function of technology 

(i.e., trust and familiarity can be embedded in use) (p. 151).

Biros et al. (2004) “System automation trust is defined as having confidence in and entrusting 

the system automation to do the appropriate action” (p. 177).

In this model, dependability and predictability contribute to an individual’s 

overall trust in a system’s automation.

Boon and Holmes (1991) “A state involving confident predictions about another’s motives with 

respect to oneself in situations entailing risk.”

Cohen et al. (1998) The perceived probability of the system’s reliability, given certain 

situations (APT model). “The problem of decision aid acceptance is 

neither under trust nor over trust, but inappropriate trust: a failure to 

understand or properly evaluate the conditions affecting good and bad aid 

performance” (p. 1).

Costa, Roe, and Taillieu (2001) “Trust is a psychological state that manifests itself in the behaviours 

towards others, is based on the expectations made upon behaviours of 

these others, and on the perceived motives and intentions in situations 

entailing risk for the relationship with those others” (p. 228).

Curral (1990) “Relying on someone under conditions of dependence and risk.”

Curral and Inkpen (2002) Trust is the decision to rely on a partner with the expectation that the 

partner will act according to a common agreement.

Deutsch (1973) “The confidence that one will find what is desired from another, rather 

than what is feared” (p. 148).

Note: References marked with a  represent definitions that are originally from another domain of trust but are referenced or 

applied to automation.
47

Definition Citation Definition

De Vries et al. (2003) “We consider trust to be the expectation of a user about the system, that 

the system will perform a certain task for him or her, while the outcome 

of that task is uncertain, in that it can have both positive and negative 

consequences. In other words, an element of risk should be present” 

(p. 722).

Fan et al. (1998) A human’s willingness to accept direction from an automated system. 

Jeannot et al. (2003) “Trust is a result of many factors such as reliability of the system and

transparency of the functions. Neither mistrust nor complacency are

desirable” (p. 4).

Lee and See (2004) “Trust can be defined as the attitude that an agent will help achieve an 

individual’s goals in a situation characterized by uncertainty and 

vulnerability” (p. 51).

Luhmann (1979) Mechanism to reduce feelings of uncertainty via a continuous feedback 

loop the person monitors the behavior of the trustee to determine whether 

the trust vested in the person is justified.

Madhavan and Wiegmann (2007) “Trust refers to the expectation of, or confidence in, another and is based 

on the probability that one party attaches to co-operative or favourable 

behaviour by other parties (Barber 1983, Muir 1987, Hwang and 

Buergers 1997)” (p. 280).

Madsen and Gregor (2000) Comprised of the reliability of the system, perceived technical 

competence of the system, perceived understandability of the system,

faith into the system and personal attachment to the system.

Madsen and Gregor (2000) Human-computer trust is defined in this study to be, the extent to which a 

user is confident in, and willing to act on the basis of the 

recommendations, actions, and decisions of an artificially intelligent 

agent.

Master et al. (2000) Trust model incorporates four dimensions: competence, predictability,

reliability, and faith in a system or machine. 

Mayer et al. (1995) “[W]illingness of a party to be vulnerable to the actions of another party 

based on the expectation that the other will perform a particular action 

important to the trustor, irrespective of the ability to monitor or control 

that party” (p. 712).

Note: References marked with a  represent definitions that are originally from another domain of trust but are referenced or 

applied to automation.
48

Definition Citation Definition

Merriam-Webster Online (2013) 1) a: assured reliance on the character, ability, strength, or truth of someone 

or something b: one in which confidence is placed 

2) a: dependence on something future or contingent : hope b: reliance on 

future payment for property (as merchandise) delivered : credit <bought 

furniture on trust>

3) a: a property interest held by one person for the benefit of another b: a 

combination of firms or corporations formed by a legal agreement,

especially : one that reduces or threatens to reduce competition 

4) archaic: trustworthiness

5) a: (1) a charge or duty imposed in faith or confidence or as a condition of 

some relationship (2) something committed or entrusted to one to be used 

or cared for in the interest of another b: responsible charge or office c: care,

custody <the child committed to her trust>

— in trust: in the care or possession of a trustee

Merrit and Ilgen (2008) “Dispositional trust reflects trust in other persons (or machines) upon

initially encountering them, even if no interaction has yet taken place” (p. 

195). 

Merrit and Ilgen (2008) “... history-based trust is founded on interactions between the person and 

another person or machine” (p. 195).

Merrit and Ilgen (2008) “... the stable, trait-like tendency to trust or not trust others, which we call

propensity to trust” (p. 195). 

Moorman et al. (1993) “Trust is defined as a willingness to rely on an exchange partner in whom 

one has confidence (Moorman, Zaltman, and Deshpande, 1992)” (p. 82).

Moray and Inagaki (1999) “... an attitude which includes the belief that the collaborator will perform 

as expected, and can, within the limits of its designers’ intentions, be relied 

on to achieve the design goals” (p. 204).

Muir (1989) The dynamic nature of trust consist of three dimensions: predictability,

dependability, and faith

Muir (1994) “...an intervening variable that mediates between the automated system and 

the user’s choice of a control strategy in his/her interaction with the system” 

Parasuraman and Miller (2004) “In practice, user trust should be calibrated to the system and context of its 

use, users should have accurate beliefs about the reliability of automation 

(and about their own capabilities)” (p. 52).

Rajaonah et al. (2006) “We define trust as a psychological state resulting from knowledge, beliefs,

and assessments (Castelfranche & Falcone, 2000) related to the decision 

making situation, that creates confident expectations (Corritore, Kracher,

and Wiedenbeck, 2003) for human machine system performance and guides 

operator reliance on automation (Lee & See, 2004)” (p. 102).

Rempel et al. (1985) “... A generalized expectation related to the subjective probability an 

individual assigns to the occurrence of some set of future events” (p. 96). 

Note: References marked with a  represent definitions that are originally from another domain of trust but are referenced or 

applied to automation.
49

Definition Citation Definition

Rempel and Holmes (1986) “In a general sense, trust is the degree of confidence you feel when you 

think about a relationship” (p. 28).

Ross (2008) “To measure objective trust of an automated system, reliance was analyzed. 

Reliance was measured as the combined total of the times the participant 

agreed with the aid” (p. 38).

Rotter (1967) “A generalized expectancy held by an individual that the word, promise,

oral or written statement of another individual or group can be relied upon” 

(p. 653).

Scanzoni (1979) An actor’s willingness to arrange and repose his or her activities on [an] 

Other because of confidence that [the] Other will provide expected 

gratifications. 

Singh et al. (1991) Complacency is a set of attitudes involving uncritical trust towards, over 

reliance on, excessive confidence in, and other related attitudes towards 

automation technology. 

Swadesh and Kasouf (1995) “Trust is related to a number of characteristics of the researcher: sincerity,

integrity, tactfulness, timeliness and ability to keep information 

confidential” (p. 199).

Note: References marked with a  represent definitions that are originally from another domain of trust but are referenced or 

applied to automation. 
50

INTENTIONALLY LEFT BLANK.
51

Appendix C. References of Experimental and Correlational Articles
52

* Abe, G.; Richardson, J. The Effect of Alarm Timing on Driver Behavior: An Investigation of 

Differencesin Driver Trust and Response to Alarms According to Alarm Timing. 

Transportation Research Part F: Traffic Psychology and Behaviour 2004, 7 (4/5), 307–322. 

doi:10.1016/j.trf.2004.09.008

*Abe, G.; Richardson, J. The Influence of Alarm Timing on Braking Response and Driver Trust 

in Low Speed Driving. Safety Science 2005, 43 (9), 639–654. doi:10.1016/j.ssci.2005.04.006

+Abe, G.; Richardson, J. The Influence of Alarm Timing on Driver Response to Collision 

Warning Systems Following System Failure. Behavior & Information Technology 2006, 25

(5), 443–452. doi:10.1080/01449290500167824

*+Bagheri, N.; Jamieson, G. Considering Subjective Trust and Monitoring Behavior in 

Assessing Automation-Induced “Complacency.” Proceedings of the Human Performance,

Situation Awareness, and Automation Conference, Marietta, GA, 2004. http://www.dtic.mil 

/cgi-bin/GetTRDoc?AD=ADA426023#page=64 (accessed 29 May 2014).

*+Bagheri, N. The Effect of Automation Reliability on Operator Monitoring Performance. 

Master’s Thesis, University of Toronto, Toronto, Ontario, Canada, 2004.

*Bliss, J. P.; Acton, S. A. Alarm Mistrust in Automobiles: How Collision Alarm Reliability 

Effects Driving. Applied Ergonomics 2003, 34 (6), 499–509. 

doi:10.1016/j.apergo.2003.07.003

*Cai, H.; Lin, Y. Coordinating Cognitive Assistance with Cognitive Engagement Control 

Approaches in Human-Machine Collaboration. IEEE Transactions on Systems, Man, and 

Cybernetics – Part A: Systems and Humans 2012, 42 (2), 286–294.

*Cai, H.; Lin, Y.; Cheng, B. Coordinating Multi-Level Cognitive Assistance and Incorporating 

Dynamic Confidence Information in Driver-Vehicle Interfaces. Human Factors and 

Ergonomics in Manufacturing & Service Industries 2012, 22 (5), 437–449. 

doi:10.1002/hfm.20399

+Cummings, M. L.; Buchin, M.; Carrigan, G.; Donmez, B. Supporting Intelligent and 

Trustworthy Maritime Path Planning Decisions. International Journal of Human-Computer 

Studies 2010, 68 (10), 616–626. doi:10.1016/j.ijhcs.2010.05.002

*de Vries, P.; Midden, C.; Bouwhuis, D. The Effects of Errors on System Trust, SelfConfidence, and the Allocation of Control in Route Planning. International Journal of 

Human-Computer Studies 2003, 58 (6), 719–735. doi:10.1016/S1071-5819(03)00039-9

*de Vries, P.; Midden, C. Effect of Indirect Information on System Trust and Control Allocation. 

Behavior & Information Technology 2008, 27 (1), 17–29. doi:10.1080/01449290600874956

 

 * represents experimental and + represents correlation findings.
53

*+Donmez, B.; Boyle, L. N.; Lee, J. D.; McGehee, D. V. Drivers’ Attitudes Toward Imperfect 

Distraction Mitigation Strategies. Transportation Research Part F 2008, 9 (6), 387–398.

doi:10.1016/j.trf.2006.02.001

+Dzindolet, M. T.; Peterson, S. A.; Pomranky, R. A.; Pierce, L. G.; Beck, H. P. The Role of 

Trust in Automation Reliance. International Journal of Human-Computer Studies 2003, 58

(6), 697–710. doi:10.1016/S1071-5819(03)00038-7

*Gupta, N.; Bisantz, A. M.; Singh, T. The Effects of Adverse Condition Warning System 

Characteristics on Driver Performance: An Investigation of Alarm Signal Type on Threshold 

Level. Behaviors & Information Technology 2002, 21 (4), 235–248. 

doi:10.1080/0144929021000013473

*Ho, G.; Wheatley, D.; Scialfa, C. T. Age Differences in Trust and Reliance of a Medication 

Management System. Interacting With Computers 2005, 17 (6), 690–710. 

doi:10.1016/j.intcom.2005.09.007

*Kazi, T.; Stanton, N.; Walker, G.; Young, M. Designer Driving: Drivers’ Conceptual Models 

and Level of Trust in Adaptive-Cruise-Control. International Journal of Vehicle Design

2007, 45 (3), 339–360. doi:10.1504/IJVD.2007.014909

*Koh, Y. J.; Sundar, S. S. Effects of Specialization in Computer, Web Sites, and Web Agents on 

E-Commerce Trust. International Journal of Human-Computer Studies 2010, 68 (12),

899–912. doi:10.1016/j.ijhcs.2010.08.002

*Koustanai, A.; Cavallo, V.; Dalhomme, P.; Mas, A. Simulator Training With a Forward 

Collision Warning System: Effects on Driver-System Interactions and Driver Trust. Human 

Factors: The Journal of the Human Factors and Ergonomics Society 2012, 54, 709–721. 

doi:10.1177/0018720812441796

+Lee, K.; Chung, N. Understanding Factors Affecting Trust in and Satisfaction With Mobile 

Banking In Korea: A Modified DeLone and McLean’s Model Perspective. Interacting With 

Computers 2009, 21 (5/6), 385–392. doi:10.1016/j.intcom.2009.06.004

+Lee, H.; Kim, J.; Kim, J. Determinants of Success for Applications Service Provider: An 

Empirical Test in Small Businesses. International Journal of Human-Computer Studies

2007, 65 (9), 796–815. doi:10.1016/j.ijhcs.2007.04.004

*Lorenz, B.; Di Nocera, F.; Rottger, S.; Parasuraman, R. Automated Fault-Management in a 

Simulated Spaceflight Micro-World. Aviation, Space, and Environmental Medicine 2002, 73

(9), 886–897. http://archlab.gmu.edu/people/rparasur/Documents/LorenzASEM02.pdf

(accessed 29 May 2014).
54

*Ma, R.; Kaber, D. B. Effects of in-Vehicle Navigation Assistance and Performance on Driver 

Trust And Vehicle Control. International Journal of Industrial Ergonomics 2007, 37 (8),

665–673. doi:10.1016/j.ergon.2007.04.005

*Madhavan, P.; Wiemann, D. A. Effects of Information Source, Pedigree, and Reliability on 

Operator Interaction With Decision Support Systems. The Journal of Human Factors and 

Ergonomics Society 2007, 49 (5), 773–785. doi:10.1518/001872007X230154

*Madhavan, P.; Wiegmann, D. A.; Lacson, F. C. Automation Failures on Tasks Easily 

Performed by Operators Undermine Trust in Automated Aids. The Journal of Human 

Factors and Ergonomics Society 2006, 48 (2), 241–256. doi:10.1518/001872006777724408

*Manzey, D.; Rottger, S.; Bahner-Heyne, J. E.; Schulze-Kissing, D.; Dietz, A.; Meixensberger,

J.; Strauss, G. Image-Guided Navigation: The Surgeon’s Perspective on Performance 

Consequences and Human Factors Issues. The Journal of Medical Robotics and Computer 

Assisted Surgery 2009, 5, 297–308. doi:10.1002/rcs.261

*Merritt, S. M.; Heimbaugh, H.; LaChapell, J.; Lee, D. I Trust It, But I Don’t Know Why: 

Effects of Implicit Attitudes Toward Automation on Trust in an Automated System. Human 

Factors: The Journal of the Human Factors and Ergonomics Society, 2012.

doi:10.1177/0018720812465081

+Merritt, S. M.; Ilgen, D. R. Not All Trust is Created Equal: Dispositional and History-Based 

Trust in Human-Automation Interactions. The Journal of Human Factors and Ergonomics 

Society 2008, 50 (2), 194–210. doi:10.1518/001872008X288574

*Moray, N.; Inagaki, T.; Itoh, M. Adaptive automation, Trust, and Self-Confidence in Fault 

Management of Time-Critical Tasks. Journal of Experimental Psychology Applied 2000, 61

(1), 44–58. doi:10.1037/1076-898X.6.1.44

+Muir, B. M.; Moray, N. Trust in Automation Part II. Experimental Studies of Trust and Human 

Intervention in a Process Control Simulation. Ergonomics 1996, 39 (3), 429–460. 

doi:10.1080/00140139608964474

*+Neyedli, H. F.; Hollands, J. G.; Jamieson, G. A. Human Reliance on an Automated Combat 

ID System: Effects of Display Format. Proceedings of the 53rd Human Factors and 

Ergonomics Society Annual Meeting 2009, 53 (4), pp 212–216.

doi:10.1177/154193120905300411

*Pak, R.; Fink, N.; Price, M.; Bass, B.; Sturre, L. Decision Support Aids With Anthropomorphic 

Characteristics Influence Trust and Performance in Younger and Older Adults. Ergonomics

2012, 55 (9), 1059–1072. doi:10.1080/00140139.2012.691554

*+Rajaonah, B.; Anceaux, F.; Vienne, F. Study of Driver Trust During Cooperation With 

Adaptive Cruise Control. Travail Humain 2006, 69 (2), 99–127.
55

+Rajaonah, B.; Tricot, N.; Anceaux, F.; Millot, P. The Role of Intervening Variables in DriverACC Cooperation. International Journal of Human-Computer Studies 2008, 66 (3),

185–197. doi:10.1016/j.ijhcs.2007.09.002

+Rani, M. R.; Sinclair, M. A.; Case, K. Human Mismatches and Preferences for Automation. 

International Journal of Production Research 2000, 38 (17), 4033–4039. 

doi:10.1080/00207540050204894

*Rovira, E.; Parasuraman, R. Transitioning to Future Air Traffic Management: Effects of 

Imperfect Automation on Controller Attention and Performance. The Journal of Human 

Factors and Ergonomics 2010, 52 (3), 411–425. doi:10.1177/0018720810375692

*Smyth, C. C. Sensitivity of Subjective Questionnaires to Cognitive Loading With Driving With 

Navigation Aids: A Pilot Study. Aviation, Space, and Environmental Medicine 2007, 78 (5),

B39–B50.

*+Spain, R. D.; Bliss, J. P. The effect of Sonification Display Pulse Rate and Reliability on 

Operator Trust and Perceived Workload During a Simulated Patient Monitoring Task. 

Ergonomics 2008, 51 (9), 1320–1337. doi:10.1080/00140130802120234

*Stedmon, A. W.; Sharples, S.; Littlewood, R.; Cox, G.; Patel, H.; Wilson, J. R. Datalink in air 

Traffic Management: Human Factors Issues in Communications. Applied Ergonomics 2007,

38 (4), 473–480. doi:10.1016/j.apergo.2007.01.013

+Strauss, G.; Koulechov, K.; Röttger, S.; Bahner, J.; Trantakis, C.; Hofer, M.; Lűth, T.

Evaluation of a Navigation System for ENT With Surgical Efficiency Criteria. The 

Laryngoscope 2006, 116 (4), 564–572. doi:10.1097/01.MLG.0000202091.34295.05

*Verberne, F. M. F.; Ham, J.; Midden, C. J. H. Trust in Smart Systems: Sharing Driving Goals 

and Giving Information to Increase Trustworthiness and Acceptability of Smart Systems in 

Cars. The Journal of the Human Factors and Ergonomics Society 2012, 54, 799–810. doi: 

10.1177/0018720812443825

+Wang, L.; Jamieson, G. A.; Hollands, J. G. Trust and Reliance on an Automated Combat 

Identification System. The Journal of Human Factors and Ergonomics 2009, 51 (3),

281–291. doi:10.1177/0018720809338842

+Wang, L.; Jamieson, G. A.; Hollands, J. G. The Effects of Design Features on Users’ Trust in 

and Reliance on a Combat Identification System. Proceedings of the 55th Annual Human 

Factors and Ergonomics Society 2011, 55 (1), 375–379. doi: 10.1177/107118131551077
56

INTENTIONALLY LEFT BLANK.
57

Appendix D. Organization Results
58

Preference for Human or Automation Assistance (Section 5.1)

De Vries et al. (2003)

Ma and Kaber (2007)

Madhavan and Wiegmann (2007) 

Task-Related Automation (Section 5.2)

Empirical Correlation

Cognitive Aids Bagheri (2004)

Bagheri and Jamieson (2004)

Cai and Lin (2012)

Cai et al. (2012)

De Vries and Midden (2008)

De Vries et al. (2003)

Ho et al. (2005)

Koh and Sundar (2010)

Madhavan and Wiegmann (2007)

Madhavan et al. (2006)

Merritt et al. (2012)

Neyedli et al. (2009)

Pak et al. (2012)

Bagheri and Jamieson (2004)

Bagheri (2004)

Cummings et al. (2010)

Dzindolet et al. (2003)

Lee and Chung (2009)

Lee et al. (2007)

Merritt and Ilgen (2008)

Neyedli et al. (2009)

Wang et al. (2009)

Wang et al. (2011)

Perceptual Aids Abe and Richardson (2004)

Abe and Richardson (2005)

Bliss and Acton (2003)

Gupta et al. (2002)

Koustanai et al. (2012)

Lorenz et al. (2002)

Abe and Richardson (2006)

Control Aids Donmez et al. (2006)

Manzey et al. (2009)

Moray et al. (2000)

Rajaonah (2006)

Smyth (2007)

Spain and Bliss (2008)

Stedmon et al. (2007)

Verberne et al. (2012)

Donmez et al. (2006)

Muir and Moray, 1996

Rajaonah (2006)

Rajaonah et al. (2008)

Spain and Bliss (2008)
59

Human-Related Factors (Section 5.3)

Empirical Correlation

Age Donmez et al. (2006)

Ho et al. (2005)

Pak et al. (2012)

Merritt and Ilgen (2008)

Cognitive Factors De Vries and Midden (2008)

Koustanai et al. (2012)

Manzey et al. (2009)

Rajaonah (2006)

Dzindolet et al. (2003)

Lee et al. (2007)

Merritt and Ilgen (2008)

Rajaonah et al. (2008)

Spain and Bliss (2008)

Wang et al. (2011)

Emotive Factors Koustanai et al. (2012)

Neyedli et al. (2009)

Verberne et al. (2012)

Donmez et al. (2006)

Lee and Chung (2009)

Lee et al. (2007)

Merritt and Ilgen (2008)

Wang et al. (2009)

Wang et al. (2011)
60

Automation-Related Factors (Section 5.4)

Empirical Correlation

Features

Cai and Lin (2012)

Cai et al. (2012)

Lorenz et al. (2002)

Merritt et al. (2012)

Moray et al. (2000)

Pak et al. (2012)

Stedmon et al. (2007)

Verberne et al. (2012)

Dzindolet et al. (2003)

Lee and Chung (2009)

Capabilities

Abe and Richardson (2004)

Abe and Richardson (2005)

Bagheri (2004)

Bagheri and Jamieson (2004)

Bliss and Acton (2003)

De Vries et al. (2003)

Donmez et al. (2006)

Gupta et al. (2002)

Ho et al. (2006)

Koh and Sundar (2010)

Madhavan and Wiegmann (2007)

Madhavan et al. (2006)

Moray et al. (2000)

Smyth (2007)

Spain and Bliss (2008)

Stedmon et al. (2007)

Abe and Richardson (2006)

Bagheri and Jamieson (2004)

Bagheri (2004)

Dzindolet et al. (2003)

Lee and Chung (2009)

Lee et al. (2007)

Muir and Moray (1996)

Rajaonah (2006)
61

1 DEFENSE TECHNICAL

(PDF) INFORMATION CTR

DTIC OCA

2 DIRECTOR

(PDF) US ARMY RESEARCH LAB

RDRL CIO LL

IMAL HRA MAIL & RECORDS MGMT

1 GOVT PRINTG OFC

(PDF) A MALHOTRA

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM D

T DAVIS

BLDG 5400 RM C242

REDSTONE ARSENAL AL 35898-7290

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRS EA DR V J RICE

BLDG 4011 RM 217

1750 GREELEY RD

FORT SAM HOUSTON TX 78234-5002

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM DG J RUBINSTEIN

BLDG 333

PICATINNY ARSENAL NJ 07806-5000

1 ARMY RSCH LABORATORY – HRED

(PDF) ARMC FIELD ELEMENT

RDRL HRM CH C BURNS

THIRD AVE BLDG 1467B RM 336

FORT KNOX KY 40121

1 ARMY RSCH LABORATORY – HRED

(PDF) AWC FIELD ELEMENT

RDRL HRM DJ D DURBIN

BLDG 4506 (DCD) RM 107

FORT RUCKER AL 36362-5000 

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM CK J REINHART

10125 KINGMAN RD BLDG 317

FORT BELVOIR VA 22060-5828

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM AY M BARNES

2520 HEALY AVE 

STE 1172 BLDG 51005

FORT HUACHUCA AZ 85613-7069

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM AP D UNGVARSKY

POPE HALL BLDG 470 

BCBL 806 HARRISON DR

FORT LEAVENWORTH KS 66027-2302

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM AT J CHEN

12423 RESEARCH PKWY

ORLANDO FL 32826-3276

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM AT C KORTENHAUS

12350 RESEARCH PKWY

ORLANDO FL 32826-3276 

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM CU B LUTAS-SPENCER

6501 E 11 MILE RD MS 284

BLDG 200A 2ND FL RM 2104

WARREN MI 48397-5000

1 ARMY RSCH LABORATORY – HRED

(PDF) FIRES CTR OF EXCELLENCE 

FIELD ELEMENT

RDRL HRM AF C HERNANDEZ

3040 NW AUSTIN RD RM 221

FORT SILL OK 73503-9043

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM AV W CULBERTSON

91012 STATION AVE 

FORT HOOD TX 76544-5073

1 ARMY RSCH LABORATORY – HRED

(PDF) HUMAN RSRCH AND ENGRNG 

DIRCTRT MCOE FIELD ELEMENT

RDRL HRM DW C CARSTENS

6450 WAY ST

BLDG 2839 RM 310

FORT BENNING GA 31905-5400

1 ARMY RSCH LABORATORY – HRED

(PDF) RDRL HRM DE A MARES

1733 PLEASONTON RD BOX 3

FORT BLISS TX 79916-6816
62

8 ARMY RSCH LABORATORY – HRED

(PDF) SIMULATION & TRAINING

TECHNOLOGY CENTER

RDRL HRT COL M CLARKE

RDRL HRT I MARTINEZ

RDRL HRT T R SOTTILARE

RDRL HRT B N FINKELSTEIN

RDRL HRT G A RODRIGUEZ

RDRL HRT I J HART

RDRL HRT M C METEVIER

RDRL HRT S B PETTIT

12423 RESEARCH PARKWAY

ORLANDO FL 32826

1 ARMY RSCH LABORATORY – HRED

(PDF) HQ USASOC

RDRL HRM CN R SPENCER

BLDG E2929 DESERT STORM DRIVE

FORT BRAGG NC 28310

1 ARMY G1

(PDF) DAPE MR B KNAPP

300 ARMY PENTAGON RM 2C489

WASHINGTON DC 20310-0300

2 ROBOTICS CTA

(PDF) J BORNSTEIN

S HILL

ABERDEEN PROVING GROUND

12 DIR USARL

(PDF) RDRL HR

L ALLENDER

P FRANASZCZUK

RDRL HRM

P SAVAGE-KNEPSHIELD

RDRL HRM AL

C PAULILLO

RDRL HRM B

J GRYNOVICKI

RDRL HRM C

L GARRETT

RDRL HRS

J LOCKETT

RDRL HRS B

M LAFIANDRA

RDRL HRS C

K MCDOWELL

RDRL HRS D

B AMREIN

RDRL HRS E

D HEADLEY

K SCHAEFER