U.S. Department 

of Transportation

Federal Railroad 

Administration

Effects of Supervisory Train Control 

Technology on Operator Attention

Office of Research 

and Development 

Washington, DC 20590

U.S. Department of Transportation 

Research and Special Programs Administration 

John A. Volpe National Transportation Systems Center

Cambridge, MA 02142

Human Factors in Railroad Operations

DOT/FRA/ORD-04/10 Final Report 

July 2005 

This document is available to the 

public through the National Technical 

 Information Service, Springfield, VA 22161 

This document is also available on

 the FRA website at www.fra.dot.gov 
 

Notice 

This document is disseminated under the sponsorship of

the Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for its contents or use thereof. 

Notice 

The United States Government does not endorse

products or manufacturers. Trade or manufacturers’ 

names appear herein solely because they are 

considered essential to the objective of this report. 
 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and

maintaining the data needed and completing and reviewing the collection of information. Send comments regarding this burden estimate or any aspects of this collection of information , including

suggestions for reducing this burden to Washington Headquarters Service, Directorate for information Operations and Reports. 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA. 222202-

4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503. 

1. AGENCY USE ONLY (LEAVE BLANK) 2. REPORT DATE 

July 2005 

3. REPORT TYPE AND DATES COVERED 

Final Report 

September 1995 - June1997

4. TITLE AND SUBTITLE 

Effects of Supervisory Train Control Technology on Operator Attention

5. FUNDING NUMBERS 

R2103/RR204 

6. AUTHOR(S) 

Helias Marinakos, Thomas Sheridan, and Jordan Multer 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Department of Transportation 

Research and Special Programs Administration 

John A. Volpe National Transportation Systems Center 

Cambridge, MA 02142-1093

8. PERFORMING ORGANIZATION

DOT-VNTSC-FRA-04-07

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

U.S. Department of Transportation 

Federal Railroad Administration 

Office of Research and Development Mail Stop 20 

1120 Vermont Avenue, NW

Washington, DC 20590

10. SPONSORING/MONITORING

AGENCY REPORT NUMBER

DOT/FRA/ORD-04/10

11. SUPPLEMENTARY NOTES 

12a. DISTRIBUTION/AVAILABILITY STATEMENT 

This document is available to the public through the National Technical Information

Service, Springfield, VA 22161. This document is also available on the FRA web site at 

www.fra.dot.gov. 

12b. DISTRIBUTION CODE 

13. ABSTRACT (Maximum 200 words) 

This report describes an experiment evaluating the effects of supervisory control automation on attention allocation while operating

a train. The study compared two levels of supervisory control (partial and full) to manual control, in terms of how it affects 

vigilance detection and situation awareness. Human performance was measured using a human-in-the-loop train simulator. To

evaluate vigilance, participants were asked to detect two types of automation failures and react to obstructions on the track. 

Situation awareness was measured using the Situation Awareness Global Assessment Technique (SAGAT) in which the simulation 

was suspended at periodic intervals and the subjects answered questions about the system. These answers were compared to 

objective measures of system performance. 

 Attention allocation varied with the method by which supervisory control was implemented. In particular, attention allocation for

speed control, a critical piece of information, varied with the two methods. Partial supervisory control, as implemented in this 

experiment, resulted in a narrowing of attention. The primary focus was on speed information. By contrast, use of full supervisory 

control resulted in participants spreading their attention more broadly, but at the expense of speed information.

14. SUBJECT TERMS 

attention allocation, automation, high-speed trains, situation awareness, train control technology, 

supervisory control, vigilance

15. NUMBER OF PAGES 

52 

16. PRICE CODE 

17. SECURITY CLASSIFICATION 

OF REPORT

Unclassified 

18. SECURITY CLASSIFICATION 

OF THIS PAGE 

Unclassified 

19. SECURITY CLASSIFICATION 

OF ABSTRACT 

Unclassified 

 LIMITATION OF ABSTRACT 

Unclassified 

NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18

298-102
 

METRIC/ENGLISH CONVERSION FACTORS 

ENGLISH TO METRIC METRIC TO ENGLISH

LENGTH (APPROXIMATE) LENGTH (APPROXIMATE) 

1 inch (in) = 2.5 centimeters (cm) 1 millimeter (mm) = 0.04 inch (in) 

1 foot (ft) = 30 centimeters (cm) 1 centimeter (cm) = 0.4 inch (in)

1 yard (yd) = 0.9 meter (m) 1 meter (m) = 3.3 feet (ft) 

1 mile (mi) = 1.6 kilometers (km) 1 meter (m) = 1.1 yards (yd)

1 kilometer (km) = 0.6 mile (mi)

AREA (APPROXIMATE) AREA (APPROXIMATE)

1 square inch (sq in, in2) = 6.5 square centimeters (cm2) 1 square centimeter (cm2) = 0.16 square inch (sq in, in2) 

1 square foot (sq ft, ft2) = 0.09 square meter (m2) 1 square meter (m2) = 1.2 square yards (sq yd, yd2) 

1 square yard (sq yd, yd2) = 0.8 square meter (m2) 1 square kilometer (km2) = 0.4 square mile (sq mi, mi2) 

1 square mile (sq mi, mi2) = 2.6 square kilometers (km2) 10,000 square meters (m2) = 1 hectare (ha) = 2.5 acres 

1 acre = 0.4 hectare (he) = 4,000 square meters (m2) 

MASS - WEIGHT (APPROXIMATE) MASS - WEIGHT (APPROXIMATE)

1 ounce (oz) = 28 grams (gm) 1 gram (gm) = 0.036 ounce (oz)

1 pound (lb) = 0.45 kilogram (kg) 1 kilogram (kg) = 2.2 pounds (lb) 

1 short ton = 2,000 

pounds (lb)

= 0.9 tonne (t) 1 tonne (t) =

=

1,000 kilograms (kg) 

1.1 short tons

VOLUME (APPROXIMATE) VOLUME (APPROXIMATE) 

1 teaspoon (tsp) = 5 milliliters (ml) 1 milliliter (ml) = 0.03 fluid ounce (fl oz) 

1 tablespoon (tbsp) = 15 milliliters (ml) 1 liter (l) = 2.1 pints (pt) 

1 fluid ounce (fl oz) = 30 milliliters (ml) 1 liter (l) = 1.06 quarts (qt) 

1 cup (c) = 0.24 liter (l) 1 liter (l) = 0.26 gallon (gal)

1 pint (pt) = 0.47 liter (l) 

 1 quart (qt) = 0.96 liter (l) 

1 gallon (gal) = 3.8 liters (l) 

1 cubic foot (cu ft, ft

3

) = 0.03 cubic meter (m3) 1 cubic meter (m3) = 36 cubic feet (cu ft, ft3) 

1 cubic yard (cu yd, yd3) = 0.76 cubic meter (m3) 1 cubic meter (m3) = 1.3 cubic yards (cu yd, yd3)

TEMPERATURE (EXACT) TEMPERATURE (EXACT) 

[(x-32)(5/9)] °F = y °C [(9/5) y + 32] °C = x °F

QUICK INCH - CENTIMETER LENGTH CONVERSION

0 1 2 3 4

Inches

Centimeters 0 1 2 6 3 4 5 7 1 8 9 10 11 12

5

3

QUICK FAHRENHEIT - CELSIUS TEMPERATURE CONVERSION

 -40° -22° -4° 14° 32° 50° 68° 86° 104° 122° 140° 158° 176° 194° 212°

 

°F

 °C -40° -30° -20° -10° 0° 10° 20° 30° 40° 50° 60° 70° 80° 90° 100°

For more exact and or other conversion factors, see NIST Miscellaneous Publication 286, Units of Weights and 

Measures. Price $2.50 SD Catalog No. C13 10286 Updated 6/17/98

ii
 

ACKNOWLEDGMENTS 

Increasing train speed makes it more difficult for the train crew to see the wayside signals and 

respond in an appropriate and timely manner. As train speed increases, less time is available for 

the locomotive engineer to react, due to the increased distance required for the vehicle to 

decelerate. Train control technology (i.e., positive train control) has been proposed as a way to 

compensate for the physical and perceptual limitations of the operator. One type of train control 

that has been proposed is supervisory control. In supervisory control, the train control system

operates the train and the locomotive engineer monitors the train control system. However, 

studies of supervisory control show that it has negative as well as positive effects on human 

performance. 

This study examined the impact of supervisory train control on two aspects of attention, 

vigilance, and situation awareness. High-speed train operations were simulated in a human-inthe-loop train simulator. Two kinds of supervisory train control: full and partial were compared 

to manual control. To evaluate attention allocation in train handling, participants operated a train 

during which equipment failures occurred at very low frequencies and vehicles obstructed the 

track. The ability of participants to respond to these events was measured. Situation awareness 

was measured by suspending the simulation periodically to obtain information from the 

participant regarding his or her knowledge related to train handling. The participant’s answers 

were compared to objective information about the locomotive’s state. 

The study documented in this report was part of a program to examine human factors issues 

related to the introduction of new train control technology in high-speed rail operations. The 

Federal Railroad Administration sponsored this work as part of its activities to develop 

Intelligent Railroad Systems (Federal Railroad Administration, 2002). 

The authors would like to acknowledge the contribution of many people who were part of this 

research: 

It was a pleasure working with Steven Villareal who developed the sound enhancement for the 

High-Speed Train Simulator and Jacob Einhorn who worked on in-cab displays for high-speed 

locomotives. Special thanks go to Dr. Edward Lanzilotta for providing his knowledge regarding 

software engineering issues and experimental procedures. Additionally, support was provided in 

various forms from the other MIT colleagues as well: Shinsuk Park, Mark Ottensmeyer, 

Nicholas Patrick, Suyeong Kim, Dave Schloerb, Shih-Ken Chen, Jie Ren, Steven Landry, Joe 

Conti, Shumei Yin Askey, Bernardo Aumond, and Michalis Kilaras. 

This work was completed at the John A. Volpe National Transportation Systems Center. The 

opportunity to work in this facility led to many productive relationships. Thanks are due to John 

K. Pollard for his involvement in the project and for providing his hardware expertise and to Dr. 

Donald Sussman for his support and interest in this research. Finally, the authors would like to 

thank Garold Thomas and Dr. Thomas Raslear from the Office of Research and Development at 

Federal Railroad Administration for sponsoring this research. 

iii
 

iv
 

TABLE OF CONTENTS 

EXECUTIVE SUMMARY ........................................................................................................ vii

1. INTRODUCTION......................................................................................................................1

Vigilance ..............................................................................................................................2 

Situation Awareness.............................................................................................................3 

2. METHOD ...................................................................................................................................5

Overview..............................................................................................................................5 

Experimental Design............................................................................................................5 

Participants...........................................................................................................................8 

Facilities: High-Speed Locomotive Simulator.....................................................................8 

Procedures..........................................................................................................................10 

3. RESULTS AND DISCUSSION ..............................................................................................13

Vigilance ............................................................................................................................13 

Situation Awareness...........................................................................................................18 

Summary............................................................................................................................22 

APPENDIX A. BONUS AND PENALTY POINT SYSTEM ..................................................25

APPENDIX B. NEW FEATURES OF HIGH-SPEED TRAIN SIMULATOR.....................29

APPENDIX C. LOCATION OF FAILURE EVENTS.............................................................31

APPENDIX D. VIGILANCE DETECTION TASK DATA.....................................................33

APPENDIX E. DATA FOR PAGE TEST.................................................................................37

REFERENCES.............................................................................................................................39

v
 

LIST OF FIGURES 

Figure Page

Figure 1. Train Simulator Layout ....................................................................................................8 

Figure 2. Instrument Panel...............................................................................................................9 

Figure 3. Track Layout ..................................................................................................................10 

Figure 4. Four Outcomes of Signal Detection Theory...................................................................13 

Figure 5. Graphical Depiction of Signal Detection Theory...........................................................14 

Figure 6. Effects of Train Control Mode on Detection Rate .........................................................15 

Figure 7. Probability of Hits vs. False Alarms by Train Control Mode and Time (The labels: 1, 2, 

and 3 represent the time block order).....................................................................................16 

Figure 8. Sensitivity and Response Criterion by Time Block and Control Mode .........................17 

Figure 9. Distance Stopped from an Obstruction..........................................................................18 

Figure 10. Effects of Train Control Mode on Speed Awareness...................................................19 

Figure 11. Effects of Train Control Mode on Bearing Temperature Awareness...........................21 

Figure 12. Effects of Train Control Mode on Position and Motor Condition ...............................21 

Figure A-1. Schedule Accuracy Bonus/Penalty Points..................................................................25 

Figure A-2. Emergency Response Time Bonus Points..................................................................26 

LIST OF TABLES 

Table Page

1. Train Control Condition.............................................................................................................. 5 

2. Presentation Order of Train Control Modes............................................................................... 6 

3. Detection of In-Cab Equipment Failures by Control Mode...................................................... 15 

4. Participant Preferences for Train Control Mode from Exit Interview...................................... 22 

A-1. Penalties Resulting from Application of the Emergency Brakes ........................................ 26 

A-2. Obstruction Hazards Bonus/Penalty Scheme ....................................................................... 26 

vi
 

EXECUTIVE SUMMARY 

As high-speed passenger operations are adopted in the United States, train control technology is 

required to enable the train crew to operate the train safely. As train speeds increase, the 

locomotive engineer has less time to acquire information needed to operate the locomotive and 

respond. In high-speed rail operations, train control technology using decision support aids, 

supervisory control, or some combination has been proposed as a means to compensate for the 

sensory, perceptual, and cognitive limitations of the locomotive engineer. 

This study focused on the impact of supervisory train control technology on human performance. 

The successful application of supervisory control in train operations requires an understanding of 

how it affects the train crews that interact with this technology. For example, locomotive 

engineers often work long hours with inadequate amounts of rest between shifts (Thomas, 

Raslear, and Kuehn, 1997). As a result, engineers can become less attentive over the course of 

their shift. Supervisory train control technology has been proposed to assist the locomotive 

engineer in operating the locomotive safely during high speeds over long periods. 

However, supervisory train control may change how the locomotive engineer allocates his or her 

attention. The question is how do these changes affect the operator's performance and what are 

the safety implications of these changes? To control the train, the engineer collects information 

both inside and outside the cab. Inside the cab, the engineer monitors locomotive health by 

monitoring visual displays, listening to the sounds made by the locomotive and on-board 

equipment, and attends to kinesthetic cues (i.e., vibrations). Outside the cab, the engineer looks 

for hazards, monitors wayside signals, landmarks, and other visual cues needed to anticipate 

future train movements. The skills and knowledge needed to operate a train are built up over 

time as the engineer learns how to control the train and learns the physical attributes of the 

territory over which he will operate. Likewise, the engineer learns how to effectively allocate his 

attention between events inside and outside the cab so that the train operates safely and 

efficiently. 

How do the locomotive engineer's attention allocation strategies change when supervisory train 

control technology is introduced? What are the safety implications of these changes? For 

example, a key issue in supervisory control is whether the operator can detect train control 

failure and take over safe operation of the train. In relying on supervisory control, the engineer’s 

ability to intervene promptly in case of an unexpected event can deteriorate (Huey and Wickens, 

1993). How does supervisory control affect the engineer’s ability to detect unexpected events?

What happens when equipment unrelated to the train control system fails? Detecting a train 

control failure itself may also be a challenge for the operator. This study will address these 

questions using two paradigms: vigilance and situation awareness. Both concepts can be used to 

examine how shifts in attention affect human performance. 

Vigilance refers to the capacity of the human operator to sustain attention and remain alert to 

stimuli over a prolonged time. The most significant aspect of the vigilance decrement is that it 

arises from searching for a relatively infrequent signal over a prolonged period (Dember and 

Warm, 1979). 

Very few vigilance studies have been performed with a very low target rate on the order of one 

target per 30 minutes (Molloy and Parasuraman, 1996; Loeb and Binford, 1970). Lanzilotta 

(1995) examined the effects of supervisory train control on vigilance in a locomotive simulator 

vii
 

and found that operators show more variability in detecting train control failures with partial 

supervisory control than full supervisory control. He concluded that with partial supervisory 

control, observers exhibited a greater tendency to attend to events outside the locomotive cab 

than with full supervisory control. However, in this study the number of failures on each trip (6) 

may have affected the detection performance of the participant. What would happen in a system

in which the number of failures was smaller? The current study examined this question

measuring participant’s ability to detect a small number of failure events. Instead of six events 

per trip found in Lanzilotta's study, participants were exposed to only two events per trip. 

Situation awareness (SA) is a broader concept than vigilance. Endsley (1993) defines SA as 

“the perception of the elements of the environment within a volume of time and 

space, the comprehension of their meaning and the projection of their status in 

the near future.” 

In locomotives with supervisory train control, the engineer can become a passive processor of 

information. The engineer needs to know what the train control is doing as well as what the other 

components are doing. However, feedback in train control systems can be inadequate. This loss 

of information can make it difficult to detect train control failures and intervene when failures 

occur. 

In Lanzilotta's study examining the effect of supervisory train control on situation awareness, 

there was an inverse relationship between situation awareness and the level of supervisory 

control. As the level of supervisory control increased, situation awareness degraded. Perceived 

situation awareness was highest in the condition where the operator controlled the train manually 

and was lowest in the full supervisory control condition. However, in this study, situation 

awareness was measured by asking participants to rank the train control modes by their 

perceived level of situation awareness. This subjective measure may not give an accurate 

measure of situation awareness as the response may be affected by the outcome of the 

experimental trials. An objective method that compares an operator's knowledge of the system

while it is being operated with information about the actual state of the system may give a more 

representative picture of the level of situation awareness. The current study used such a method 

developed by Endsley (1995b) to evaluate the level of situation awareness for different kinds of 

train control. 

Three control modes for the operation of the train were evaluated: manual, partial supervisory, 

and full supervisory control. The three control modes represent the range of control modes found 

in current railroad operations. Under manual control, the human was solely responsible for 

controlling the speed of train, while in partial supervisory control the human operator set the 

desired speed (i.e., cruise control) and the train control system was responsible for achieving and 

maintaining it. Operation of the vehicle under full supervisory control normally required no 

human intervention, as the train control system was programmed to respond to the appropriate 

signals. 

A laboratory experiment, using a human-in-the-loop locomotive simulator, was conducted to 

examine the effects of three control modes on operator attention. A series of events both inside 

and outside the locomotive were presented to participants during the course of operating the train 

and the ability to detect these events was measured. The participant's task was to control the 

vehicle speed according to the signal and civil speed limits indicated in each block, look out for 

potential hazards, and respond to them in an appropriate manner. During the course of the 

viii
 

experiment, the simulation was interrupted and the participants were questioned about their 

knowledge of the locomotive's health and its track position. 

To measure vigilance, two types of equipment failure and an obstruction were introduced. The 

failures remained active for a set period, unless detected and reset by the participant. If not reset 

by the participant within the specified time interval, the failure was automatically reset. The 

amount of time to complete a round trip (45 minutes) was broken down into three equal time

blocks and the detection rate was measured in each block. The percentage of failures detected by 

the participants in each time block was used as a measure of vigilance (Molloy and Parasuraman, 

1996) inside the cab. Outside the cab, vigilance was measured by the distance from the

obstruction where the train stopped. To measure situation awareness, Endsley's Situation 

Awareness Global Assessment Technique (SAGAT) was used (1995b). In this method, the 

simulation was paused and the displays were blanked at periodic intervals. The participant was 

asked several questions about the status of the locomotive's operating condition and its position 

on the track.

Overall, supervisory train control systems improved situation awareness and vigilance 

performance compared to manual operation. However, differences in the way the supervisory 

control systems operated contributed to differences in attention allocation. These differences 

were greatest for unexpected events such as equipment failures and obstructions at highwayrailroad grade crossings. A signal detection analysis indicated that the supervisory control system

influenced the observer’s response strategies, but not perceptual sensitivity in detecting 

equipment failures. There were no differences in sensitivity between the three control modes. By 

contrast, participants in the partial supervisory control condition exhibited a tendency similar to 

the manual control condition, to say in-cab failures did not occur, while the participants in the 

full supervisory control condition exhibited a slight preference to say an in-cab failure did occur. 

While participants relied upon the system for routine speed control decisions, participant had to 

intervene when an event outside the scope of the train control system (i.e., an obstruction on the 

track) occurred. They responded by deactivating the supervisory control system and taking over 

active control of the train. Although the participant may have detected the obstruction as far 

away as in the manual mode, making a decision about the proper amount of braking was delayed 

while speed information was gathered. Compared to the manual control condition, more time

was required to understand the situation. This finding is consistent with Wickens and Kessel’s 

(1979) research showing that observers were better at detecting a sudden change in system state 

when manually controlling the system than when monitoring the system. 

The challenge for developers of train control systems is learning how to design the interface so 

the operator remains actively engaged in safety critical tasks. Attention to activities directly 

related to train movement is critical to safe operation. However, using technology in a way that 

degrades the direct involvement of the operator with the system can make it more difficult to 

detect and recover from errors (Huey and Wickens, 1993). This degradation in safety critical 

aspects of their performance was also reflected in the participant’s response to questions about 

workload and stress. Participants rated the full supervisory condition as having the lowest 

workload, but the highest stress. These comments indicated their discomfort with not being 

actively engaged in a safety critical aspect of the train operation. 

The partial supervisory control mode method for assisting with speed control showed some

similarity with the full supervisory control system in its impact on attention allocation. Like the 

ix
 

supervisory control mode, this control mode resulted in participants monitoring events such as 

bearing temperature, traction motor status, and block location better than in the manual mode. It 

also showed an important difference from the full supervisory control mode. Unlike the full 

supervisory control mode, speed awareness performance was better in the partial supervisory 

control mode than in the manual mode. The need to allocate more attention resources to speed 

was attributed by the participants to their difficulty in setting the desired speed. 

The results of this experiment indicate that supervisory control systems can impact operator 

performance in non-intuitive ways. The two types of supervisory control in this study both 

controlled speed, but they affected human performance differently. The complex relationship 

between supervisory control systems and human performance, suggests the need for human-inthe-loop testing to evaluate new train control systems for their impact on human performance 

before they are implemented in revenue service. Human-in-the-loop testing of supervisory 

control systems can identify the likely human errors. Through an iterative design process and 

training, some of these errors may be eliminated. For errors that cannot be eliminated, designers 

can facilitate recovery by making actions reversible and creating warnings to alert the operator 

when they occur. 

x
 

1. INTRODUCTION 

The era of high-speed passenger rail service in the United States began with the service on the 

Northeast Corridor. High-speed rail service plays a central role in mass transportation systems in 

Europe and Japan. Examples include the German Inter-City Express (ICE), the French Train à 

Grand Vitesse (TGV), and the Japanese Shinkansen. 

However, there are safety implications that arise from speed increases that affect locomotive 

engineer performance. First, as train speeds increase, the locomotive engineer has less time to 

acquire information needed to operate the locomotive and respond. A study conducted in France 

found that the maximum speed for accurate perception of wayside signals was 137 mph (220 

km/h) (Askey, 1995). Second, as speed increases, the locomotive engineer has less time to slow 

or stop the train and must decide to slow or stop the train earlier, due to the increased stopping 

distance. As train speed increases beyond the perceptual and cognitive limits of the locomotive 

engineer, the locomotive cannot be safely operated without assistance. 

Train control technology can provide this assistance to the engineer. In high-speed rail 

operations, train control technology using decision support aids, supervisory control, or some

combination has been proposed as a means to compensate for the sensory, perceptual, and 

cognitive limitations of the locomotive engineer. The use of train control technology can be 

divided into two approaches: 

Decision aids. Decision support aids provide information to assist the engineer in train control. In 

such an approach, the locomotive engineer makes decisions regarding vehicle operation and 

actively controls the vehicle. Askey (1995) looked at the use of different levels of decision aids 

to help the engineer in train control operations. She found that decision aids improved safety by 

reducing time needed to respond to emergency events and by reducing the need for emergency 

braking. 

Supervisory control. In this approach, all necessary information for vehicle operation is passed to 

the control system that is responsible for operating the train according to prescribed rules. The 

locomotive engineer’s role shifts to that of a supervisor who monitors the proper functioning of 

the control system. 

The type of train control used in the current operational high-speed rail systems (ICE, TGV, 

Shinkansen) varies by country. According to Sheridan, Lanzilotta, and Askey (1994): “The 

German philosophy (ICE) of rail development emphasizes supervisory control with use of the 

human as a system monitor, while the French (TGV) and Japanese (Shinkansen) depend more on 

the human for control decisions.” 

This study focused on the impact of supervisory control on locomotive engineer performance. 

The successful application of supervisory control in train operations requires understanding how 

it will affect the engineers who interact with this technology. For example, locomotive engineers 

may work long hours with inadequate amounts of rest between shifts (Thomas, Raslear, and 

Kuehn, 1997). Trains cover large distances and locomotive engineers must attend to tasks over 

prolonged periods. As a result, operators can become less attentive over the course of their shift. 

A supervisory control system could protect against the sleeping driver, but could create new 

problems related to attention allocation. 

1
 

Supervisory control may also change how the locomotive engineer allocates his or her attention. 

The question is how do these changes affect the operator's performance and what are the safety 

implications of these changes? To operate the train, the engineer acquires information inside and 

outside the cab. Inside the cab, the engineer monitors locomotive health by monitoring visual 

displays, listening to the sounds made by the locomotive and on-board equipment, and attends to 

kinesthetic cues (i.e., vibrations). Outside the cab, the engineer looks for hazards, monitors 

wayside signals, landmarks, and other visual cues needed for anticipating future train 

movements. The skills and knowledge needed to operate a train build up over time as engineers 

learn how to control the train and the physical attributes of the territory over which they operate. 

Likewise, engineers learn how to effectively allocate attention between events inside and outside

the cab so that the train operates safely and efficiently. 

How do the locomotive engineer's attention allocation strategies change when supervising train 

control systems? What are the safety implications of these changes? For example, a key issue in 

supervisory control is whether the engineer can detect equipment failure and take over safe 

control of the train. In relying on supervisory control, the engineer’s ability to intervene promptly 

in case of an unexpected event can deteriorate (Huey and Wickens, 1993). How does supervisory 

control affect the engineer’s ability to detect unexpected events? This study will address these 

questions using two paradigms: vigilance and situation awareness. Both concepts can be used to 

examine how shifts in attention affect human performance. 

Vigilance 

Vigilance refers to the capacity of the human operator to sustain attention and remain alert to 

stimuli over prolonged periods. Vigilance studies originated during World War II when the 

Royal Air Force (RAF) commissioned Norman H. Mackworth (1961) to study the radar 

observers’ decrement in detection rate of enemy submarines. He discovered that a vigilance 

decrement occurred after only 30 minutes. The most significant aspect of the vigilance 

decrement was that it arose from searching for a relatively infrequent signal over a prolonged 

period of time (Dember and Warm, 1979). 

While vigilance has been studied extensively in the context of laboratory tasks, relatively few 

studies have been performed in environments that closely simulate actual work settings, the main 

reason being the difficulty of applying signal detection theory. An important question is the 

extent to which data from laboratory tasks can be extrapolated to operational tasks because of the 

event rate used. For the majority of vigilance studies, a target rate of 1 target per minute is 

considered low. However, in an operational setting, “events” that need to be distinguished and 

acted upon, such as failures, occur much less frequently. Few studies have been performed with a 

very low target rate on the order of 1 target per 30 minutes (Molloy and Parasuraman, 1996; 

Loeb and Binford, 1970). Lanzilotta (1995) examined the effects of supervisory control on 

vigilance in a human-in-the-loop locomotive simulator and found that operators showed more 

variability in detecting equipment failures with partial supervisory control than full supervisory 

control. He concluded that with the partial supervisory control, observers exhibited a greater 

tendency to attend to events outside the locomotive cab than with full supervisory control. 

However, in this study the number of failures on each 45-minute trip (6) may have affected the 

participant’s detection performance (1 failure per 7.3-minute period). What would happen in a 

system in which the number of failures was smaller? The current study examined this question 

using a small number of failure events. Instead of 6 events per 45-minute trip found in 

2
 

Lanzilotta's study, participants were exposed to only 2 events per 45-minute trip (1 failure per 

22.5-minute period). 

Situation Awareness 

Situation awareness (SA) is a broader concept than vigilance. Endsley (1993) defines SA as 

“the perception of the elements of the environment within a volume of time and 

space, the comprehension of their meaning and the projection of their status in 

the near future.” 

For the locomotive engineer, situation awareness represents the operator's mental model of the 

state of the system. Endsley proposed that this mental model could exist at several levels. First, 

the engineer must perceive the system status. Second, the perceived information must be 

integrated and compared to the operational goals of the system to provide comprehension. Third, 

the operator uses understanding of the system to predict future events and enabling proactive 

behavior. In locomotives with supervisory control systems, the engineer can become a passive 

processor of information. The engineer needs to understand what the train control system is 

doing. However, feedback in supervisory control systems is frequently inadequate. This loss of 

information can make it difficult to detect control failures and take over control when failures 

occur. 

In Lanzilotta's study examining the effect of supervisory train control on situation awareness, 

there was an inverse relationship between situation awareness and the amount of supervisory 

control. As the level of supervisory control increased, situation awareness degraded. Perceived 

situation awareness was highest in the condition where the operator controlled the train manually 

and was lowest in the full supervisory control condition. However, in that study, situation 

awareness was measured by asking participants to rank the supervisory control modes by their 

perceived level of situation awareness. This subjective measure may not give an accurate 

measure of situation awareness as the response may be affected by the outcome of the 

experimental trials. 

An objective method, that compares an operator's knowledge of the system while it is being 

operated with information about the actual state of the system, may give a more representative 

picture of situation awareness. The current study used such a method developed by Endsley 

(1995b) to evaluate the level of situation awareness for different levels of supervisory train 

control. 

Endsley's (1995b) method is called Situation Awareness Global Assessment Technique 

(SAGAT). In this method, participants answer questions about their current perceptions of the 

situation at randomly selected times during the simulation. The simulation is suspended and the 

displays are blanked while the participant answers questions. After the simulation has ended, 

answers to the experimenter's questions are compared to the real situation as measured by the 

computer simulation to provide an objective measure of situation awareness. 

To summarize, the purpose of the current research was to determine the implications of

supervisory train control in high-speed train operations with regard to vigilance monitoring and 

situation awareness. Using a human-in-the-loop locomotive simulator, train control failures were 

introduced to learn how quickly operators could detect the failures when the number of events 

requiring action by the operators was very low for different levels of supervisory control. 

3
 

Situation awareness measurements were collected during the simulation, using the SAGAT 

developed by Endsley. 

Three train control modes were evaluated: manual, partial, and full supervisory control. Under 

manual control, the operator actively controlled the train. Under partial supervisory control, the 

human operator set the desired speed (i.e., cruise control) and the train control system was 

responsible for achieving and maintaining it. Under full supervisory control, no human 

intervention was required as the train control system was programmed to adjust train speed in 

response to the appropriate signals and civil speed limits. 

4
 

2. METHOD 

Overview

A laboratory experiment, using a human-in-the-loop locomotive simulator, was conducted to 

examine the effects of three train control modes on operator attention. Specifically, the 

experiment examined the effects of train control in the situation where action by the engineer 

was not required for a long period (low event rate). As part of this study, two levels of 

supervisory train control (full supervisory control, partial supervisory control), and manual 

control operation were evaluated for their effects on vigilance and situation awareness. A series 

of events inside and outside the locomotive were presented to participants while operating the 

train and the ability to detect these events was measured. 

To measure vigilance, two types of train control failure and an obstruction were introduced. The 

failures remained active for a set period, unless detected and reset by the participant. If not reset 

by the participant within the specified time interval, the failure was automatically reset. The time 

to complete a round trip (45 minutes) was broken down into three 15-minute blocks and the 

detection rate was measured in each block. The percentage of failures detected by the 

participants in each time block was used as a measure of vigilance (Molloy and Parasuraman, 

1996) inside the cab. Outside the cab, vigilance was measured by the distance from the

obstruction where the train stopped. 

To measure situation awareness, Endsley's Situation Awareness Global Assessment Technique 

was used (1995b). In this method, the simulation was paused and the displays were blanked at 

periodic intervals. The participant was asked several questions about the status of the 

locomotive's operating condition and its position on the track. 

Experimental Design 

The independent variable was train control mode. There were two types of supervisory control 

and one type of manual control as shown in Table 1. All participants were exposed to all train 

control conditions. 

Table 1. Train Control Condition

Type of Train Control 

Full Supervisory 

Partial Supervisory 

Manual 

To counterbalance possible learning effects across participants, each participant was introduced 

to the control modes in the order shown in Table 2. To encourage the participant to perform

according to the objectives of the experiment, an incentive system was implemented. Points were 

awarded or taken away according to the bonus and penalty schedule described in Appendix A. 

Points were converted to monetary rewards at a rate of one dollar per 1000 points. 

5
 

Table 2. Presentation Order of Train Control Modes 

Participant Presentation order

1 2 3

1 Full Supervisory Manual Partial Supervisory 

2 Full Supervisory Partial Supervisory Manual 

3 Manual Partial Supervisory Full Supervisory 

4 Manual Full Supervisory Partial Supervisory 

5 Partial Supervisory Full Supervisory Manual 

6 Partial Supervisory Manual Partial Supervisory 

Failure Detection 

To assess vigilance performance inside the cab, two types of equipment failures were introduced. 

Appendix C shows the track position where the failures occurred. 

Motor failure. Under normal operating conditions, electric current flowed through each of four 

motors when thrust was applied. The ammeters gauge displayed the amount of current flowing 

through each motor. A motor failure occurred when current failed to flow through one of the 

motors, indicated on the ammeter gauge. If the participant detected a motor failure, the 

participant was instructed to remove power from all the motors by pulling the control lever back 

to the braking position, press the appropriate key for resetting the failure, and resume operation. 

A motor failure lasted 12 seconds unless reset by the participant. If the participant failed to reset 

the motor failure within that time, the system was automatically reset. 

Bearing failure. In principle, bearing temperature normally follows the speed of the train with 

some time lag, but never rises above a threshold (131o F) for safety reasons. In practice, the 

bearing temperature fluctuated around a mean value, depending on the average speed of the 

vehicle. When a bearing failed, the temperature increased at a rate almost double the maximum 

rate regardless of the train’s speed. Detecting a bearing failure required the participant to observe 

this abnormal temperature change. The participant was instructed to press a reset key to correct 

the bearing failure. The bearing failure lasted 20 seconds unless reset by the participant. If the 

participant failed to reset the bearing failure within that time, it was reset automatically. 

Within each combination of control mode and time block nine equipment failures occurred. With 

nine combinations of control mode and time block, 81 equipment failures occurred. Appendix D 

shows the responses made to the equipment failures and obstructions for each participant. The 

average number of failures (motor or bearing) for a round trip was two, one for each type of 

failure.

Bonus points were awarded for detecting and resetting the failures. Penalty points were given for 

pressing the reset key when no failure existed. The incentive system is described in Appendix A. 

6
 

Obstruction Detection 

There were five grade crossings in this experiment. At each grade crossing, highway motor 

vehicles could cross in front of the train from either direction. These motor vehicles were visible 

at over 0.3 miles (0.5 km) from the train. Each block containing grade crossings had a civil speed 

limit of 62-mph (100 km/hr). Traffic at the grade crossings arrived according to a probabilistic 

process. A car would proceed across the crossing only if there was sufficient distance to clear the 

crossing before the train arrived. However, it was possible for a car to become disabled as it was 

crossing the tracks, which would obstruct the track. The participant was instructed to bring the 

train to a complete stop before the intersection if a disabled motor vehicle was detected. If the 

participant could not avoid a collision, a crack appeared on the locomotive cab window. The 

crack remained on the window for the remainder of the trip. 

It was important for the participant to quickly determine whether the train should stop or 

proceed. The participant had to balance the consequences of a collision with the costs of 

schedule delays due to braking the train. It was up to the participant to evaluate the situation and 

determine the best course of action. This type of emergency was used to assess the effects of 

train control in attending to events outside the locomotive. 

If the participant collided with an obstruction, the participant was instructed to stop the train and 

notify the dispatcher. The participant then continued on the trip. The cracked window remained 

displayed for the remainder of the trip. The participant was awarded points for detecting and 

avoiding the hazard according the schedule shown in Appendix A. 

Situation Awareness 

Situation awareness was measured using a technique developed by Endsley (1995b), whereby 

the simulation was temporarily paused while the experimenter asked the participant questions 

about the state of the train and its position. While the simulation was paused, the out-the-window 

view and instrument panel appeared blank. The train remained in the same position until the 

question and answer session was completed. The participant then resumed operating the train at 

the same speed and position at which the simulation was paused. The experimenter asked the 

participant to give answers about the train's position with respect to block number, milepost, 

train speed, and bearing temperature. 

Two measures were used to assess situation awareness: percent of correctly answered questions 

or Mean Absolute Error (MAE). MAE is a measure of deviation between the participant’s 

response and the actual state of the variable. It was defined as: 

∑=

− =

n

i n

x x MAE

1

| ˆ |

where x was the participant's response, was the actual value and n was the number of 

participants. Speed and bearing temperature were calculated using MAE. For motor current, 

block number and milepost answers were classified either as correct or wrong and reported as a 

percent of the correct answers. 

xˆ

7
 

Participants

Fifteen people participated in the experiment. Twelve participants were undergraduate or 

graduate students at the Massachusetts Institute of Technology (MIT). The remaining three 

participants were people who responded to notices placed on the MIT campus requesting 

participation in the experiment. Two students participated in a pilot study for fine-tuning the 

experiment, while the remaining 13 participated in the actual experimental trials. Data from one 

participant was thrown out after inspection showed that he repeatedly failed to comply with 

speed limits. Each participant was paid $10 per hour. 

Facilities: High-Speed Locomotive Simulator

The High-Speed Locomotive Simulator was a real-time human-in-the-loop locomotive simulator 

developed to conduct human factors research in high-speed rail operations. The computer 

hardware consisted of two Silicon Graphics (SGI) personal Iris Workstations, one SGI Indigo-2 

workstation, an IBM compatible 486 computer, a Barco projector, and projection screen. The 

participant sat in the cab where the controls and instrument displays were located. Figure 1 

shows the layout. The out-the-window view was displayed on a wall-mounted projection screen 

in front of the cab. 

Projector

Instrument

Display

Out-the-window

view

Figure 1. Train Simulator Layout

The operator controlled the locomotive’s speed with a lever that combined both throttle and 

brake functions. When the control lever was in the center (vertical) position, the train control was 

in neutral. Pushing the control lever forward away from the operator accelerated the train; 

pulling the lever toward the operator decelerated the train. The operator had push buttons to the 

left and right of the instrument display for controlling the emergency brake, alerter, bell, horn, 

and traction motors. The SGI Indigo-II workstation generated the out-the-window view as well 

as computing the vehicle dynamics that enable the operator to control the train. One personal Iris 

displayed the instrument panel shown in Figure 2, while the other personal Iris served as the 

8
 

dispatcher workstation. All three machines communicated with each other over a Local Area 

Network (LAN). Appendix B describes the features added to the locomotive simulator for this 

experiment.

Figure 2. Instrument Panel 

mph

The simulator exhibited three control modes: manual control, partial supervisory control, and full 

supervisory control. In manual control, the operator was solely responsible for controlling the 

speed and the position of the vehicle using the combined control lever. The partial supervisory 

control system was designed to maintain a constant speed set by the operator. When operating 

the train in partial supervisory control mode, the operator set the desired speed. In this mode, the 

train control applied the proper amount of thrust to achieve and maintain the set speed. Partial 

supervisory control did not affect braking, so the operator had to apply the brakes manually when 

needed. In full supervisory control, speed control of the vehicle was exclusively assumed by the 

train control system, which was responsible for setting the speed in accordance to the speed 

limits of each block. Speed control included application of both the throttle and brakes. In both 

the full and partial supervisory control modes, manual application of the brakes by the operator 

disengaged the supervisory control system. The full supervisory control system controlled the 

speed of the vehicle without human intervention. 

The territory in the experiment was comprised of two fictional stations, named East Station and 

West Station and a single track between the two stations, as shown in Figure 3. Thirty-one miles 

(50 km) of single track separated the two stations. Loops at the end of each station were used to 

reverse the vehicle’s direction on the main track. One-way travel time from station to station was 

9
 

approximately 20 minutes. Travel time around the loops was almost 5 minutes. Hence, the round 

trip lasted roughly 45 minutes. 

West

Station

East

Station

Figure 3. Track Layout 

The current simulation used block signaling to communicate permission to occupy the track and 

speed limits. With block signaling, the track was divided into sections. In the current simulation, 

all blocks between stations were 1.24 miles (2 km), and all blocks in the loop sections were 0.62 

miles (1 km) in length. A color-coded signal was located at each block boundary. This signal 

indicated the maximum speed permitted through the block ahead. It was the responsibility of the 

operator to identify the signal and modify the train speed accordingly. Normally, only one train 

can occupy a block at any given time. A red signal indicated that another train currently occupied 

the block. An approaching train was not permitted to enter that block. As trains approached a 

block occupied by another train, the signals for each block became increasingly restrictive. This 

system was designed to prevent collisions between trains by slowing the approaching train in 

time to stop before entering the occupied block. 

In addition to the speed limits imposed by the block signal system, there were also civil speed

limits. These speed limits were dictated by specific right-of-way characteristics or track 

geometry. In the current simulation, the civil speed limits varied depending upon whether the 

block contained a grade crossing. Blocks with grade crossings had lower civil speed limits than 

blocks without them. In all cases, the prevailing speed limit was the lesser of the block signal 

limit and the civil speed limit. 

Procedures

Training 

Due to the complexity of the task and to familiarize the participants with train operations, each 

participant was given a written tutorial to read prior to their participation in the experiment. The 

10
 

tutorial covered general operating rules that govern railroad operations, how to operate the 

locomotive simulator, and described the experimental task. 

Upon arriving for the experiment, the participant filled out a consent form and completed a 

multiple-choice test. The test served as a tool to assess their understanding of the material in the 

tutorial and as a participant selection tool. The test took approximately 10 minutes. Participants 

scoring correctly on 50 percent or better proceeded to the training phase of the study. All 

participants achieved this criterion. The experimenter used the test results to identify potential 

problems with the material that needed to be clarified during the training sessions that followed. 

The training session lasted three hours and gave hands-on exposure to the system. During the 

first hour, the experimenter demonstrated the proper train operation. Next, the experimenter 

familiarized the participant with the emergency scenarios and the appropriate responses. The 

remaining two hours were devoted to practice operating the locomotive simulator with and 

without the failure scenarios to become acquainted with the train control system and its operating 

modes. During this period, the participant completed a run from station to station using one 

control mode. During the practice runs, the experimenter was physically present in the cab and 

monitored how the participants controlled the locomotive. The experimenter recommended 

proper actions under each control mode. The experimenter also gave the participant suggestions 

regarding speed compliance. These practice runs lasted about one hour and a half. During the 

final half-hour of training, the participant completed a run from station to station with the 

opportunity to experiment with the simulator in any control mode. During this run, the 

experimenter split his time between the cab and the dispatcher workstation. Additionally, the 

experimenter suspended the simulation three to four times to familiarize the participants with the 

interruptions of the simulation that would take place during the experiment to assess situation 

awareness. 

After the training session ended, the experimenter gave the participant a short break and asked 

whether he or she felt ready to begin the experiment or wanted additional practice operating the 

train. The participant either requested additional practice trips or proceeded to the experimental 

trials. 

Experimental Task 

The participant's task was to control the vehicle speed according to the signal and civil speed 

limits indicated in each block, look out for potential hazards and respond to them in an 

appropriate manner. 

The participant operated the simulated locomotive from West Station to East Station as shown in 

Figure 3 and back (one trip), using one particular control mode. At the end of a round trip, the 

participant was given a brief rest and then began another round trip under a different control 

mode. Each participant completed three round trips, one for each control mode. Each trip lasted 

about 45 minutes. Between trips, the participant rested for 10 minutes or longer before 

proceeding to the next trial. Following completion of the three trips, the participant answered 

questions regarding their preferences and the workload and stress imposed by the three control 

modes. 

11

 

3. RESULTS AND DISCUSSION 

Vigilance 

Signal detection theory was used to measure the participant’s ability to detect equipment failures. 

In signal detection theory, detection is a function of two processes: the observer’s perceptual 

sensitivity and response bias. This theory enables separation of the effects of perceptual 

sensitivity, in this case, the observer’s ability to detect equipment failures, from response bias. 

Response bias represents the observer’s tendency to favor one response over another (MacMillan 

and Creelman, 1991) (i.e., willingness to say “yes” or “no”). 

In signal detection theory, events can be categorized in a 2 x 2 matrix showing the relationship 

between the state of the world and the observer’s response to that state. Figure 4 shows the four 

possible categories. A hit occurs when a signal is present and the observer reports that the signal 

is present. A false alarm occurs when a signal is absent and the observer reports that the signal is 

present. A miss occurs when a signal is present and the observer reports that the signal is absent. 

A correct rejection occurs when a signal is absent and the observer reports that the signal is 

absent. 

State of the World 

Signal Noise

Yes Hit False 

Observer’s alarm

Response No Miss Correct 

rejection 

Figure 4. Four Outcomes of Signal Detection Theory 

Figure 5 shows how signal detection theory characterizes the relationship between the observer’s 

responses for two hypothetical distributions. The distribution on the right represents the 

probability that a signal (i.e., an equipment failure) was present. The distribution on the left

represents a probability that noise (i.e., equipment functioned properly) was present. The 

observer’s ability to detect equipment failure (sensitivity) is reflected by the amount of overlap in 

the two distributions. Sensitivity increases as the amount of overlap in the two distribution 

decreases. Response bias is represented by the vertical line showing the response criterion. When 

the value of the event is to the right of the criterion, the observer will say “yes.” When the value 

of the event is to the left of the criterion, the observer will say “no.” The response criterion is 

neutral (zero) when the false alarm rate and miss rate are equal. Moving the criterion to the left 

increases the likelihood that the observer will say “yes,” while moving the criterion to the right 

increases the likelihood that the observer will say “no.” Numerically, a positive bias represents 

the tendency to say no, while a negative bias represents the tendency to say yes. 

Sensitivity is typically measured numerically by an index called d prime and response bias is 

typically measured numerically by an index called beta (β). D prime (d') corresponds to the 

separation of the mean of two distributions expressed in units of their standard deviations as 

shown in Figure 5. The response bias, was measured by the ratio of the probability of saying 

“yes” when the signal is present to the probability of saying “yes” when only noise is present. 

13
 

For the reader interested in learning more about signal detection theory, refer to books by Green 

and Swets (1988), Egan (1975), or MacMillan and Creelman (1991). 

In this experiment, an alternative measure of sensitivity and response bias was used. D prime and 

beta cannot be calculated if either hits or false alarms equal zero or 100 percent, as was the case 

for several conditions. Therefore, nonparametric measures were used to measure sensitivity and 

response bias. A prime (A') was used (Grier, 1971) to measure sensitivity and B double prime 

(B'') was used to measure response bias. A' measures the area under curve for the measured data 

point and numerically corresponds to the ratio of hits to false alarms for a given event state. For 

A prime, values ranged between zero and one. For B double prime, values ranged between minus 

one and plus one. 

To evaluate participants’ ability to detect equipment failures, the number of hits and false alarms

were recorded. The results for the train equipment failures are summarized in tabular form in 

Table 3 and graphically in Figure 6. Figure 6 shows the percentage of correctly detected signals 

(for both motor and bearing failures) in each time block for by control mode. In detecting the 

train equipment failures, evidence of a statistically significant vigilance decrement is present for 

all three control modes.1

 For all three conditions, detection of equipment failure declined over 

time. Detection of the equipment failures under the full supervisory mode was consistently 

higher than for the other two control modes across all time blocks. Detection performance was 

the worst in the partial supervisory mode, with the manual control mode falling between the two 

supervisory control modes. The difference in detection performance between the modes was not 

statistically significant. However, Lanzilotta (1995) found the same pattern in his study. The lack 

of statistical significance may be due to the small size of the effect and the small number of data 

points on which the statistical test was based. 

Figure 5. Graphical Depiction of Signal Detection Theory 

 

1 The statistical analysis was based upon the Page test. The Page test is a distribution-free test for ordered 

alternatives based on Friedman rank sums. A discussion of this test is found on page 147 in Hollander and 

Wolfe, (1973). The test statistic L=41.5 is greater than the critical value for l (0.05,3,3)=41 at the 5 percent 

level of significance. 

14
 

Table 3. Detection of In-Cab Equipment Failures by Control Mode 

Time block 1 Time block 2 Time block 3 Control Mode

Total 

Hits

Full Supervisory 8 7 7 22

Partial Supervisory 6 6 6 18

Manual 8 7 6 21

Time Block Total 22 20 19 61 

False Alarms

Full Supervisory 3 3 0 6

Partial Supervisory 2 1 0 3

Manual 0 2 0 2

Time Block Total 5 6 0 11 

False alarms decreased over time for the two supervisory control conditions. By contrast, in the 

manual control condition, false alarms exhibited an inverted u-shaped function, where the 

number of false alarms went from zero to two and back to zero. 

0

10

20

30

40

50

60

70

80

90

100

1 2 3

Time Block

Detection rate (%)

Autopilot

Manual Control

Cruise Control

Figure 6. Effects of Train Control Mode on Detection Rate 

As shown in Figure 7, the two distributions can also be characterized by the relationship between 

the probability of hits and false alarms. The vertical axis shows the probability of hits and the 

horizontal axis shows the probability of false alarms. Sensitivity increases as points on the curve 

move from the lower right-hand corner to the upper left-hand corner of the chart. Response bias 

changes from positive to negative as the data points move from the lower left-hand corner to the 

upper right-hand corner. 

When the detection rate and false alarms were examined together, the results show declines in 

the number of hits and in the number of false alarms over time. This observation suggests that 

participants changed their response bias over time. For both control mode and time block, 

15
 

sensitivity appeared to change little. Changes in sensitivity are reflected in this chart by 

differences in the position of the data points along the minor diagonal line. 

1

2

3

Full

Partial

Manual

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

P(False Alarm)

P(Hit)

Time Block

Control Mode

Figure 7. Probability of Hits vs. False Alarms by Train Control Mode and 

Time (The labels: 1, 2, and 3 represent the time block order) 

The impact of control mode and time on detecting equipment failures can be more easily seen by 

separating the effects of sensitivity and response bias. Figure 8 shows the impact of sensitivity 

and response bias separately by control mode and time sequence. The top graphs in Figure 8 

show the impact of time block and control mode on sensitivity. There is little change in 

sensitivity across time block or control mode. Sensitivity is relatively high across time and 

control mode. The differences in sensitivity were not statistically significant. 

By contrast, differences were observed for effect of time block and control mode on response 

bias. The bottom two graphs in Figure 8 show the impact of time block and control mode on 

response bias. For time block, the response is similar for the first two time blocks. Here the 

response criterion is neutral with a B'' value close to zero. However, for the third time block, the 

response criterion is highly positive, with a B'' at 1.0. These differences were statistically 

significant (B''=1.0 comparing time block 1 to time block 3, confidence level 0.95 =0.47; B''=0.96 

comparing time block 2 to time block 3, confidence level 0.95 = 0.45). Thus, participants shifted 

their response criterion over time so they were highly likely to say, “no, an equipment failure 

was not present.” This is consistent with past research in the vigilance decrement literature 

(Getty, Swets, Pickett, and Gonthier, 1995). The observers moved their response criterion (i.e., 

response bias) as time elapsed so that there were fewer false alarms and fewer hits to reflect the 

actual probability with which those events were likely to occur. 

16
 

0

0.2

0.4

0.6

0.8

1

1 2 3

Time Block

Sensitivity (A')

0

0.2

0.4

0.6

0.8

1

Full

Supervisory

Partial

Supervisory

Manual

Control Mode

Sensitivity (A')

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1 2 3

Time Block

Response Criterion (B")

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Full

Supervisory

Partial

Supervisory

Manual

Control Mode

Response Criterion (B")

Figure 8. Sensitivity and Response Criterion by Time Block and Control Mode 

Response criterion differences were also observed between the three control modes. In the full 

supervisory mode, participants exhibited slightly negative response criterion, meaning that 

participants were slightly more likely to say, “yes, an equipment failure occurred” than “no.” By 

contrast, for both the partial supervisory mode and manual control mode, response criterion was 

positive. Participants were more likely to say, “No, an equipment failure did not occur” or miss 

reporting an equipment failure when it did occur. The response bias differences between the full 

supervisory control mode and the two other modes were not statistically significant at the 95 

percent confidence level. The response criterion difference between the full supervisory mode 

and the manual mode was statistically significant at the 90 percent confidence level. The lack of 

statistical significance at the 95 percent confidence level may have been due to the small sample 

size (27 data points per condition). The response criterion differences between the partial control 

mode and the manual mode were not statistically significant. This behavior suggests that 

17
 

observers would be less likely to miss equipment failures when operating full supervisory train 

control systems than partial or manual train control systems. 

Vigilance monitoring out-the-window between the three control modes differed from the 

vigilance monitoring associated with the in-cab displays. Figure 9 shows the distance from the 

obstruction that the vehicle stopped for the three train control modes. Upon detecting an 

obstruction, all participants made a full service brake application. Next, the participants decided 

if and when to apply the emergency brakes. In every case but one, participants applied the 

emergency brakes to avoid a collision. 

Participants stopped farthest from the obstruction in the manual control condition followed by 

the partial supervisory control condition and the full supervisory condition. The differences 

between the manual mode and both supervisory modes were statistically significant. Using the ttest, the t value for the comparison between manual mode and full supervisory control was t (18) = 

2.037, p < 0.05.2

 For the comparison between manual mode and partial supervisory control, the t 

value was t (18) = 1.901, p < 0.05. The difference in stopping distance between the two 

supervisory control modes was not statistically significant. 

0

100

200

300

400

500

600

Full Supervisory Partial Supervisory Manual

Control Mode

Distance from Obstruction (feet)

Figure 9. Distance Stopped from an Obstruction 

Situation Awareness 

During the experimental trials, the simulation was suspended and each participant was asked to 

report the following items: speed, block number, milepost, brake pressure, bearing temperature 

 

2

 The t-value represents a ratio of systematic errors or unsystematic errors. It measures the difference between two 

sample means divided by the standard deviation for the sample. The phrase "p < .05" means there are 5 chances in

100 that the observed result was due to chance. 

18
 

and current through the motors. Because the brake pressure was stable over time, there was little 

deviation in the brake pressure gauges. As a result, there were no reported significant deviations 

from the actual brake pressure. 

The results for the remaining situation awareness measures are presented in Figure 10, Figure 11, 

and Figure 12. Figure 10 shows the relationship between train control mode and speed situation 

awareness. Figure 11 shows the relationship between train control mode and bearing temperature 

situation awareness. Figure 12 shows the relationship between train control mode and situation 

awareness with respect to train position and motor status. 

For speed awareness, the MAE was lowest in the partial supervisory control mode, followed by 

the manual control mode and the full supervisory control mode, respectively. A smaller error 

indicated better situation awareness. These differences were statistically significant.3

 

0

2

4

6

8

10

12

14

Full Supervisory Partial Supervisory Manual

Control Mode

Speed Deviation (

mph)

Figure 10. Effects of Train Control Mode on Speed Awareness 

Speed awareness was the only measure for which participants performed best in the partial 

supervisory control mode. However, as Table 4 shows, none of the participants preferred this 

hybrid control mode. Comments from the participants suggest that the operational 

implementation of the partial supervisory control mode affected the participants' attention 

allocation strategies. Many participants complained of difficulty setting the train controls. In the 

supervisory control mode, participants set the speed using the keyboard function keys while 

using the control lever to brake or accelerate manually. As a result, they paid more attention to 

the speedometer, compared to the other control modes to make sure that the train control was 

achieving the set speed. The difficulty setting the speed control may have contributed to the 

 

3

 The test statistic L=154 was greater than the critical value for l (0.05,3,12)=153, p<0.05. The data for this test is

presented in Appendix G. 

19
 

lower detection rate for in-cab equipment failures compared to the other modes and the higher 

situation awareness scores for train speed. 

In the full supervisory control mode, participants reported less time monitoring speed than in the 

other modes, while spending more time attending to other visual displays in the locomotive. To 

quote one participant: 

“The full supervisory mode imposed the least workload since I could keep an eye 

on what was happening, yet I didn’t have to worry as much about watching the 

speed. I had to check to make sure that train control system was following the 

speed limits, but I didn’t actually have to adjust the speed myself. So I could 

spend my time systematically checking all the information.” 

Participants spent less time monitoring train speed, since the supervisory control system 

controlled both acceleration and braking and focused more attention elsewhere. 

For all other measures of situation awareness (temperature, position, and traction motor 

awareness), a different pattern emerged from that found with speed awareness. The best 

performance was observed in the full supervisory control mode, followed by the partial 

supervisory control mode and manual control mode showing the poorest situation awareness. For 

temperature, the MAE was lowest for the full supervisory control followed by the partial 

supervisory control mode and the manual control mode, respectively. This difference was 

statistically significant.4

 This supports many of the participants' comments that they spent more 

time monitoring other displays inside the cab compared to the speedometer in the full 

supervisory control mode. 

For train position (as indicated by block number and milepost) and motor status, participants 

gave the highest percentage of correct answers in the full supervisory control mode. 

 

4

 The test statistic L=156 was larger than the critical value l(0.05,3,12)=153, p<0.05. The data for this test is 

presented in Appendix G. 

20
 

0

2

4

6

8

10

12

Full Supervisory Partial Supervisory Manual

Control Mode

Temperature Deviation (oF)

Figure 11. Effects of Train Control Mode on Bearing Temperature Awareness 

By contrast, in the manual control mode, block position and motor status awareness were lower 

than the two supervisory control modes. For milepost, situation awareness was equal between the 

manual control mode and the partial supervisory control mode. 

0

10

20

30

40

50

60

70

80

90

100

Block Milepost Motor Status

Percentage of Correct Responses

Full Supervisory

Partial Supervisory

Manual

Figure 12. Effects of Train Control Mode on Position and Motor 

Condition 

At the conclusion of the experiment, participants were asked to indicate which control mode 

created the lowest workload and highest stress level. Participants were also asked to indicate 

21
 

their preferred control mode. The results are shown in Table 4. For workload, participants felt the 

full supervisory control mode imposed the lowest workload. For the two remaining modes, 

participants felt the partial supervisory control mode imposed greater workload than the manual 

control mode. For stress level, participants felt that both supervisory control modes imposed 

more stress than the manual control mode. Of the two supervisory control modes, full 

supervisory control was more stressful than the partial supervisory control. A comment by one of 

the participants described how the control mode affected their stress levels. 

“I believe that the partial supervisory control and full supervisory control 

imposed the greatest amount of stress when an obstruction was detected because 

after setting the speed, I would remove my hands from the control and begin to 

focus on other potential problems and information that I had to know. When an 

obstruction occurred, I had to jump from passively driving to actively driving and 

grab the controls in order to slow down.” 

This comment reflects the challenge required in transitioning between train control modes. 

Responding to an obstruction on the track in a supervisory mode required the participant to 

transition from a passive monitor of train speed to an active controller of train speed. Once the 

obstruction was detected, the participant had a relatively short period to determine what the train 

was doing and decide how to respond. 

While the majority of participants felt full supervisory control imposed the lowest workload, they 

preferred operating the train manually to full supervisory control. Participants cited a feeling of

control over the vehicle along with active engagement in the task, as explanations. No participant 

chose the partial supervisory control condition as the preferred mode of operation. Participants 

who preferred the full supervisory control mode cited the lower workload associated with this 

condition. 

Table 4. Participant Preferences for Train Control Mode from Exit Interview

Train Control Mode 

Full Partial Manual

Lowest Workload (%) 83 0 17 

Most Stressful (%) 75 25 0 

Preference (%) 33 0 67 

Summary 

Overall, supervisory train control systems improved situation awareness and vigilance 

performance compared to manual operation. However, differences in the way the supervisory 

control systems operated contributed to differences in attention allocation. These differences 

were greatest for unexpected events such as equipment failures and obstructions at highwayrailroad grade crossings. A signal detection analysis indicated that the supervisory control system

influenced the observer’s response strategies, but not perceptual sensitivity in detecting 

equipment failures. There were no differences in sensitivity between the three control modes. By 

contrast, participants in the partial supervisory control condition exhibited a tendency similar to 

the manual control condition, to say in-cab failures did not occur, while the participants in the 

full supervisory control condition exhibited a slight preference to say an in-cab failure did occur. 

22
 

In the full supervisory control system, the system adjusted speed based upon information from

the in-cab signal system. If the speed limit changed, the train control system made the adjustment 

without assistance from the operator. This assistance enabled the participant to attend to other 

tasks. Compared to manual operation, participants showed greater situation awareness for nonspeed-related events such as bearing temperature, traction motor status, milepost, and block 

location. 

However, participants showed greater difficulty with speed-related tasks compared to the manual 

control mode. Participants exhibited the poorest situation awareness for speed in the full 

supervisory control condition. The implications of this behavior can be seen in obstruction 

detection. Participants took longest to stop when an obstruction was present in full supervisory 

control. Obstruction detection performance in the partial supervisory control mode was similar to 

the full supervisory control mode. While the participant relied upon the system for routine speed 

control decisions, the participant had to intervene when an event outside the scope of the train 

control system (i.e., an obstruction on the track) occurred. The participant responded by 

deactivating the supervisory control system and taking over active control of the train. Although 

the participant may have detected the obstruction as early or earlier than in the manual mode, 

making a decision about the proper amount of braking was delayed while speed information was 

gathered. Compared to the manual control condition, more time was required to understand the 

situation. Where supervisory control systems are provided to enable train control operations that 

could not take place safely without them (i.e., high-speed operation), the likelihood for the 

locomotive engineer to successfully intervene when the train control system encounters an event 

it is not prepared to handle, will decrease. This finding is consistent with Wickens and Kessel’s 

(1979) research showing that observers were better at detecting a sudden change in system state 

when manually controlling the system than when monitoring the system. 

The challenge for developers of train control systems is learning how to design the interface so 

the operator remains actively engaged in safety critical tasks. Attention to activities directly 

related to train movement is critical to safe operation. However, using technology in a way that 

degrades the direct involvement of the operator with the system can make it more difficult to 

detect and recover from errors (Huey and Wickens, 1993). This degradation in safety critical 

aspects of their performance was also reflected in the participant’s response to questions about 

workload and stress. Participants rated the full supervisory condition as having the lowest 

workload, but the highest stress. These comments indicated their discomfort with not being 

actively engaged in a safety critical aspect of the train operation. 

The partial supervisory control mode method for assisting with speed control showed some

similarity with the full supervisory control system in its impact on attention allocation. Like the 

supervisory control mode, this control mode resulted in participants monitoring events such as 

bearing temperature, traction motor status, and block location better than in manual mode. It also

showed an important difference from the full supervisory control mode. Unlike the full 

supervisory control mode, performance in the speed awareness was better in the partial 

supervisory control mode than in the manual mode. 

The participants attributed need to allocate more attention resources to speed to their difficulty in 

setting the desired speed. The partial supervisory control system in this study increased perceived 

workload compared to the manual control mode and focused the participant’s attention more 

narrowly than the full supervisory control mode. Participants attended more closely to the speed 

display compared to the full supervisory control mode. This design may have prevented 

23
 

participants from allocating their attention as broadly as in the full supervisory mode and 

contributed to the vigilance decrement for unexpected events like equipment failures. 

Additionally, greater speed awareness failed to translate into better performance when it came 

time to take over active control of the train when the train control system was not equipped to 

handle the situation. Again, participants took longer to stop in response to a track obstruction 

compared to manual control. 

The current study examined equipment failures that were unrelated to the supervisory control 

system, itself. However, the problem of reliance may be compounded if the supervisory control 

system, itself fails without adequate warning to the operator. Stanton and Young (1998) discuss 

three studies examining the use of adaptive cruise control in motor vehicles. These studies 

evaluated the ability of motorists to detect and respond to adaptive cruise control failures. When 

the adaptive cruise control failed, drivers collided with moving or stationary vehicles more 

frequently compared to manual operation. Expectations by the driver that the cruise control 

technology would deal with this situation may have contributed to this detection failure. In other 

words, the driver trusted the system based upon experience showing reliable behavior. The more 

reliable the control system appears to be, the more the operator may depend on the control 

system. For highly reliable control systems, failures may be very difficult to detect without an 

adequate warning. 

The results of this experiment indicate that supervisory control systems can impact operator 

performance in non-intuitive ways. The two types of supervisory control in this study both 

controlled speed, but they affected human performance differently. The complex relationship 

between supervisory control systems and human performance, suggests the need for human-inthe-loop testing to evaluate new train control systems for their impact on human performance 

before they are implemented in revenue service. Human-in-the-loop testing of supervisory 

control systems can identify the likely human errors. Through an iterative design process and 

training, some of these errors may be eliminated. For errors that cannot be eliminated, designers 

can facilitate recovery by making actions reversible and creating warnings to alert the operator 

when they occur. 

24
 

APPENDIX A. BONUS AND PENALTY POINT SYSTEM 

An incentive system was created to encourage participants to attend to the schedule and failure 

events. For each participant, the points were converted to money at the rate of one dollar for 

every 1000 points. A participant with a negative score would receive no money beyond what all 

participants were paid for participating in the study. Participants were paid after completing the 

experiment. Bonus or penalty points were awarded for the following behaviors: 

• Schedule accuracy, 

• Response to motor failure, 

• Response to bearing failure, 

• Response to brake failures, 

• Application of emergency brake, 

• Response to a grade crossing obstruction, and 

• Answers to situation awareness questions.

Figure A-1 shows the number of points awarded to the participant for schedule accuracy. 

Completing the trip in less then 45 minutes results in the participant receiving (bonus) points. 

Completing the trip in more than 45 minutes results in the participant losing points (penalty). 

Figure A-2 summarizes the incentive system applied to motor and bearing failures. 

Schedule Accuracy

-1000

-800

-600

-400

-200

0

200

400

600

800

40 42 44 46 48 50 52 54

Trip Time (minutes)

Bonus/Penalty Points

Figure A-1. Schedule Accuracy Bonus/Penalty Points 

25
 

Failure Detection

Motor Failure

Bearing Failure

0

200

400

600

800

1000

1200

02468 10 12 14 16 18 20

Failure Detection Time (s)

Bonus Points

Figure A-2. Emergency Response Time Bonus Points 

When a brake failure occurred, participants were given 800 points for observing and resetting the 

brakes. Penalties were given for application of the emergency brakes. Table A-1 shows the 

penalty schedule for brake applications under different circumstances. In fact, penalties 

regarding inappropriate application of the emergency brakes will result as shown in Table A-1. 

Table A-1. Penalties Resulting from Application of the Emergency 

Brakes 

Emergency Stop braking Penalties 

Triggered by ATP: -2000 

Triggered by the Alerter: -2000 

Triggered by the operator - collision 

avoidance: 

-100 

Triggered by the operator - no reason: -300 

For obstructions at grade crossings, bonus and penalty points were assessed according to the 

schedule shown in Table A-2. 

Table A-2. Obstruction Hazards Bonus/Penalty Scheme 

Obstruction Outcome Bonus/Penalty 

Collision avoidance: +1000 

Collision Speed: >=55.9 mph -1000 

Collision 43.5 mph <= Speed <= 90 mph -800 

Collision 24.8 mph <= Speed <= 43.5 mph -600 

Collision 6.2 mph <= Speed <= 24.8 mph -450 

Collision Speed<= 6.2 mph -250 

26
 

Participants were awarded 1000 bonus points for each correct answer to the situation awareness 

questions. Participants received no points if the answer was incorrect. 

27

 

APPENDIX B. NEW FEATURES OF HIGH-SPEED TRAIN SIMULATOR 

For the purposes of the present research, several new features were added to the train simulator. 

These features are described below: 

• Suspend/resume capabilities in the train simulator. This feature is designed for gathering 

situation awareness data. This feature enables the experimenter to suspend the simulation and 

ask questions relevant to the state of the vehicle. Two versions were designed and 

implemented: in the version used in the experiment, the out-the-window view and dashboard 

displays were blanked. Only the communications window remains active so that the 

participant can communicate with the experimenter. In the other version, both screens are 

just frozen but not blanked. One can invoke the later version by using the -noblank flag in

the command line argument when running the train simulation. In any case, provision has 

been made for the simulator to record everything pertaining to the state of the train as well as 

the participant’s response. Additional provision was made to reset the simulation clock. 

• Enable the experimenter to activate/deactivate Automatic Train Protection from the CTC 

screen. This option can be invoked using the -atp flag when running the CTC simulation. 

• A new version of the autopilot. In the previous version of the simulator, the full supervisory 

control mode (autopilot), once invoked, would control the speed of the train adhering only to 

civil speed limits. When a signal was present the autopilot would fail to recognize it and the 

vehicle operator needed to assume manual control to adjust the vehicle speed. In the current 

version, the full supervisory control mode would recognize civil and signal speed limits 

simultaneously and adhere to the lesser of the two, thus replicating existing train supervisory 

control mode such as the one used in the German ICE high speed train. Under normal 

conditions, the operator need not interfere with speed control of the train but only monitor 

that the full supervisory control mode adheres to the speed limits of each block. This version 

of the full supervisory control mode can be realized using the -full_auto flag in the 

command line argument when running the train simulation. 

• A bearing failure. In this scenario, the temperature of a faulty bearing rises due to increased 

friction in its rollers. The rate at which the temperature goes up under failure conditions is 

higher than under normal conditions. 

• A failure reset mode to gather vigilance data. In older versions of the simulator, once the 

failures were set they remained active until the participant noticed them. In the current 

version, the software can reset the failures according to the time that elapsed or the distance 

that the train traveled since the onset of the failure, in the event that the participant has not 

been able to spot and reset them in time. This mode was selected using the -fail_reset

option in the command line argument of the main simulation. The programmer must enter the 

keywords time followed by the time in milliseconds the failure will remain active, or 

distance followed by the distance that the vehicle will travel before the failure is reset in 

the failure input file. 

29

 

APPENDIX C. LOCATION OF FAILURE EVENTS

The positions on the track at which failure events were set to occur are shown below. 

Distance from station (miles) Direction Time block 

Motor Failure 

8.2 Eastbound 1st

25.2 Eastbound 2nd

19.0 Westbound 3rd

Bearing failure 

20.7 Eastbound 1st

1.9 Westbound 2nd

16.2 Westbound 3rd

Grading crossing 

0.9 Eastbound - 

13.3 Eastbound - 

14.1 Eastbound - 

14.0 Westbound - 

16.5 Westbound - 

29.9 Westbound - 

31

 

APPENDIX D. VIGILANCE DETECTION TASK DATA 

Subject 1 

Control mode Task Response Reaction 

Full Supervisory m1

b3 

o6 

hit 

miss 

No collision 

8.205 s 

- 

Stop 299.27 ft

Partial 

Supervisory

m2

b3 

o5 

hit 

hit 

No collision 

2.28 s 

5.595 s 

Stop 262.92 ft

Manual b1

b2 

m3

o4 

hit 

hit 

hit 

No collision 

8.22 s 

15 s 

1.981 s 

Stop 607.18 ft

Subject 2 

Control mode Task Response Reaction 

Full Supervisory b1 

b2 

m3

o1 

hit 

hit 

hit 

No collision 

7.206 s 

13.194 s 

3 

Stop 217.74 ft

Partial 

Supervisory

b2 

m3

o3 

miss 

hit 

No collision 

- 

3.1 

Stop 292.68 ft

Manual m1

b3 

o6 

hit 

miss 

No collision 

9.48 

- 

Stop 261.15

Subject 3 

Control mode Task Response Reaction 

Full Supervisory b2 

m3

o3 

hit 

hit 

No collision 

9.86 s 

8.7 s 

Stop 459.31 ft

Partial Supervisory m1

b3 

o6 

hit 

hit 

No collision 

5.45 s 

13.1 s 

Stop 86.77 ft 

Manual b1

m2

b3 

o1 

hit 

miss 

hit 

No collision 

12.85 s 

- 

15.7 s 

645.56 ft 

Legend 

b = bearing failure s = seconds 

m = motor failure ft = feet

o = obstruction 

33
 

Subject 4 

Control mode Task Response Reaction 

Full Supervisory b2 

m3

o3 

miss 

hit 

No collision 

- 

7.38 s 

Stop 365.81 ft

Partial 

Supervisory

m1

m3

o6 

hit 

miss 

No collision 

2.945 s 

- 

Stop 249.90 ft

Manual b1

m2

b3 

hit 

miss 

miss 

6.364 s 

- 

- 

Subject 5 

Control mode Task Response Reaction 

Full Supervisory b1 

m2

o5 

hit 

hit 

No collision 

12.98 s 

3.54 s 

Stop 14.43 ft 

Partial 

Supervisory

m1

m3

o6 

miss 

hit 

No collision 

- 

3.06 s 

Stop 12.17 ft 

Manual b1

m2

b3 

04 

hit 

hit 

hit 

No collision 

3.46 s 

3.66 s 

17.6 s 

Stop 310.95 ft

Subject 6 

Control mode Task Response Reaction 

Full Supervisory b1 

m2

b3 

o5 

hit 

miss 

hit 

No collision 

15.59 s 

- 

15.74

Stop 123.58 ft

Partial 

Supervisory

m2

b1 

o6 

hit 

miss 

No collision 

7.8 s 

- 

Stop 55.77 ft 

Manual b1

m2

o1 

hit 

hit 

No collision 

8.154 s 

6.24 s 

Stop 229.95 ft

Legend

b = bearing failure s = seconds 

m = motor failure ft = feet

o = obstruction 

34
 

Subject 7 

Control mode Task Response Reaction 

Full Supervisory b1 

m2

o5 

hit 

hit 

No collision 

16.06 s 

3.13 s 

Stop 89.17 ft 

Partial 

Supervisory

m1

b2 

o6 

hit 

hit 

No collision 

2.465 s 

17.132 s 

292.61 ft 

Manual b2

b3 

o1 

hit 

hit 

No collision 

16.87 s 

19.68 s 

Stop 283.46 ft

Subject 8 

Control mode Task Response Reaction 

Full Supervisory m1

b2 

o5 

miss 

hit 

No collision 

- 

15.6 s 

Stop 185.30 ft

Partial 

Supervisory

m1

b2 

m3

o6 

miss 

miss 

hit 

No collision 

- 

- 

3.48 s 

171.1 m

Manual m1

m3

o1 

hit 

hit 

No collision 

10.8 s 

7.2 s 

Stop 225.81 ft

Subject 9 

Control mode Task Response Reaction 

Full Supervisory m1

b3 

o6 

hit 

hit 

collision 

3.545 s 

17.405 s 

- 

Partial 

Supervisory

m2

b3 

o5 

hit 

miss 

No collision 

6.481 s 

- 

Stop 518.37 ft

Manual m1

b2 

m3

o1 

miss 

miss 

hit 

No collision 

- 

- 

s 

Stop 435.23 ft

Legend

b = bearing failure s = seconds 

m = motor failure ft = feet

o = obstruction 

35
 

Subject 10 

Control mode Task Response Reaction 

Full Supervisory b2 

m3

hit 

hit 

16.2 s 

3 s 

Partial 

Supervisory

b1 

m2

b3 

o5 

hit 

miss 

miss 

collision 

7.32 s 

- 

- 

- 

Manual m1

b2 

o6 

hit 

hit 

No collision 

3.125 s 

13.8 s 

Stop 163.66 m 

Subject 11 

Control mode Task Response Reaction 

Full Supervisory m1

m3

o5 

hit 

miss 

No collision 

3.125 s 

- 

Stop 95.75 m 

Partial 

Supervisory

b1 

m2

b3 

o2 

hit 

hit 

hit 

collision 

8.288 s 

4.51 s 

11.04 s 

Stop 62.79 m 

Manual m2

m3

o4 

hit 

miss 

No collision 

3.361 s 

- 

Stop 125.44 m 

Subject 12 

Control mode Task Response Reaction 

Full Supervisory b1 

m2

b3 

o4 

hit 

hit 

hit 

No collision 

12.91 s 

4.4 s 

15.34 s 

Stop 134.88 m 

Partial 

Supervisory

m1

b2 

m3

o5 

hit 

hit 

miss 

No collision 

2.9 s 

7.261 s 

- 

Stop at 142.92 m

Manual b2

o3 

hit 

No collision 

8.4 s 

Stop 199.39 m 

Legend

b = bearing failure s = seconds 

m = motor failure ft = feet

o = obstruction 

36
 

APPENDIX E. DATA FOR PAGE TEST 

Differences between the participants' speed estimates and the actual speed were used to compute 

the Mean Absolute Error (MAE) in the speed category. The numbers in parentheses indicate the 

ranks as required by the test. Details on how the ranks are used by the test can be found in 

Hollander and Wolfe (1973), page 147. 

ParticipantSpeed Deviation (mph) by Control Modes 

Full

Supervisory 

Partial 

Supervisory Manual 

1 1.9 (3) 1.1 (1) 1.4 (2) 

2 17.4 (3) 2.9 (1) 16.8 (2) 

3 13.0 (3) 3.1 (2) 2.8 (1) 

4 14.9 (3) 4.1 (1) 5.6 (2) 

5 1.2 (1) 10.6 (3) 6.2 (2) 

6 5.3 (3) 1.4 (2) 0.2 (1) 

7 3.1 (2) 1.2 (1) 4.7 (3) 

8 10.6 (3) 1 (1) 1.3 (2) 

9 6.2 (1) 9.3 (3) 8.7 (2) 

10 9.3 (3) 5.5 (1) 9.3 (2) 

11 6.0 (3) 0.4 (1) 2.3 (2) 

12 3.2 (1) 6.8 (2) 8.7 (3) 

Differences between the participants' temperature estimates and the actual temperature were used 

to compute the MAE in the temperature category. The numbers in parentheses indicate the ranks 

as required by the test. Details on how the ranks are used by the test can be found in Hollander 

and Wolfe (1973), page 147. 

ParticipantTemperature Deviation (o

F) by Control Mode 

Full Supervisory Partial Supervisory Manual 

1 39 (1) 44 (2) 52 (3) 

2 39 (1) 40 (2) 55 (3) 

3 37 (1) 38 (2) 50 (3) 

4 40 (3) 34 (1) 38 (2) 

5 33 (1) 35 (2) 40 (3) 

6 37 (1) 37 (2) 37 (3) 

7 33 (2) 35 (3) 32 (1) 

8 50 (3) 37 (1) 44 (2) 

9 33 (1) 37 (3) 35 (2) 

10 34 (2) 33 (1) 34 (3) 

11 35 (1) 41 (2) 42 (3) 

12 35 (1) 45 (3) 38 (2) 

37

 

REFERENCES 

Askey, S.Y. (1995). Design and Evaluation of Decision Aids for Control of High-Speed Trains: 

Experiments and Model. Ph.D. thesis, Massachusetts Institute of Technology. 

Billings, C.E. (1997). Aviation Automation: The Search for a Human Centered Approach. 

Mahwah, NJ: Lawrence Erlbaum Associates 

Cinlar, E. (1975). Introduction to Stochastic Processes. Englewood Cliffs, New Jersey: PrenticeHall. 

Cinlar, E. (1975). Markov Renewal Theory: A Survey. Management Science, 21, 727-752. 

Dember, W.N. and Warm, J.S. (1979). Psychology of Perception. (2nd ed). New York: Holt, 

Rinehart & Winston. 

Drake, A. W. (1967). Fundamentals of Applied Probability Theory. New York: McGraw-Hill,. 

Egan, J.P. (1975). Signal Detection Theory and ROC Analysis. New York: Academic Press. 

Endsley, M. (1993). Situation awareness in dynamic human decision-making: Theory.

Proceedings at the first International Conference on Situational Awareness in Complex 

Systems. Orlando, Florida. 

Endsley, M. (1995a). Toward a Theory of Situation Awareness in Dynamic Systems. Human 

Factors, 37(1), 32-64. 

Federal Railroad Administration. (2002). A Vision for the Future: Intelligent Railroad Systems 

Five-Year Strategic Plan for Railroad Research, Development, and Demonstrations. Report 

No. FRA/RDV-02/02 Washington, DC: U.S. Department of Transportation. 

Endsley, M. (1995b). Measurement of Situation Awareness in Dynamic Systems. Human 

Factors, 37(1), 65-84. 

Gallagher, R. G. (1996). Discrete Stochastic Processes. Kluwer Academic Publishers.

Getty, D.J., Swets, J.A., Pickett, R.M. and Gonthier, D. (1995). System operator response to 

warnings of danger: a laboratory investigation of the effects of the predictive value of a 

warning on human response time. Journal of Experimental Psychology: Applied, Vol. 1(1),

19-33. 

Green, D. M. and Swets, J. A. (1966). Signal detection theory and psychophysics. New York: 

Wiley & Sons. 

Grier, J.B. (1971). Nonparametric Indexes for Sensitivity and Bias: Computing Formulas. 

Psychological Bulletin, 75 (6), 424-429. 

Huey, B.M. and Wickens, C.D. (eds.) (1993) Workload Transition. Washington, DC: National 

Academy Press. 

Hollander, M. and Wolfe, D. (1973). Nonparametric Statistical Methods. New York: Wiley & 

Sons. 

39
 

Lanzilotta, E.J. (1995). Dynamic Risk Estimation: Development of the Safety State Model and 

Application to High-Speed Rail Operation. Ph.D. thesis, Massachusetts Institute of

Technology. 

Loeb, M. and Binford, J. (1970). Examination of some factors influencing performance on an 

auditory monitoring task with one signal per session. Journal of Experimental Psychology, 83 

(1), 40-44. 

Mackworth, N.H. (1961). Researches on the measurement of human performance. In H.W.

Sinako (Ed), Selected papers on human factors in the design and use of control systems. New 

York: Dover (Reprinted from Medical Research Council Special Report Series 268). (1950). 

London: His Majesty's Stationary Office. 

Macmillan, N.A. and Creelman, C. D. (1991). Detection Theory: A user’s guide. Cambridge, 

England, Cambridge University Press. 

Medhi, J. (1982). Stochastic Processes. New York: Wiley & Sons. 

Molloy, R. and Parasuraman, R. (1996). Monitoring an Automated System for a Single Failure: 

Vigilance and Task Complexity Effects. Human Factors, 38 (2) 311-322. 

Parasuraman, R. (1979). Memory load and event rate control sensitivity decrements in sustained 

attention. Science. 205, 724-727. 

Parasuraman, R. and Davies, D. R. (1977). A taxonomic analysis of vigilance performance. In 

Mackie, R. R. Ed., Vigilance: Theory, Operational Performance and Physiological 

Correlates, Plenum, New York. 

Parasuraman, R. and Mouloua, M. (1987). Interaction of signal discriminability and task type in 

vigilance decrement. Perception and Psychophysics, 41 (1), 17-22. 

Ross, S. (1983). Stochastic Processes. New York: Wiley & Sons. 

Sheridan, T. (1992). Telerobotics. Automation and Human Supervisory Control. MIT Press. 

Sheridan, T. and Ferrell, W (1974). Man-machine systems. MIT Press. 

Sheridan, T. Lanzilotta, E. Askey, S.Y. (1994). Safety of High-Speed Guided Ground 

Transportation Systems. Human Factors. Phase I: Functional Analyses and Theoretical 

Considerations. Washington, DC: U.S. Department of Transportation, Federal Railroad 

Administration. 

Stanton, N.A. and Young, M.S. (1998). Vehicle automation and driving performance. 

Ergonomics. 41 (7), 1014-1028. 

Swets, J. A. (1964). Signal Detection and Recognition by Human Observers. New York: Wiley. 

Thomas, G., Raslear, T. and Kuehn, G.I. (1997). The Effects of Work Schedule on Train 

Handling Performance and Sleep of Locomotive Engineer: A Simulator Study. Report No. 

DOT/FRA/ORD-97/09. Washington, DC: U.S. Department of Transportation, Federal 

Railroad Administration. 

Wickens, C.D. and Kessel, C. (1979). The effects of participatory mode and task workload on 

the detection of dynamic system failures. IEEE Transactions on systems, Man and 

Cybernetics. SMC-9(1), 24-34. 

40