Scaling Robotic Displays: Visual and Multimodal Options 

for Navigation by Dismounted Soldiers 

by Elizabeth S. Redden, Rodger A. Pettitt, Christian B. Carstens, 

Linda R. Elliott, and Dave Rudnick 

ARL-TR-4708 January 2009 

Approved for public release; distribution unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position 

unless so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or 

approval of the use thereof. 

Destroy this report when it is no longer needed. Do not return it to the originator. 
Army Research Laboratory 

Aberdeen Proving Ground, MD 21005-5425 

ARL-TR-4708 January 2009 

Scaling Robotic Displays: Visual and Multimodal Options 

for Navigation by Dismounted Soldiers 

Elizabeth S. Redden, Rodger A. Pettitt, Christian B. Carstens, 

and Linda R. Elliott 

Human Research & Engineering Directorate, ARL 

and 

Dave Rudnick 

DCS Corporation 

Approved for public release; distribution unlimited. 
 

ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing 

data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate 

or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, 

Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be 

aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not 

display a currently valid OMB control number. 

1. REPORT DATE (DD-MM-YYYY)

January 2009 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

Scaling Robotic Displays: Visual and Multimodal Options for Navigation by 

Dismounted Soldiers 

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

6. AUTHOR(S) 

Elizabeth S. Redden, Rodger A. Pettitt, Christian B. Carstens, Linda R. Elliott, 

and Dave Rudnick 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

ATTN: AMSRD-ARL-HR-MW 

Aberdeen Proving Ground, MD 21005-5425

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-4708

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

This study investigates the impact of three types of navigation map displays on navigation performance, using Soldiers from 

the Officer Candidate School (OCS) at Fort Benning, GA. After training to use the TALON Robot system, each Soldier 

completed navigation exercises using three navigation display configurations: a 6.5 in. split screen display with a driving 

display on top and a map display on bottom, allowing simultaneous or near simultaneous viewing; a 3.5 in. display in which 

the Soldier could toggle between the driving and map displays; and a multimodal 3.5 in. display using a tactile belt that 

transmitted directional information to the Soldier concurrently with the driving camera display (Soldiers could also toggle to 

the map display to determine TALON’s specific location). The terrain, targets, and hazards were counterbalanced to control 

for the effect of learning. Display configuration and usability for robotic driving were evaluated based on objective 

performance data, data collector observations, and Soldier questionnaires. Findings indicated that Soldiers navigated equally 

effectively using the multimodal 3.5 in. and 6.5 in. split screen displays. Their performance with both the multimodal and 

split screen displays was better than with the 3.5 in. display that required toggling between the driving and map displays. 

15. SUBJECT TERMS

Tele-operation; display scalability; robotic navigation display; tactile navigation display; dismounted Soldiers; multimodal 

display; robotic performance 

16. SECURITY CLASSIFICATION OF:

19a. NAME OF RESPONSIBLE PERSON

Elizabeth S. Redden 

a. REPORT 

U 

b. ABSTRACT 

U 

c. THIS PAGE 

U 

17. LIMITATION 

OF 

ABSTRACT 

UU 

18. NUMBER

OF 

PAGES 

58 19b. TELEPHONE NUMBER (Include area code) 

706-545-5493 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18 
 

iii 

Contents 

List of Figures v

List of Tables v

Acknowledgments vi

1. Introduction 1

1.1 Background .....................................................................................................................1 

1.1.1 Display Size.........................................................................................................2 

1.1.2 Navigation Displays ............................................................................................3 

1.2 Overview of Experiment .................................................................................................4 

1.3 Objective .........................................................................................................................5 

2. Method 5

2.1 Participants ......................................................................................................................5 

2.2 Instruments and Apparatus..............................................................................................5 

2.2.1 TALON Robot.....................................................................................................5 

2.2.2 Robotic Vehicle Displays....................................................................................6 

2.2.3 Tactile Belt ..........................................................................................................8 

2.2.4 Tablet Display .....................................................................................................9 

2.2.5 Robotic Navigation Course ...............................................................................10 

2.2.6 Motion Sickness Questionnaire.........................................................................12 

2.2.7 Soldier Subjective Questionnaires.....................................................................12 

2.2.8 NASA-TLX .......................................................................................................12 

2.2.9 SynWin..............................................................................................................12 

2.3 Procedures .....................................................................................................................13 

2.3.1 Demographics....................................................................................................13 

2.3.2 Training .............................................................................................................13 

2.3.3 Teleoperation Trials ..........................................................................................14 

2.3.4 SynWin..............................................................................................................15 

3. Results 15

3.1 Demographics................................................................................................................15 
 

iv 

3.2 Training Evaluation.......................................................................................................15 

3.3 Performance Data..........................................................................................................16 

3.3.1 Course Times.....................................................................................................16 

3.3.2 Situation Awareness..........................................................................................18 

3.3.3 Driving Errors....................................................................................................19 

3.3.4 SynWin..............................................................................................................19 

3.3.5 NASA-TLX .......................................................................................................20 

3.4 Subjective Results .........................................................................................................20 

3.4.1 Post-iteration Questionnaire..............................................................................20 

3.4.2 End of Experiment Questionnaire .....................................................................21 

3.5 Engineering Lessons Learned .......................................................................................22 

4. Discussion 23

5. Recommendations 25

6. References 26

Appendix. Soldier Questionnaire Results 29

Acronyms 48 

Distribution List 49 
 

v 

List of Figures 

Figure 1. TALON robot. .................................................................................................................6 

Figure 2. Display A, split screen.....................................................................................................7 

Figure 3. Driving camera scene for Displays B and C. ..................................................................7 

Figure 4. Map display scene for Displays B and C.........................................................................8 

Figure 5. TACTICS tactile belt system...........................................................................................9 

Figure 6. Display.............................................................................................................................9 

Figure 7. Robotic navigation course. ............................................................................................10 

Figure 8. Marked path from first leg of robotic navigation course...............................................11 

Figure 9. Complex obstacle. .........................................................................................................11 

Figure 10. Presentation of hand signals. .......................................................................................11 

Figure 11. SynWin tasks presented in the four quadrants. ...........................................................13 

Figure 12. Course completion times (sec), all display conditions. ...............................................17 

Figure 13. Log10 course completion times (sec), all display conditions......................................17 

Figure 14. NASA-TLX means by display condition. ...................................................................20 

Figure 15. Paired display preferences...........................................................................................22 

List of Tables 

Table 1. Order of treatments and lanes. ........................................................................................14 

Table 2. Mean course completion times. ......................................................................................16

Table 3. Log10 mean course completion times. ...........................................................................18 

Table 4. Planned comparisons, course completion times. ............................................................18 

Table 5. Mean percentage of hand signal identified.....................................................................19 

Table 6. Planned comparisons, percentage of hand signal identified...........................................19 

Table 7. Mean number of driving errors.......................................................................................19 

Table 8. SynWin subtest and composite means............................................................................20 
 

vi 

Acknowledgments 

We wish to thank Michael J. Barnes of the U.S. Army Research Laboratory’s Human Research 

and Engineering Directorate for his support and guidance throughout the process of preparing 

this report. We would also like to thank Terry Tierney and Larry Sieh from Tank and 

Automotive Research, Development and Engineering Center (TARDEC); Charlie Dean and 

Shawn Thayer from Foster-Miller; and Dave Rudnick from DCS Corporation. Their assistance 

in obtaining and maintaining the robotic equipment and displays used during this effort was key 

to the successful completion of the experiment. DCS and TARDEC also implemented the user 

interface and protocols for all the aids required in this experiment. We would like to express our 

appreciation to Shawn Stafford, Julian Abich, and Gretchen Knaack from the University of 

Central Florida (UCF), SSG Brian Burwell from Headquarters Company 1/11th Infantry 

Regiment, and Mr. Richard Ivey from the Soldier Battle Lab for their assistance with data 

collection. We particularly thank the Soldier-participants, who applied their experience and 

capability diligently, and who shared many valuable insights throughout the series of events. 
 

1 

1. Introduction 

1.1 Background 

This is the third in a series of experiments designed to investigate how best to scale robot 

controls and displays for dismounted Soldiers who need smaller and lighter devices. The first 

two experiments in this series addressed screen size for the dismounted Soldier’s driving camera 

display (Redden, Pettitt, Carstens, and Elliott, 2008) and controller options for the dismounted 

Soldier to drive the robot and maneuver the robotic arm (Pettitt, Redden, Carstens, and Elliott, 

2008). The environments of dismounted Soldiers are rugged and physically demanding, and 

Soldiers must carry their robotic operator control units (OCU) along with all their protective and 

fighting equipment in these challenging environments. Relatively large displays typically used 

in a stationary environment or inside a combat vehicle are not appropriate and could have an 

adverse impact on the dismounted Soldiers’ missions. Scaling robotic interfaces involves the 

design and development of smaller, lighter versions that are still rugged, easy to use, easy to 

learn, and easy to maintain. Scaling ensures that training transfer is easy across environments 

and that interfaces are tailored to the environment in which they are used. The key to successful 

scaling is to consider the range of devices that Soldiers will use (e.g., vehicle-mounted robot 

control devices, other controller devices) and also their context of use. A smaller controller may 

be easy to learn and use if it is similar to existing controllers. On the other hand, some controller 

characteristics will not be as effective in a smaller unit. Consider the increased difficulty of 

typing on a QWERTY keyboard on a cell phone compared to a computer keyboard. Context of 

use also becomes a factor, when Soldiers must use their displays in rough terrain, in bright 

daylight, or perhaps, while on the move. Trade-offs in controller options for different task 

demands must be recognized and considered. Ultimately, scaling depends on user evaluations 

and experimental controlled investigations under realistic task demands. 

Several past studies have found no performance differences created by reduced display size 

(Alexander, Wickens, and Hardy, 2005; Minkov, Perry and Oron-Gilad, 2007; Muthard and 

Wickens, 2004; Redden et al. 2008; Stark, Comstock, Prinzel, Burdette, and Scerbo, 2001; 

Stelzer and Wickens, 2006). However, there were conflicting results with some investigations 

indicating a benefit of larger displays. In addition, previous experiments were conducted for a 

variety of task demands, some of which were not directly generalizable to the dismounted robot 

control situation. Redden et al. (2008) conducted an experiment to evaluate display size options 

in a manner that maximizes external generalizability—experienced Soldiers used real control 

unit options to accomplish typical dismount controller tasks. That study indicated that smaller 

displays (i.e., 3.5-in. and 6.5-in. diagonal displays) were as effective for teleoperation and local 

surveillance with small, slower speed robots as a 10.5 in. diagonal display and a goggle-mounted 

display. The finding that the camera display can be smaller allows more screen real estate to be 
 

2 

made available for other purposes (i.e., map displays, menus, etc.) and enables the use of lower 

resolution cameras, which would allow faster video processing with less bandwidth requirement. 

It is important to note that the findings of this experiment were limited to camera displays for 

driving small robots and performing local surveillance; they did not address other tasks or other 

types of displays (e.g., map displays, touch screens, etc.). Additional tasks may require different, 

additional, or larger displays. 

This experiment builds upon the Redden et al. (2008) display size experiment by addressing an 

additional task that is fundamental to dismounted Soldiers—that of navigation. Many existing 

robot control units provide a combination of a camera display and a map display to enable 

operators to navigate to locations that are out of line-of-sight. These displays are typically split 

screen displays. In this study, we compare the split screen display option with the option of 

toggling between the camera and map displays, which requires less screen size. We also explore 

the utility of tactile direction cues to support navigation and obstacle avoidance when using the 

camera view. Tactile cues may minimize the need for toggling between screens and enable the 

operator to maintain situation awareness (SA) and facilitate navigation around obstacles. 

1.1.1 Display Size 

The finding that a 3.5-in. diagonal display is as effective as larger displays for driving a small 

robot and for detection of threats in close proximity to the robot is appealing because smaller 

display sizes can potentially reduce the overall OCU size requirement for control of unmanned 

vehicles as well as the weight of this equipment. In addition, more space on the OCU can be 

allocated for displaying menus, maps, and representations of vehicle status. Although a 3.5-in. 

diagonal display was found to be sufficient for remote driving with a small robot, it may not be 

adequate for navigating a small robot using a map display because display size has been found to 

affect performance and workload differently, depending on the task and the conditions. For 

example, research with pilots (Wickens, Muthard, Alexander, van Olffen, and Podczerwinski, 

2003) found that display size had no impact on overall surveillance, change detection, tracking, 

or response time when using complex and dynamic aerial map displays with icons. In a 

subsequent study using a low-fidelity two-dimensional (2-D) and three-dimensional (3-D) 

tracking task, Stelzer and Wickens (2006) found that performance was degraded when urgency 

was high and displays were smaller. They also compared effects of display size during 

surveillance and target search tasks using complex and dynamic aerial map display icons. They 

found that smaller displays were associated with a higher rate of error in heading and altitude, as 

well as increased deviations from flight path. However, display size did not affect response time 

or ability to estimate distance between aircraft, detect changes, or maintain target distance from 

the lead aircraft. These findings indicate the need to determine whether smaller displays can be 

used effectively by dismounted Soldiers performing a variety of tasks such as land navigation. 
 

3 

1.1.2 Navigation Displays 

Optimal display design may depend on whether specific task groupings are performed 

simultaneously/near-simultaneously or sequentially. It is unclear whether driving (e.g., camerabased steering, control, and obstacle avoidance) and navigating (e.g., map-based progress from 

waypoint to waypoint) are tasks that are very closely coupled or if they can be performed in an 

interactive but sequential manner. Perceptual and cognitive requirements associated with these 

tasks have an impact on display design. If the tasks can be performed sequentially, then the 

same display real estate can be used for both tasks by having the operator toggle between driving 

camera and map displays. If the tasks can be performed simultaneously or even near 

simultaneously, then two different displays would likely make driving and navigating more 

efficient. Minkov, Perry, and Oron-Gilan (2007) used split screen displays for unmanned aerial 

vehicle (UAV) navigation and monitoring. However, the operators in this experiment only 

monitored the UAV mounted camera while they navigated and they did not actually fly/drive the 

UAV. If both driving and navigation displays must be provided together, and the driving display 

is 3.5 in. in diagonal, then simultaneous display of the driving camera information and 

navigational map information will require display space larger than 3.5 in. 

While the toggle option would allow for a smaller size display, it is likely that Soldiers will 

experience higher manual and cognitive workload. If the task is very dynamic, such as when 

ascertaining and avoiding obstacles, Soldiers will have to toggle back and forth frequently to 

adjust their position while keeping visual SA through the camera display. During the task, the 

Soldier must keep the alternate screen information in his visual memory. In contrast, the split 

screen allows the Soldier to process peripheral visual information, thus allowing multi-tasking 

with reduced workload (Wickens, 2002). This leads to our first three hypotheses: 

• HO1. Soldiers will perform the robot navigation task more quickly with the split screen 

than with the toggle display. 

• HO2. Soldiers will perform the robot navigation task with fewer errors with the split 

screen than with the toggle display. 

• HO3. Soldiers will identify hand signals more quickly and accurately with the split screen 

than with the toggle display. 

An alternative approach for providing two displays would be to supply a visual display for 

driving and reconnaissance, and a tactile display to support navigation. This would enable the 

reduction of screen size to 3.5 in. while providing both navigation and camera information. 

According to Multiple Resource Theory (MRT), this approach could reduce the burden on the 

visual channel and could be even more efficient than providing two visual displays via a split 

screen (Wickens, 2002), depending on task demands and overall workload. Also, torso-mounted 

tactile arrays have been used successfully for personal land navigation (Dorneich, Ververs, 

Whitlow, and Mathan, 2006; Duistermaat, Elliott, van Erp, and Redden, 2007; Elliott, Redden, 
 

4 

Pettitt, Carstens, van Erp, and Duistermaat, 2006; Gilson, Redden and Elliott, 2007; Van Erp, 

2007, 2005). Elliott, Duistermaat, Redden, and van Erp (2007) found that Soldiers using a 

helmet-mounted display to navigate at night simultaneously with cross-country movement found 

fewer enemy targets and took longer to navigate the terrain than Soldiers who used a tactile 

navigation device while moving cross-country. Displaying navigation information to a different 

information channel allowed simultaneous navigation and cross-country movement without 

visual overload. In this study condition, torso-mounted tactile cues would provide direction 

guidance toward the next waypoint in a manner that enables the Soldier to head toward the next 

waypoint, while using the camera display to drive the robot and negotiate obstacles. Because the 

information is presented simultaneously, we expect the multimodal display to be as effective as 

the split screen and more effective than the toggle screen: 

• HO4. Soldiers will perform the robot navigation task more quickly with the multimodal 

display than with the toggle display. There will be no difference in navigation times 

between the multimodal and the split screen displays. 

• HO5. Soldiers will perform the robot navigation task with fewer errors with the 

multimodal display than with the toggle display. There will be no difference in Soldier 

driving performance between the multimodal and the split screen displays. 

• HO6. Soldiers will identify hand signals more quickly and accurately with the multimodal 

display than with the toggle display. There will be no difference in Soldier performance 

between the multimodal and the split screen displays. 

1.2 Overview of Experiment 

This study investigated the effect of visual and multimodal display options on driving, 

performing local surveillance, and navigating a small robotic vehicle. It took place at Fort 

Benning, GA, and used Soldiers from the Officer Candidate School (OCS). After training on the 

operation of the TALON system, each Soldier completed exercises using three different display 

concepts. The first concept was a 6.5-in. split screen display that presented camera-based 

driving information on the top portion of the display and the map display on the bottom. The 

second concept consisted of a 3.5-in. display that required the Soldier to toggle between the 

driving camera and map displays. The third concept was a 3.5-in. display that was used 

primarily for driving, while directional information was provided to the Soldier via a tactile belt. 

If the Soldier needed to see the map display to find distance information or landmark 

information, he could toggle the visual display from the driving camera scene to the map display. 

The terrain, targets, and hazards were counterbalanced along with display condition to control 

for order effects, such as practice, learning, boredom, or fatigue. The display options were 

evaluated for effects on task performance, workload, and usability, based on objective 

performance data, data collector observations, and Soldier questionnaires. 
 

5 

1.3 Objective 

The objective of this experiment was to assess the effectiveness of a toggle-based 3.5 in. display 

and the multimodal 3.5 in. display, as compared to a 6.5 in. split screen display. 

Performance measures addressed the following: 

• How does the use of the different display conditions affect the number of driving errors? 

• How does the use of the different display conditions affect the overall course completion 

times? 

• How does the use of the different display conditions affect the SA of the operator around 

the vehicle? 

2. Method 

2.1 Participants 

Thirty-three Soldiers from the Fort Benning OCS volunteered to participate in the study. These 

Soldiers had experience as enlisted Soldiers and came from varied military occupational 

specialties. 

2.2 Instruments and Apparatus 

2.2.1 TALON Robot 

The TALON is a lightweight robot designed for missions ranging from reconnaissance to 

weapons delivery (figure 1). Built with all-weather, day/night, and amphibious capabilities, the 

TALON can operate during adverse conditions over almost any terrain. The suitcase-portable 

robot is controlled through a two-way radio frequency line from a portable OCU that provides 

continuous data and video feedback for precise vehicle positioning. It was developed for the 

Explosive Ordnance Disposal Technology Directorate of the U.S. Army’s Armament Research, 

Development, and Engineering Center at Picatinny Arsenal, NJ, by the engineering and 

technology development firm of Foster-Miller. The TALON began being used in military 

operations in Bosnia in 2000. It was deployed to Afghanistan in early 2002 and has been in Iraq 

since the war started, assisting with improvised explosive device detection and removal. 
 

6 

Figure 1. TALON robot. 

For this experiment, the TALON was equipped with a video camera that enabled Soldiers to 

maneuver the vehicle and assess enemy activity along the route to the objective. It was also 

equipped with a global positioning system (GPS) that informed the navigation display of its 

position and orientation. 

2.2.2 Robotic Vehicle Displays 

DCS Corporation and Tank Automotive Research Development and Engineering Center 

(TARDEC) implemented the user interface and protocols for all the aids required in this 

experiment. Three different display conditions were used to conduct this experiment. Each 

display condition was plugged into the existing TALON control system so that everything 

remained constant except the displays. 

Display A was a 6.5-in. display that provided a split screen capability. The top portion of the 

screen presented the driving camera information and the bottom portion presented the map 

display (figure 2). Both displays could be viewed simultaneously. 
 

7 

Figure 2. Display A, split screen. 

Display B was a 3.5-in. display that could present either the driving camera (figure 3) or the map 

(figure 4). Soldiers toggled between the displays as needed by pushing a single toggle button on 

the controls. The displays had to be viewed sequentially. 

Figure 3. Driving camera scene for Displays B and C. 
 

8 

Figure 4. Map display scene for Displays B and C. 

Display C was the same 3.5-in. display used for Display B (figures 3 and 4) with the addition of 

a tactile belt (see paragraph 2.2.3 for a description of the belt). The driving camera scene was 

used primarily, but the Soldier could toggle to the map display when specific location 

information was needed. 

2.2.3 Tactile Belt 

The TACTICS tactile system was developed by the University of Central Florida (UCF) under 

Defense Advanced Research Project Agency contract number DAAE0703CL143 (figure 5). It is 

capable of remotely conveying covert signals, cues, and messages by touch. The system for this 

experiment consisted of a tactile display worn around the waist with a receiver unit. The display 

itself consisted of eight tactile drivers that created a strong localized sensation on the body and 

worked similar to a plunger. The eight tactors corresponded to the eight cardinal directions. The 

tactors could be activated individually, sequentially, or in groups; and the duration and signal 

frequency could be varied to provide a specific sensation or to create unique patterns of 

vibration. The control unit received wireless signals from the GPS onboard the TALON and 

converted them into recognizable patterns of vibration. When the TALON was going in the 

correct direction, the tactor at the front of the Soldier’s waist activated (200 ms on and 1800 ms 

off). Thus, the object was to drive the robot in the direction that kept the front tactor active. 

When the TALON was within 5 m of the waypoint, the tactor’s pulse increased to 100 ms on, 

200 ms off, 100 ms on, 600 ms off. When it moved to within 2 m of the waypoint, the waypoint 

was considered to be achieved and all the tactors activated for 3000 ms. If a Soldier went off 

course more than 2 m, the tactor that corresponded with a “steer to” direction for returning to the 

correct course route activated at 200 ms on and 1800 ms off. 
 

9 

Figure 5. TACTICS tactile belt system. 

2.2.4 Tablet Display 

The visual display that was used to present the 3.5 in. map and camera display and the split 

screen display was the AMREL Rocky DR7-M (figure 6). It is 9.8 in. by 7.4 in. and weighs 

2.4 lb (including the battery) and has GPS and Bluetooth integrated options. It is a 1024 by 768 

extended graphics array (XGA). 

Figure 6. Display. 
 

10 

2.2.5 Robotic Navigation Course 

The training portion of the course consisted of a lane that allowed the Soldier to practice 

teleoperating the robot around a complex obstacle in each direction using the driving camera and 

control unit. It also consisted of another section with waypoints that allowed the Soldier to 

navigate the robot to the waypoints using each of the navigation system configurations. The 

experiment course consisted of three different lanes. This allowed the Soldier to drive on a 

different lane with each display condition. The total length of each lane was approximately 

100 m (figure 7). The first leg of each lane was a marked path approximately 30 m long 

(figure 8). A complex obstacle was placed at the end of the marked path that required the 

operator to navigate around it (figure 9). The obstacle did not show up on the map display so the 

operator was required to drive around the obstacle and continue to the end waypoint (objective). 

The second leg of the course required the operator to drive as quickly and efficiently as possible 

using the GPS feedback to the final waypoint. Hand signals were presented by a data collector 

who was moving with the vehicle to monitor driving errors. The hand signals were presented up 

to 10 different times (depending on how quickly the operator navigated the course) during the 

final leg of the course. The data collector ensured that each signal was presented in front of the 

driving camera so that it could be clearly seen by the TALON operator if he was attending to the 

driving camera. Soldiers reported when they saw the hand signal and which specific signal they 

saw to the data collector sitting next to them in the tent, who was also timing the course 

completion. Soldiers teleoperated the TALON from inside a tent during training and actual 

navigation course negotiation, which prevented them from teleoperating the vehicle using line of 

sight rather than the driving camera display. 

Figure 7. Robotic navigation course. 
 

11 

Figure 8. Marked path from first leg of robotic navigation course. 

Figure 9. Complex obstacle. 

Figure 10. Presentation of hand signals. 
 

12 

2.2.6 Motion Sickness Questionnaire 

The Motion Sickness Questionnaire requires the Soldiers to describe any form of motion 

sickness they might have experienced during the preceding exercise (Gianaros, Muth, Mordkoff, 

Levine, and Stern, 2001). 

2.2.7 Soldier Subjective Questionnaires 

Questionnaires included a post-iteration questionnaire and an end of experiment questionnaire. 

The questionnaires were designed to elicit Soldiers’ opinions about their performance and 

experiences with each of the display conditions. The post-iteration questionnaires had the 

Soldiers rate the display concepts and training on a 7-point semantic differential scale ranging 

from “extremely good/easy” to “extremely bad/difficult.” The end of experiment questionnaire 

had the Soldiers compare and contrast the display concepts. 

2.2.8 NASA-TLX 

The National Aeronautics and Space Administration (NASA)-Task Load Index (TLX) is a 

subjective workload assessment tool, which allows subjective workload assessments on 

operator(s) working with various human-machine systems (Hart and Staveland, 1988). It uses a 

multi-dimensional rating procedure that derives an overall workload score based on a weighted 

average of ratings on six subscales. These subscales include mental demands, physical demands, 

temporal demands, own performance, effort, and frustration. It can be used to assess workload 

in various human-machine environments such as aircraft cockpits; command, control, and 

communication workstations; supervisory and process control environments; simulations; and 

laboratory tests. 

2.2.9 SynWin 

The SynWork multitask synthetic work program (Elsmore, 1994) is a battery of tasks that 

represents some of the perceptual and cognitive skills required in many kinds of complex work. 

It has been found to be a reliable and sensitive measure of multitask performance in a variety of 

settings (Proctor, Wang, and Pick, 1998). SynWin is the Windows-based version of the same 

program. It includes four tasks that can be set up to be presented simultaneously. The tasks 

include a simple memory task, an arithmetic computation task, a visual monitoring task, and an 

auditory monitoring task. The memory task briefly shows a string of letters and the participant 

has several seconds to memorize the string before it disappears. After this, a single letter appears 

and the participant must indicate if that letter is part of the original string. The arithmetic 

computation task displays two or three numbers (three digits long) and the participant must add 

the numbers. The visual monitoring task presents a fuel gauge and a pointer ticks down from 

100 to 0. The participant attempts to click on the gauge to “refill” it, causing it to jump back to 

100. The closer the pointer is to 0 when the gauge is clicked, the greater the number of points 
 

13 

awarded. However, if the participant waits too long, there is a large score penalty. The audio 

monitoring task plays one of two tones at set intervals and the participant must listen for a 

particular tone. When that tone is played, the participant clicks an alert button. When the tasks 

are presented simultaneously, each is presented in a separate quadrant on the screen (figure 11). 

Figure 11. SynWin tasks presented in the four quadrants. 

2.3 Procedures 

2.3.1 Demographics 

Demographic data, as well as data concerning the Soldiers’ Army and robotic experience, were 

documented for each Soldier. 

2.3.2 Training 

The experiment Soldiers reported in groups of six for one day each. Before the first training 

presentation, experiment Soldiers received a roster number, which was used to identify them 

throughout the evaluation. They were also given an operations order that explained the robotic 
 

14 

mission that they would undertake during the experiment. Driving requirements, obstacle 

negotiation requirements, and identification of hand signals were all explained and any questions 

the Soldiers had concerning the experiment were answered. A representative from Foster-Miller 

trained the Soldiers in the use of the TALON robot. Soldiers practiced teleoperating and 

navigating the robot on a training course that was similar to the actual course used during the 

experiment. This course required driving, obstacle negotiation, and navigation. Soldiers were 

trained using each of the displays. Before they could begin the actual experiment trials, they had 

to demonstrate proficiency with the system and displays. 

2.3.3 Teleoperation Trials 

Soldiers were assigned to displays and lanes according to the matrix presented in table 1. The 

matrix was constructed using a Williams Squares design for treatment conditions, lanes, and 

preceding device (so that no device always followed another) (Williams, 1949). 

Table 1. Order of treatments and lanes. 

Iteration 

1 2 3 

Roster Display Course Display Course Display Course 

1, 7, 13, 19 C 3 B 2 A 1 

2, 8, 14, 20 A 2 C 1 B 3 

3, 9, 15, 21 B 1 A 3 C 2 

4, 10, 16, 22 A 1 B 2 C 3 

5, 11, 17, 23 B 3 C 1 A 2 

6, 12, 18, 24 C 2 A 3 B 1 

The Soldiers were required to teleoperate the robot within the marked leg of the course as 

quickly and efficiently as possible, negotiate around the obstacle in the path, and navigate to the 

waypoint. A 15-min time limit was given for completion of each lane. A data collector walked 

behind the TALON on each trial, sending hand signals and documenting the number of driving 

errors (backups, turning in the wrong direction, etc.) on each lane. A data collector sitting next 

to the operator documented the number of hand signals correctly identified and the overall course 

completion time. 

Upon completion of the navigation course with each navigation system, the Soldiers were given 

a questionnaire designed to assess their perception of the training adequacy and their ease of 

performance with the navigation device. Questions about the amount of practice time given, the 

level of detail presented, the adequacy of training aids, the ease of operation, and their SA were 

asked. The motion sickness questionnaire and the NASA-TLX were also given after each course 

iteration. At the end of the day, Soldiers completed a questionnaire comparing the three different 

navigation concepts. 
 

15 

2.3.4 SynWin 

The SynWin was administered to each Soldier to assess their ability to multitask. Soldiers were 

trained first with a PowerPoint presentation, then with a 5 min practice session where they 

started first with the memory task. After 1 min, the math task was added. After another minute 

the fuel gauge task was added, then after another minute, the audio task was added. Soldiers had 

no problem comprehending the nature of the individual tasks and the need to manage the tasks in 

such a way as to maximize the composite score. After training, the Soldiers completed the 

SynWin trial and scores for each of the tasks and a composite score were documented. 

3. Results 

3.1 Demographics 

The original sample size of Soldiers was 33. This sample was reduced to 25 because 3 of the 

Soldiers failed to meet the proficiency requirements during the time limit allowed for the training 

course, and the other 5 experienced a robotic malfunction or loss of GPS signal on the navigation 

course during one or more of their iterations that adversely affected their completion times. 

Since this was a within subject evaluation, their times from the other trials also had to be deleted. 

The 25 OCS Soldiers who participated in this experiment averaged 4 years in the military. They 

came from a variety of Army fields, including Infantry, Aviation, Armor, Administration, 

Vehicle Maintenance, Transportation, and Logistics. Six of the Soldiers had been deployed in a 

combat area. Fourteen of the Soldiers had used a GPS and the majority classified themselves as 

having intermediate land navigation experience. While only 9 had robotic experience, all but 2 

Soldiers had video game experience. The weights of the Soldiers ranged from the 40th percentile 

female to the 98th percentile male (130 to 230 lb). Their heights ranged from the 20th percentile 

female to the 99th percentile male (62 to 75 in.). The average age of these Soldiers was 30 years. 

Seventeen Soldiers were left-eye dominant and 12 wore prescription lenses. See the appendix 

for detailed demographic information. 

3.2 Training Evaluation 

The participants were generally positive about the quality of training on the use of the controls 

and displays. There were several comments about activities that were challenging to learn. 

Some of the Soldiers mentioned the difficulty in coping with the time lag in the GPS signal. 

They had some difficulty in judging distances using the display screen. Several participants 

noted that it took some time to get the feel of the joystick control. 

Detailed responses to questions pertaining to the training are included in the post iteration 

questionnaire in the appendix. 
 

16 

3.3 Performance Data 

3.3.1 Course Times 

Table 2 shows the mean course time for the three displays. 

Table 2. Mean course completion times.

min:sec Display Mean SD 

A 6:06 2:33 

B 7:29 2:35 

C 6:26 2:37 

Figure 12, which shows the distribution of course times summed across all three display 

conditions, demonstrates that the course times were highly skewed. The Pearson coefficient of 

skew for course times was Skp = 0.68. Winer, Brown, and Michels (1991) recommend log 

transformations to address positive skewness that frequently occurs when the dependent variable 

is time required to complete a task. It is important to re-examine the distribution after 

transformation to ensure that the skewness is substantially reduced (Tabachnick and Fidell, 

2007). A log10 transformation (Winer, Brown, and Michels, 1991) was used to produce a 

distribution that more closely approximates the normal distribution, thus better satisfying the 

assumptions of analysis of variance (ANOVA). As shown in figure 13, the log10 transformation 

was successful in providing a better approximation to the normal curve. The Pearson coefficient 

of skew for the log10 course times was Skp = 0.17. 
 

17 

0

2

4

6

8

10

12

14

16

<4min

4-5miin

5-6min

6-7min

7-8min

8-9min

9-10min

10-11min

11-12min

12-13min

>13min

Number of cases

Figure 12. Course completion times (sec), all display conditions. 

0

2

4

6

8

10

12

14

16

18

<2.30 2.30-

2.39

2.40-

2.49

2.50-

2.59

2.60-

2.69

2.70-

2.79

2.80-

2.89

>2.89

Number of cases

Figure 13. Log10 course completion times (sec), all display conditions. 
 

18 

Table 3 shows the log10 means. 

Table 3. Log10 mean course completion times. 

Log10 

Display Mean SD 

A 2.5340 0.1565 

B 2.6284 0.1476 

C 2.5526 0.1757 

A repeated measures ANOVA conducted on the log10 course times indicated that there was a 

significant difference among the means: F(2,48) = 4.34, p = .018, η2

p = .153. 

Since we were investigating whether the smaller screen size can be used as effectively as the 

larger one, the performance times with each of the 3.5 in. screen versions was compared to the 

time with the 6.5 in. screen. For such a priori comparisons, t-tests are conducted using the error 

term from the ANOVA to calculate a pooled error term for the t-tests (Winer, Brown and 

Michels, 1991). The results of the planned comparisons are shown in table 4. 

Table 4. Planned comparisons, course completion times. 

Pair t df 

logA - logB 3.93a 24 

logA - logC 0.78 24 

logB - logC 3.16a 24 a

 p < .01, 1-tailed

Hypothesis HO1 that said Soldiers will perform the robot navigation task more quickly with the 

split screen (A) than the toggle screen (B) is supported. The first portion of hypothesis HO4 that 

stated Soldiers will perform the robot navigation task more quickly with the multimodal display 

(C) than with the toggle display (B) is also supported. The portion of hypothesis HO4 that stated 

there will be no difference between the multimodal and the split screen displays is supported as 

well. 

3.3.2 Situation Awareness 

Table 5 shows the mean percentage of hand signals correctly identified with each display type. 

(The data are presented as percentages because the number of hand signals presented varied from 

5 to 10.) The omnibus differences among the means approached statistical significance: 

F(2,48) = 2.96, p = .061, η2

p =0.061. The pre-planned comparisons can be found in table 6. 
 

19 

Table 5. Mean percentage of hand signal identified. 

Display Mean SD 

A 96.8 8.5 

B 90.0 13.2 

C 93.2 11.4 

Table 6. Planned comparisons, percentage of hand signal identified. 

Pair t df 

A - B 3.44a 24 

A - C 1.82 24 

B - C 1.62 24 a

 p < .01, 1-tailed

Hypothesis HO3 that said Soldiers will identify hand signals more quickly and accurately with 

the split screen (A) than with the toggle screen (B) is supported. The first portion of hypothesis 

HO6 that stated Soldiers will identify hand signals more quickly and accurately with the 

multimodal display than with the toggle display is not supported. The portion of hypothesis HO3 

that stated there will be no difference between the multimodal and the split screen displays is 

supported. 

3.3.3 Driving Errors 

The mean number of driving errors with each display type is shown in table 7. In each condition, 

participants averaged less than one error per course completion. The difference among the 

means was not statistically significant: F(2,48) < 1.00, p = .748, η2

p = .012. 

Table 7. Mean number of driving errors. 

Display Mean SD 

A 0.16 0.47 

B 0.12 0.33 

C 0.08 0.28 

Hypothesis HO2 that said Soldiers will perform the robot navigation task with fewer errors with 

the split screen than with the toggle screen is not supported. Neither is HO5, which stated that 

Soldiers will perform the robot navigation task with fewer errors with the multimodal display 

than with the toggle display. There was no difference between the multimodal and the split 

screen displays. 

3.3.4 SynWin 

Descriptive statistics for the SynWin task are shown in table 8. The SynWin scores did not 

correlate significantly with course completion times. 
 

20 

Table 8. SynWin subtest and composite means. 

Subtest Mean SD 

Memory 110.4 125.3 

Math 70.4 60.0 

Visual 114.0 28.5 

Alert/audio 52.8 41.8 

Composite 347.6 182.1 

3.3.5 NASA-TLX 

The results of NASA-TLX workload questionnaire are summarized in figure 14. The graph 

clearly demonstrates that there was very little difference among display conditions for any of the 

workload categories, including total workload. 

0

5

10

15

20

25

30

35

Mental

Physical

Temporal

Performance

Effort

Frustration

Tot workload

Weighted mean

A

B

C

Figure 14. NASA-TLX means by display condition. 

3.4 Subjective Results 

3.4.1 Post-iteration Questionnaire 

Task difficulty ratings for the navigation course were generally worse for the toggle display than 

for the split screen or tactile displays. The Soldiers were very positive about the wealth of 

information provided by the split screen display. They were also very positive about the tactile 

information system. Several commented that they liked being able to give their full attention to 

the driving camera display and rely on the tactile belt for the navigation information. One 

A 

B 

C 
 

21 

Soldier commented that the tactile display allowed him to concentrate on objects closer to the 

robot because he did not have to use distant objects to orient for direction of travel. The Soldiers 

were also generally positive about using the toggle display, but some were annoyed by the need 

to toggle between views. 

The participants rated their SA as being lower with the toggle display than with the split screen 

or tactile displays. They complained that toggling back and forth between views caused them to 

miss some of the hand signals. Some were more efficient than others in glancing at the map 

display and quickly toggling back to the camera display. 

Some of the problems experienced by the Soldiers were related to the camera on the vehicle and 

not the specific display type. For example, there were complaints about the quality of the video 

display and several Soldiers commented negatively about the blind spots around the vehicle 

(because there was a single camera). Also a few Soldiers expressed problems with the camera’s 

fixed position, which caused them to be unable to see the road when negotiating an upward slope 

(when the vehicle is on a forward slope, the camera points up because it is in a fixed position). 

Soldiers experienced difficulty with the latency between the vehicle and driving camera display 

and between the vehicle position and GPS updates. Most of the Soldiers learned to compensate 

somewhat for the latency but several did not. These Soldiers consistently overcompensated, 

which resulted in increased course times and longer routes to the waypoints. 

The egocentric GPS caused a problem for one Soldier. He stated that during the time he toggled 

to the map display to get reoriented after negotiating around an obstacle, the map rotated without 

him realizing it. Thus, when he toggled back to the map display, he started off in the wrong 

direction.

Soldiers’ overall mean ratings of the split screen and the tactile system were very good. Their 

overall mean ratings of the toggle system were neutral. The majority of the Soldiers felt that the 

split screen and tactile displays were both very good for robotic navigation. 

3.4.2 End of Experiment Questionnaire 

The participants were asked their preference for each pair of displays (i.e., Display A vs. Display 

B, Display A vs. Display C, and Display B vs. Display C). Figure 15 shows the proportion of 

times that each display type was selected over one of the other display types. This type of 

scaling, referred to as the Thurstone method of paired comparisons, is designed to yield an equalinterval scale of preference. The split screen was the most preferred display, closely followed by 

the multimodal, and the toggle was the least preferred. Several participants suggested that the 

optimal display would combine the split screen with the tactile belt. 
 

22 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Split-screen Tactile Toggle

Display

%Preference

Figure 15. Paired display preferences. 

The Soldiers had a number of suggestions for enhancing the quality of the displays. They asked 

for better resolution and brightness, and less jitter in the video picture. There was a general 

consensus that robotic driving would be easier if the latencies were reduced or eliminated (both 

the latency in the GPS refresh rate and the latency between the driving inputs and the displayed 

movements). Two Soldiers stated that the tactile signals were not strong enough and one stated 

that the strength of the signals was irritating 

3.5 Engineering Lessons Learned 

Accurate GPS positioning of the robot was key to this experiment. Variations of the GPS drift 

per day and during the day greatly affected the ability of the operators to successfully accomplish 

their task because of the relatively short distances used in this experiment. The operator’s only 

indication of the target position was the waypoint depiction on the map relative to the robot 

depiction on the map. The waypoint position was always a fixed latitude/longitude position 

while the robot position was a GPS returned latitude/longitude. The GPS position for the same 

physical position tended to drift during the day by approximately ±15 m. The drift made the 

physical marking for the waypoints differ from the nominated latitude/longitude of the map 

depiction of the waypoint. Because of this GPS error, operators tended to drive slightly different 

courses to achieve the waypoints, sometimes forcing a data collector to call that the waypoint 
 

23 

was achieved when the operator drove past it to the left or right. The GPS also has a tendency to 

“pop” or jump by ±5 m, which confused the operator concerning the position of the robot 

relative to the target waypoint. Coupled with the errors inherent in GPS were various technical 

problems associated with the GPS system mounted on the TALON for this experiment, which 

was inexpensive and tended to be inaccurate. 

Accurate robotic compass headings were necessary to reduce operator confusion when reading 

the robot’s position on the map. Initially, problems were experienced with the heading reported 

by the robots, which made the use of the robot depiction on the map nonsensical. Correct 

calibration of the robot’s onboard compass solved this issue. 

The tactile direction signals were calculated from the GPS position reported by the robot relative 

to the target waypoint position. The update rate of the GPS was approximately once per second. 

Thus, there is a relationship between the rate of tactile signals and the speed with which the robot 

changes its position and that position change is reported by the GPS. The signals used for our 

experiment need to be modified for future experiments to allow for the timing between the GPS 

updates to reduce operator confusion. 

The robots used required manual calibration of the internal compasses. Various Foster-Miller 

robot software patches were required to decode the correct GPS message packets. The FosterMiller GPS cabling had to be rerouted away from the drive motors to reduce interference that 

precluded the GPS position from updating until the drive motors were stopped. 

Inexperienced operators tended to neutral steer, changing the robot direction on the spot. This 

stopped one side of the robot from moving by actively holding the motor in its current position 

while the drive motor on the opposite side turned the robot around. When this happened 

frequently, the motors overheated. When the drive motors overheated, they shutdown until they 

cooled enough to operate again. This gave the impression that the operator no longer had 

control. 

4. Discussion 

The results from this experiment demonstrate the efficacy of the addition of a tactile belt to a 

robotic OCU for dismounted operations. Soldiers performed equally as well with the smaller 

display size that included a tactile navigation display (the multimodal display) as they did with 

the larger split screen display. Sequential operation of a visual driving camera display and a 

visual map display was not as effective. Specifically, Soldiers’ timed performance with the split 

screen and the multimodal displays was significantly better than their timed performance with 

the toggle display. There was no difference between their timed performance with the split 

screen and multimodal displays. While there was no difference in the Soldiers’ driving errors 
 

24 

with any of the displays, it should be noted that there were very few driving errors committed on 

this short course. More errors might be present if a longer course with more obstacles and terrain 

features was used and differences between displays might become more apparent. 

As hypothesized, the Soldiers identified more hand signals with the spilt screen display than with 

the toggle display and there was no difference between the number of hand signals identified 

with the split screen and multimodal displays. While no differences were found between the 

toggle and multimodal displays in terms of the operators’ SA, further study is warranted. The 

availability of the robots limited the scope of this experiment in terms of SA trials. The limited 

number of hand signals given in this experiment and the ease of detection of the hand signals 

resulted in somewhat of a ceiling effect. The original plan included a leg of the course dedicated 

to SA trials that necessitated the operators to search for information along the route. The search 

was designed to require the operators to look carefully along both sides of the route and to look 

for objects that had low contrast with the surroundings. This task would have required more 

time in the searching mode and would have allowed less time for looking at a visual map. The 

more demanding SA task also would not have provided an attention-getting movement cue (like 

when the data collector’s hands moved in front of the camera). Thus, if the operator workload 

had been higher, with more competition for visual resources, results may have favored the 

multimodal option to a greater degree. Since much of the navigation task depends on ambient 

and focal visual information processing, Wicken’s MRT predicts that workload for this task can 

be offloaded to a different sensory channel (Boles, 2001; Wickens, 2002). This would free the 

visual channel for driving and searching, thus theoretically resulting in higher SA. Wickens 

(2002) notes that when tasks are complementary, there will be less competition for resources and 

more efficient time sharing. 

Egocentric map perspectives like the one used in this experiment have been found to be more 

efficient in route-guiding situations than exocentric maps (Porathe, 2007; Hermann, Bieber, and 

Duesterhoeft, 2003). However, this type of map was problematic during this experiment when 

operators had to attend to a driving camera display and did not have the opportunity to pay 

constant attention to the map display. Seager and Stanton-Fraser (2007) found this to be true 

when their experiment demonstrated that users find it difficult to recognize a map that rotates 

automatically when they are not looking at the map. The addition of a tactile navigation display 

would alleviate this problem. 

Two Soldiers in this experiment stated that the tactile signals were not strong enough and one 

stated that the strength of the signals was irritating. Previous research with tactile displays 

indicates that tactor intensity strength should be adjustable to account for individual differences, 

stress, and differences task intensity levels (Redden, Carstens, Turner, and Elliott, 2006; Redden 

and Elliott, 2007; van Erp, 2007, 2002). 
 

25 

5. Recommendations 

Further experimentation using a tactile belt for dismounted robotic navigation is warranted. 

While this experiment demonstrated that adding a tactile belt to the robotic display allowed the 

use of a smaller screen size, more experimentation that addresses the SA of the operator is 

needed because MRT suggests that a tactile display might provide increased SA around the 

vehicle over a split screen display. 

A possible solution for the robotic GPS problems would be to include an inertial positioning 

system in addition to the GPS on the robot to compensate for dropped and popped GPS 

positions. An inertial system is used on larger robotic vehicles for exactly these reasons. Use of 

a GPS only system is useful for single estimations of position but not so useful in a continuous 

accurate positioning system. 

The tactile signals used for robotic navigation need to be modified to allow for the timing 

between GPS updates because of the relationship between the rate of the tactile signals sent by 

the belt and the speed with which the robot changes its position and that position change is 

reported by the GPS. The tactile belt should be modified so that individual Soldiers have the 

capability of increasing or decreasing the tactor strength based upon their own pain and 

discrimination thresholds or upon the intensity of the situation. 
 

26 

6. References 

Alexander, A. L.; Wickens, C. D.; Hardy, T. J. Synthetic Vision Systems: The Effects of 

Guidance Symbology, Display Size, and Field of View. Human Factors 2005, 47, 693–707. 

Boles, D. B. Multiple Resources. In International Encyclopedia of Ergonomics and Human 

Factors, W. Karwowski, Ed.; Taylor & Francis, 2001, pp 271–275,. 

Dorneich, M.;Ververs, Pl; Whitlow, S.; Mathan, S. Evaluation of a tactile navigation cueing 

system and real-time assessment of cognitive state. In Proceedings of the Human Factors 

and Ergonomics Society 50th Annual Meeting, Human Factors and Ergonomic Society: Santa 

Monica, CA, 2006, pp 2600–2604. 

Duistermaat, M.; Elliott, L.; van Erp, J.; Redden, E. Tactile land navigation of dismount 

soldiers. In Human Factors Issues in Complex Environment Performance, D. de Waard, G. 

Hockey, P. Nickel, K. Brookhuis, Eds.; Europe chapter of the Human Factors and 

Ergonomics Society, Shaker Publishing, 2007. 

Elliott, L. R.; Redden, E. S.; Pettitt, R. A.; Carstens, C. B.; van Erp, J.; Duistermaat, M. Tactile 

Guidance for Land Navigation; ARL-TR-3814; U.S. Army Research Laboratory: Aberdeen 

Proving Ground, MD, 2006. 

Elliott, L. R.; Duistermaat, M.; Redden, E. S.; van Erp, J. Multimodal Guidance for Land 

Navigation; ARL-TR-4295; U.S. Army Research Laboratory: Aberdeen Proving Ground, 

MD, 2007. 

Elsmore, T. F. SYNWORK1: a PC-based Tool for Assessment of Performance in a Simulated 

Work Environment. Behavior Research Methods, Instruments, & Computers 1994, 26, 421–

426. 

Gilson, R.; Redden, E. S.; Elliott, L. R. Remote Tactile Displays for Future Soldiers; ARL-SR0152; U.S. Army Research Laboratory: Aberdeen Proving Ground, MD, 2007. 

Gianaros, P. J.; Muth, E. R.; Mordkoff, J. T.; Levine, M. E.; Stern, R. M. A Questionnaire for 

the Assessment of the Multiple Dimensions of Motion Sickness. Aviation, Space, and 

Environmental Medicine 2001, 72, 115–119 

Hart, S. G.; Staveland, L. E. Development of NASA-TLX (task load index): Results of 

empirical and theoretical research. In Human Mental Workload, P. A. Hancock & N. 

Meshkati, Eds.; Elsevier Science Publishers: North-Holland, 1988, pp 139–183. 
 

27 

Hermann, F.; Bieber, G.; Duesterhoeft, A. Egocentric Maps on Mobile Devices. In Proceedings 

of the 4th International Workshop on Mobile Computing, IMC, G. Bieber and T. Kirste, Eds.; 

IRB Verlag: Stuttgart, 2003, pp 32–37. 

Minkov, Y.; Perry, S.; Oran-Gilan, T. The Effect of Display Size on Performance of Operational 

Tasks with UAVs. In Proceedings of the Human Factors and Ergonomics Society 51st

Annual Meeting, Human Factors and Ergonomics Society: Santa Monica, CA, 2007, pp 

1091–1095. 

Muthard, E. K.; Wickens, C. D. Compensation for Display Enlargement in Flight Control and 

Surveillance; Technical Report AHFE-04-03/NASA-04-1; University of Illinois Human 

Factors Division, 2004. 

Pettitt, R. A.; Redden, E. S.; Carstens, C. B., Elliott, L. R. Scalability of Robotic Controllers: 

An Evaluation of Alternatives; ARL-TR-4457; U.S. Army Research Laboratory: Aberdeen 

Proving Ground, MD, 2008. 

Porathe, T. User-Centered Map Design. Paper presented at the Usability Professionals’ 

Association Conference 2007, Austin, TX, 2007. 

Proctor, R. W.; Wang, D. Y.; Pick, D. F. An empirical Evaluation of the SYNWORK1 MultipleTask Work Environment. Behavior Research Methods, Instruments & Computers 1998, 30, 

287–305. 

Redden, E. S.; Carstens, C. B.; Turner, D. D.; Elliott, L. R. Localization of Tactile Signals as a 

Function of Tactor Operating Characteristics; ARL-TR-3971; U.S. Army Research 

Laboratory: Aberdeen Proving Ground, MD, 2006. 

Redden, E. S.; Elliott, L. R. Research-Based Display design guidelines for Vehicle Crewman 

and Ground Warrior Interfaces which enhance Situational Understanding and Decision 

Cycle Performance; ARL-TR-4231; U.S. Army Research Laboratory: Aberdeen Proving 

Ground, MD, 2007. 

Redden, E. S.; Pettitt, R. A.; Carstens, C. B.; Elliott, L. R. Scalability of Robotic Displays: 

Display Size Investigation; ARL-TR-4456; U.S. Army Research Laboratory: Aberdeen 

Proving Ground, MD, 2008. 

Seager, W.; Stanton-Fraser, D. Comparing Physical, Automatic and Manual Map Rotation for 

Pedestrian Navigation. In ACM SIGCHI Conference on Computer-Human Interaction (CHI 

2007), ACM Press, May 2007, pp 767–776. 

Stark, J. M.; Comstock, J. R.; Prinzel, L. J.; Burdette, D. W.; Scerbo, M. W. A Preliminary 

Examination of Situation Awareness and Pilot Performance in a Synthetic Vision 

Environment. In Proceedings of the Human Factors and Ergonomics Society 45th Annual 

Meeting, Human Factors and Ergonomics Society: Santa Monica, CA, 2001, pp 40-43. 
 

28 

Stelzer, E.; Wickens, C. Pilots Strategically Compensate for Display Enlargements in 

Surveillance and Flight Control Tasks. Human Factors 2006, 48, 1, 166–181. 

Tabachnick, B. G.; Fidell, L. S. Using Multivariate Statistics; Boston: Pearson, 2007. 

Van Erp, J.B.F. Tactile displays for navigation and orientation: Perception and behavior; 

Mostert & Van Onderen: Leiden, The Netherlands, 2007. 

Van Erp, J.B.F. Presenting directions with a vibrotactile torso display. Ergonomics 2005, 48, 

302–313. 

Van Erp, J.B.F. Guidelines for the use of vibro-tactile displays in Human Computer Interaction; 

In Proceedings of Eurohaptics 2002, Eds. S.A. Wall, B. Riedel, Crossan, A. & McGee, M.R. 

University of Edinburgh, Edinburgh, Scotland, 2002, pp 18–22. 

Wickens, C. D. Multiple resources and performance prediction. Theoretical Issues in Ergonomic 

Sciences 2002, 4, 1–19. 

Wickens, C.; Muthard, E.; Alexander, A.; van Olffen, P.; Podczerwinski, P. The Influences of 

Display Highlighting and Size and Event Eccentricity for Aviation Surveillance. In

Proceedings of the 47th Annual Meeting of the Human Factors and Ergonomics Society,

Human Factors and Ergonomics Society: Santa Monica, CA, 2003, pp 149–153. 

Williams, E. J. Experimental designs balanced for the estimation of residual effects of 

treatments. Australian Journal of Physical Sciences 1949, A2, 149–168. 

Winer, B. J.; Brown, D. R.; Michels, K. M. Statistical Principles in Experimental Design; 

McGraw-Hill, New York, 1991. 
 

29 

Appendix. Soldier Questionnaire Results 

DEMOGRAPHICS 

SAMPLE SIZE = 25 

MOS: 09S – 9 42A – 3 94R – 1 

 11B – 2 88 – 1 97E – 1 

 19D – 1 92A – 1 OCS – 2 

 25V – 1 92Y – 1 NR – 1 

 35P – 1 

RANK: DUTY POSITION

E-4 – 5 Avionic Systems Maintainer 1 

E-5 – 13 Combat Camera 1 

E-6 – 5 Motor Pool 1 

E-7 – 1 OCS 13 

NR – 1 Human Resources NCO 2 

 Sniper Section NCO 1 

 Supply 2 

 No response 4 

1. What is your age? 30 years (mean ) – 22-38 range 

2. What is your height? 69 inches (mean) – 62-75 range 

3. What is your weight? 180 lbs (mean) – 130-230 range 

4. With which hand do you most often write? Right – 21 Left – 3 NR – 1 

 

5. With which hand do you most often fire a weapon? Right – 20 Left – 5 

6. Do you wear prescription lenses? Yes – 12 No – 13 

7. If Yes, which do you most often wear? Glasses - 6 Contacts – 6 

 

8. Which is your dominant eye? Right – 17 Left – 7 NR – 1 

9. Have you ever used a GPS system? Yes – 14 No – 10 NR – 1 

10. If Yes, what kind of GPS? Automotive Nav System 3 

 Garmin 9 

 Cell phone 1 

 PLGR 1 

 No response 10 
 

30 

11. How many months in military? 48 (mean) 

 

12. Have you had an infantry-related job? No – 21 Yes – 3 NR – 1 

13. How many months in infantry-related job? 93 (mean) 

14. Have you been a fire team leader? No – 22 Yes – 3 

15. How many months as a fire-team leader? 22 (mean) 

16. Have you been a squad leader? No – 15 Yes – 10 

17. How many months as a squad leader? 20 (mean) 

18. Have you been deployed overseas? No – 14 Yes – 10 NR – 1 

19. How many months deployed overseas? 20 (mean) 

20. Have you been deployed in a combat area? No – 19 Yes – 6 

21. How many months deployed in combat zone? 13 (mean) 

None Beginner Intermediate Expert NR

(number of responses) 

Land navigation 0 8 12 5 0 

Operating ground unmanned vehicles 13 9 1 1 1 

Operating unmanned aerial vehicles 23 1 0 0 1 

Target detection and identification 8 10 3 3 1 

Playing commercial video games 2 6 9 8 0 

Training with Army video simulations 6 11 5 2 1 
 

31 

POST ITERATION QUESTIONNAIRE 

SAMPLE SIZE = 25 

A (Spilt Screen); B (Toggle); C (Tactile); 

1. Using the scale below, please rate your ability to perform each of the following 

navigation tasks based on your experience with the display that you just used: 

1 2 3 4 5 6 7 

Extremely difficult Very difficult Difficult Neutral Easy Very easy Extremely easy 

NAVIGATION TASKS MEAN RESPONSE 

A B C 

a. Move the robot in the correct direction 4.96 4.72 5.37 

b. Identify any other terrain features that might have an adverse 

effect on the ability of the robot to maneuver through the terrain 

4.65 4.80 5.09 

c. Locate the robotic vehicle on the map 6.05 6.00 5.96 

d. Determine the robot's orientation on the map 6.00 5.74 5.91 

e. Determine when you need to make a course correction 5.55 4.83 6.00 

f. Identify if you are on the correct course 5.55 4.75 6.18 

g. Navigate far enough ahead to plan route in advance 5.58 4.80 5.67 

h. Navigate well enough to drive at slowest speeds 5.95 5.39 6.05 

i. Navigate well enough to drive at medium speeds 5.46 5.04 5.54 

j. Navigate well enough to drive at fastest speeds 4.92 4.25 4.58 

k. Navigate and drive simultaneously 5.48 4.20 5.43 

l. Navigate around obstacles 5.17 4.88 5.33 

m. Return to the correct route after navigating around obstacles 5.58 5.00 5.83 

n. Stay on course 5.52 4.64 6.00 

o. Overall ability to perform driving and navigation tasks 5.59 4.67 5.87 

Comments No. of Responses

A (Split Screen)

Good system, course corrections are made easily and quickly. 1 

Being able to see the map and the camera feed at the same time was very helpful 

with efficiency and not missing any vital information. 

1 

I found that the ability to navigate was greatly improved with the split screen rather 

than the toggle screen. 

1 

Helpful to have both screens in order to focus on objective and not to miss anything 

on the video feed. 

1 

The spilt display helps to determine which course to take. 1 
 

32 

Comments No. of Responses

This was the easiest setup to utilize. The ability to have everything on one screen, 

with no toggling was a major impact on the overall effectiveness of the mission. 

1 

The tasks were fairly comfortable to perform. I can't say it was easy because I had to 

make adjustments and corrections. I can't say it was difficult because the 

adjustments made were not hard to do. Overall, it was a neutral process that was 

not easy nor was it difficult. 

1 

Controls are squishy, but the split screen makes driving and navigation easier. 1 

The time lag from controller to robot was the hardest part about navigating the robot 

during the task. The moisture on the lens was not that bad for the task. 

1 

I had little trouble orienting myself even when thrown off course by the obstacle. 

Only when I came within 20 meters of the objective did I become unsure of its 

location as it was behind the tent which I felt slightly uncomfortable navigating 

around. In reality, I was right on line the entire time. 

1 

Battery on robot ran low while performing test making maneuvering more difficult. 1 

Black and white screen made it hard to identify obstacles and terrain features. 1 

Bushes will roll the robot over. 1 

B (Toggle)

It was fair. 1 

The main benefit was that you had a decent enough camera angle to see far enough 

ahead to accurately maneuver the vehicle 

1 

There is a slight lag between the controls and the robot's response, but with practice 

the user can easily adapt to this. 

1 

The toggle method made it harder to navigate and drive the robot. The difficulty was 

compounded by the lag of the controller to the robot. 

1 

Sometimes the camera would get fuzzy and it was difficult to navigate during these 

times. Additionally, the lag time between the GPS system and the robot was quite 

slow and one had to constantly stop and wait for the robot's position to catch up on 

the course making staying on course somewhat trying. 

1 

I found navigation of the robot to be fairly easy. However, I experienced an error 

where the robot would not steer until it was kicked from behind. I do not know 

what the problem was but after the jolt, everything worked well. 

1 

I would rather have the GPS map on screen so I can glance at it when navigating. I 

was often hesitant to do so because I did not want to miss any of the information 

that I was supposed to record. 

1 
 

33 

Comments No. of Responses

It was difficult switching from GPS map to the view screen while staying on course. 1 

It was hard to flip between screens because it seemed like if I flipped I would miss 

mission essential information. 

1 

It will take time to learn how to navigate the robot with the controller. 1 

Map orientation was off track this time when compared to other previous tests. 1 

Sometime robot will overheat which says it is probably not made for continuous use 

and long hours of usage. 

1 

Task seemed moderate in difficulty, robot appeared to drift to the right. I couldn't 

tell if I was over compensating or if it was a function of the robot. 

1 

The problems came with getting the vehicle around an obstacle or depth perception 

through the camera view. Throughout the phase where the vehicle needed to go 

around the obstacle, I found myself wishing there had been a wide angle lens, or 

side oriented camera to see the obstacle; and know when to cut the vehicle back so 

as not to go too far around the obstacle. 

1 

Toggling back and forth is a pain. 1 

C (Tactile)

Best of the three. 1 

Easy. 1 

The belt made it much easier. 1 

I thought the tactile belt made up for the lack of a map on the screen at all times. I 

always felt comfortable about where I was and where I was heading. I also found 

the objective easier with the tactile belt. 

1 

The belt system helped by incorporating a new sense to the equation. It helped by 

not overloading your vision. 

1 

The tactile belt made navigating so easy a caveman could do it. 1 

This was much easier to use than the toggle view. Instead of flipping screens I used 

the camera view and the belt to guide the robot this way I didn't miss any of the 

hand signals. 

1 

Tasks were very good; the split screen with belt combination would be very helpful 

so you can see simultaneously the map to associate yourself to way point and 

video of where you’re going on ground. This I think will enable you to not get too 

far off your target in relation to your way point. 

1 

Color was out on camera. 1 

I flipped the screens to get reoriented after getting around the obstacle and the screen 

was flipped for me. I started off in the wrong direction and corrected it. Close to 

the end the robot didn't turn right for me to get to the finishing point. 

1 
 

34 

Comments No. of Responses

The video relay was a little fuzzy at times, but this could have been from being out 

of reach. 

1 

The GPS needs work, too much lag, switching between view and map is distracting 1 

2. Using the scale below, please rate your ability to ability to maintain situation 

awareness around the vehicle with the display that you just used: 

1 2 3 4 5 6 7 

Extremely difficult Very difficult Difficult Neutral Easy Very easy Extremely easy 

SITUATION AWARENESS TASKS MEAN RESPONSE 

A B C 

a. Awareness of terrain 5.29 4.54 5.33 

b. Awareness of hand signals 6.13 5.17 6.14 

c. Overall situation awareness of the environment around vehicle 5.17 4.50 5.33 

A (Split Screen)

Good views! 1 

The image on the screen can have better clarity and definition, but it was effective 

enough to get the job done. 

1 

Blind spots are still making the situational awareness of the vehicle difficult to say 

the least. 

1 

Single camera made it difficult to be aware of the environment to the left and right 

of the vehicle. 

1 

The camera's fixed position on the upward slope made it harder to see the terrain 

over the crest making it a little harder to see what was coming next with the 

situation and test. 

1 

B (Toggle)

Clicking back and forth to see where you were on the GPS system made it tricky to 

see the hand signals. 

1 

I had to be fast when I had to toggle between the camera and map. If I was not fast 

then I would have missed a hand signal. 

1 

It was more difficult to observe hand signals because I had to change between 

navigation aid screen and driving camera screen. 

1 

The camera was having difficulties so at times it was hard to tell what hand signal 

was being given. Also when I would flip screens I would miss or almost miss a 

hand signal. 

1 

Toggling the window brought about a concern that I would miss a hand signal as 

well as not seeing where I was going. 

2 

Need better visual field. 1 
 

35 

Comments No. of Responses

Some times picture quality will be fuzzy when going over rough, uneven terrain. 1 

This would be the main downfall of the equipment. The single camera view, allows 

at most 60 degrees of vision. The other 300 degrees were completely black, and 

could have held any type of enemy. For instance, I did not realize there was a 

group of people walking with the robot, until the end of the exercise. This is 

because they were in the blind spot the entire time. The terrain awareness is 

limited due to the low camera angle. It is mainly a depth perception problem. 

1 

C (Tactile)

Too easy. 2 

Using the tactile belt, I was able to keep my eyes on the camera display and only 

used to GPS to confirm my waypoint. 

1 

I came up quickly on the ditch or gully that was hard to see due to the view point or 

camera position. Other than that it was good. 

1 

The tactile vibrations allowed for me to move to the objective without having to 

navigate using the GPS. It allowed me to focus on the screen. With practice I 

believe it to be an effective method. The vibrations could be different though, they 

tend to be slightly irritating. 

1 

A guide identified a restriction on the terrain, but I saw it before he got in front of 

the UGV. 

1 

Situational awareness around the vehicle is still a major concern, as the blind spots 

are immense. 

1 

Using only the forward camera made it difficult to know when the vehicle had 

passed the obstacle. 

1 

You can only be aware of the situation around you when you flip back to the video, 

however if you've flipped to map you have no situational awareness when you take 

time to orient yourself to your current position in relation to your way point for 

those ten or more seconds it may take to do so. 

1 

3. Please check any of the following conditions that you may have experienced during this trial. 

CONDITION No. of Responses 

A B C 

Eyestrain 1 1 0 

Tunnel vision 0 0 0 

Headaches 0 0 0 

Motion sickness 0 0 0 

Nausea 0 0 0 

Disorientation 1 2 1 

Dizziness 0 0 0 

Any other problems? 0 1 0 
 

36 

Comments No. of Responses

A (Split Screen)

No problems. 1 

Slight disorientation when first starting; able to overcome after a minute. 1 

B (Toggle)

Difficult due to visual field limitation. 1 

I was more concerned about the time and direction with having to stop and orient the 

robot to the way-point. 

1 

Robot appeared to drift to the right. I couldn't tell if I was over compensating or if it 

was a function of the robot. 

1 

Slight disorientation when first starting out. 1 

Without GPS on hand, I was not sure which way I was heading at times. 1 

C (Tactile)

Slight disorientation because one would not be used to it at first, but as you practice 

you become more proficient in orienting yourself. 

1 

4. Using the scale below, please rate the following characteristics (if applicable) of the 

display that you just used: 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN 

DISPLAY CHARACTERISTICS RESPONSE 

A B C 

a. Resolution (clarity) of the display 4.44 4.28 4.62 

b. Size of objects appearing in the driving camera display 5.16 4.88 4.83 

c. Size of objects appearing in the map display 5.46 5.12 5.17 

d. Comfort of viewing the driving display 5.62 5.24 5.21 

e. Comfort of viewing the map display 5.83 5.24 5.46 

f. Comfort of using the tactile display - - 5.70 

g. Strength of tactile display signal - - 5.50 

h. Contrast between objects on the driving display 5.08 5.12 5.04 

i. Contrast between objects on the map display 5.52 5.20 5.21 

j. Driving display color 4.96 5.08 4.91 

k. Map display color 5.68 5.80 5.67 

l. Driving display brightness 5.28 5.36 5.46 

m. Map display brightness 5.64 5.63 5.62 

n. Accuracy of map display 4.92 5.00 4.74 

o. Accuracy of tactile display signal - - 5.48 

p. Amount of lag between vehicle position & driving camera 5.20 5.25 4.74 

q. Amount of lag between vehicle position & navigation aid update 4.08 3.84 4.00 

r. Adequacy of display of this type for teleoperating a robotic 

vehicle 

5.32 4.32 5.43 
 

37 

Comments No. of Responses

A (Split Screen)

This display was the most effective thus far. It was easy to use, and provided the 

most information in one package. 

1 

At first, I did not like the split screen, but when I needed to see the map for 

navigation the page was below. This is good because an operator may miss a 

important camera event while looking at the map view. 

1 

Resolution is not good. 1 

Black and white screen made it hard to distinguish between objects. 1 

Video blurs sometimes, GPS is slow to plot position. 1 

The lag made the navigation uneasy, but once you understand the lag then you can 

compensate for it. However, it does require practice for appropriate compensation. 

1 

I didn't like the lag time. It would be nice to have a faster response with the robot. 1 

It appeared to have the same amount of GPS lag as with the toggle view. After 

rolling the robot, the GPS did not work correctly. 

1 

The lag time really makes it impossible to navigate the vehicle based on the 

navigation aid display. 

1 

The camera cut out on me during a crucial moment as I was attempting to maneuver 

around the obstacle. The map display is lagging a good amount which takes some 

getting used to. It is still helpful in plotting a long distance course but can easily 

allow the user to overcorrect. 

1 

B (Toggle)

Overall, I think that the operation of the robot was good. 1 

Afraid of missing important hand signals. 1 

I think there is a problem with lag time on the GPS display. I am not certain whether 

or not that can be corrected. I believe this could facilitate driving errors due to the 

temporary inaccuracy of the map display. 

1 

The display had a lag that would make certain features, such as the GPS; combat 

ineffective. The lag would make you sit and wait, and then the heading is 

constantly adjusting when moving constantly. If it were a little quicker on the lag 

side, and then accounted for the heading better it could serve as a highly effective 

tool for soldiers. 

1 

Poor refresh rate with navigation aid screen made it impossible to navigate on this 

mode. Switching between screens made progress slow and choppy. 

1 

The toggling made it more difficult to navigate. I did use the map more to navigate 

in relation to the actual terrain and way-point. But, it wasn't as nice as seeing 

where you were going. 

1 

Too hard to see details. 1 

Video feed was in and out at times. 1 
 

38 

Comments No. of Responses

None. 1 

Visual field limitation needs improvement. 1 

With no prior experience in operating a robotic vehicle, it is impossible for me to 

judge the adequacy of the visual device I used based upon available devices in the 

field or on the market. 

1 

Would much rather have the GPS on display at all times. 1 

C (Tactile)

I believe the tactile display is the easiest to use. 1 

I liked the tactile display; it was another aid to reach the way-point and I didn't have 

to use distant objects to orient for direction of travel. I could concentrate on what 

was closer to the robot. 

1 

The display is a sound concept that needs only minor combat-ready modifications to 

get it to the stage where it can be tested in a more realistic combat environment. 

1 

The display is good but for missions > 30 minutes it could possibly cause eye strain. 1 

Visual clarity and visual fields can be improved with more quality and fielding. 1 

Camera definition could be better. 1 

Color was out on camera view. 1 

I think that training on all displays may be necessary depending on the mission. 1 

The lag can be a slight setback if you’re trying to turn within a split second, because 

one may tend to overturn if not reacting as you would in real time situational 

awareness. 

1 

The lag on the navigation aid update made it difficult to navigate the vehicle. Low 

camera quality made it difficult to distinguish between terrain features (including 

the obstacle). 

1 

5. Using the scale below, what is your overall rating of the display that you used this 

iteration? 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN RESPONSE 

A B C 

5.52 4.44 5.50 

A (Split Screen)

This would be my choice thus far. 1 

I was able to read the hand signals much better with this display. 1 
 

39 

Comments No. of Responses

I used the map very little and attempted to use objects in the distance in relation to 

the map for direction and navigation. I used the map mostly to tell if I reached the 

way-point and basic knowledge of direction and terrain. 

1 

It was a fairly good navigation activity given the lag within the equipment and the 

limited clarity and visual field from the camera. There is a lot of room for 

improvement to remove the equipment lag, to improve the vision clarity and to 

expand the visual field from the camera. A larger visual field would allow for 

better field bearing and navigation. 

1 

Once the soldier is used to map lag times, it is not too difficult to navigate the 

machine. Improvements would include map lag times, camera reliability, and 

camera resolution. 

1 

B (Toggle)

I believe that the toggle display could be effectively used in nearly any situation with 

the proper amount of practice. 

1 

Good system, it will take longer to train on but can be as effective as split screen if 

soldier used map strictly for navigation due to slow GPS plotting. 

1 

Overall, I would say the system is a sound idea. There are a few things that need to 

be worked out to make the device combat ready. The main things being a better 

sense of depth perception on the video display, possibly achieved through raising 

the camera height; lowering the lag between GPS and actual position, and then I 

would also recommend calibrating the system to be used in a stressful 

environment. We used it while our bodies are at ease, and the joystick/system 

required a very sensitive touch. 

1 

If the lag time could be fixed the display was pretty good. The camera fuzziness 

made it difficult to see terrain features such as rocks, bumps in the road, etc. 

Hitting these features also caused the camera to be fuzzy so it would help to see 

the features better prior to hitting them. 

1 

It was harder to navigate by not seeing or feeling and view what was actually in 

front of you. It limited the ability to accurately travel and operate the robot. 

1 

Overheating may become problematic if continuous usage is required. 1 

Sometimes the image will be distorted with slight green streaks. 1 

Toggle was less convenient than the split due to the flipping back and forth. 1 

C (Tactile)

My favorite of the three. 1 

Too easy. 1 

This so far this has been the best system as I do not have to take my eyes off the 

driving camera in order to navigate to my objective. An effective system that is 

also helpful when finding exact points such as the objective. 

1 

Belt is easy to use and much more accurate than the navigation alone. 1 

Overall a good system to use on any maneuver/search operation. 1 
 

40 

Comments No. of Responses

This setup was my 2nd favorite choice. I think that the additional sense added, 

helped the user; by not allowing him to overload one particular sense. In this case, 

the sense of sight was complimented by a sense of touch. The combat 

effectiveness of this approach could prove problematic though. The average 

soldier in combat would have to have the ability to distinguish between the signals 

from the belt, and then other outside factors of touch, i.e., something hitting them, 

overcoming the adrenaline factor. 

1 

This was an very effective method of controlling the machine. The belt was slightly 

irritating though, but the thought behind such a belt makes a lot of sense. 

1 

The vibrations are only helpful if you concentrate on the vibrations, it's easier to 

follow camera/map. 

1 

Gianoros Motion Sickness Question:

6. Using the scale below, please rate how accurately the following statements describe 

your current state. 

1 2 3 4 5 6 7 8 9 

Not at all < ---------------------------------------------------------- > Severely 

MEAN RESPONSE 

A B C 

I felt sick to my stomach 1.04 1.13 1.00 

I felt disoriented 1.16 1.13 1.12 

I felt faint-like 1.04 1.00 1.00 

I felt tired/fatigued 1.24 1.33 1.40 

I felt annoyed/irritated 1.20 1.50 1.64 

I felt nauseated 1.04 1.00 1.00 

I felt sweaty 1.64 1.71 1.68 

I felt hot/warm 2.24 1.87 2.08 

I felt queasy 1.04 1.00 1.00 

I felt dizzy 1.04 1.00 1.00 

I felt lightheaded 1.04 1.00 1.00 

I felt like I was spinning 1.04 1.00 1.00 

I felt drowsy 1.08 1.21 1.16 

I felt as if I may vomit 1.00 1.00 1.00 

I felt clammy/cold sweat 1.00 1.00 1.00 

I felt uneasy 1.08 1.09 1.08 
 

41 

END OF EXPERIMENT 

SAMPLE SIZE = 25 

1. Using the scale below, please rate the training that you received in the following areas: 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN RESPONSE 

a. How to drive the robot 6.04 

b. Time provided to practice driving the robot 5.96 

c. How to complete Leg A (obstacle negotiation) of the course 5.92 

d. Evaluation of the practice lane 5.92 

e. How to use the navigation map 6.00 

f. How to use the tactile belt 6.17 

g. How well you expected to perform on the actual course after training 5.80 

h. Overall evaluation of the training course 6.24 

2. What were the easiest and hardest training tasks to learn? 

Comments No. of Responses

Easiest

How to operate the robot. 5 

The whole thing was pretty easy. 1 

The actual task. 1 

Moving the vehicle forward and backward in any given direction. 1 

Basic movement. 1 

Clear concise instructions made the task of operating the robot easy. 2 

Learning the control functions. 1 

The easiest training tasks were orienting to the map and camera feed. 1 

Even though I did well in training, I was still a bit apprehensive about the actual 

course. After I started it though I regained my confidence. 

1 

Tactile belt. 3 

Negotiating the obstacle course using the tactile belt. 2 

Familiarization with the use of the map and tactile belt. 1 

Negotiation and navigation of the course and time. 1 

Learning how to use the controller with the robot and the time delay. 1 

Operation of the viewing screen. 1 

Split screen and Belt. 1 
 

42 

Comments No. of Responses

Understanding the navigation map. It was easy to manipulate the course via the 

navigation system. 

1 

Viewing direction via the monitor from the robot's perspective. 1 

Hardest

The GPS lag was pretty bad in places. 2 

The lag on the response time when controlling the robot made this course more 

difficult than it should be. 

2 

The lag time and orienting the robot. 1 

The camera needs some work. 1 

Beginning. 1 

Control and maneuvering the robot, i.e., trying not to over compensate your turns. 1 

Controlling the robot with the joystick and negotiating the terrain. 1 

The most difficult thing was getting the hang of how much pressure to put on the joy 

stick to get it to move in the direction you want it to go. 

1 

Using the joystick at first, after training it was easy. 1 

Driving, because the controls can be touchy sometimes and non-responsive at others. 1 

Learning how to turn the robot around corners when moving. 1 

Learning the feel from the delay on the controller versus the response of the robot. 1 

Judge the distance to object and navigate around object outside of camera view. 1 

Judging distances from the monitor. 1 

Keep speed slow. 1 

Turning the robot at higher speeds. 1 

Navigation around objects. 2 

Negotiating the obstacles. 1 

The most difficult task was negotiating the obstacle course using the flip screen. 2 

Speed and turning control, adjusting for the weaker resolution of the driving camera, 

and flipping back and forth between the camera and the map while navigating and 

attempting to remember the information using the "toggle" method. 

1 

Steering the robot took some getting used to. 1 

Steering the vehicle left and right without overcompensating. 1 
 

43 

3. What are your comments on the training course? 

Comments No. of Responses

Felt comfortable driving the robot after only fifteen minutes of initial training. 1 

Good course, technical difficulties were numerous although staff did a good job of 

working through them. 

1 

Good. 2 

I think training was well organized and that every trainer was trained themselves and 

capable. 

1 

More time on the obstacle negotiations would be beneficial. 1 

The course was set up well and enjoyed practicing with the robot and appreciate the 

opportunity. 

1 

The lengthy breaks in between operator rotations gives the operator sufficient time 

to rest eyes. 

1 

The robotic training course was very interesting and will be beneficial to me in my 

military career. 

1 

The staff was professional and thorough in their briefings and training. The 

expectations were clearly expressed and all participants were given the same 

treatment and scenarios. 

1 

The training course was excellent and very informative in future technology 1 

Overall the training was great, but I feel that more than one Robot should be 

available to continue training. When the Robot overheats due to mechanical 

failure there should be more time allotted, even though the time started once you 

restarted over again. You should always have second course of action if 

mechanical failures with equipment take place. 

1 

The training should have included all the types of terrain the robot should be able to 

operate. 

1 

Need a wider area to romp and gain a feel for the robot. 1 

The turnaround point should be more visible. 1 

4. Please rank order the displays in the order of your preference – with 1 being your 

favorite, 2 your second choice, etc. 

Display # preferred 

% 

preferred 

A (split screen) 39 52% 

B (toggle) 7 7% 

C (tactile) 31 41% 

Sum 75 100% 
 

44 

5. Do you have any suggestions for ways to increase the effectiveness of the displays? 

Comments No. of Responses

Display A (split screen) 

I thought it was easy to see both screens at the same time. 1 

It was good. 1 

Having the screen on the right instead of the bottom. 1 

I would move my eyes around less by using tactile with the map, i.e., in navigating 

in woods (or open desert) with everything seeming the same. The tactile would 

point me in the right direction and my eyes would remain on the view screen. It is 

another layer to navigating the robot. 

1 

If at all possible, try and work out some of the delay in the video, whether it be from 

power output or whatever requirements it might take. 

1 

If the GPS lag can be lessened the course would be easier. 3 

Increase the brightness of the screen. 1 

Increase the resolution of the main screen. It is hard to view the course depending 

on what part of the day it is. 

2 

Video with higher resolution. Bigger Image on the screen. 1 

Adjust the controls to compensate for the "touchiness". 1 

Make it bigger; the camera display is too small to see terrain details. 1 

Make the camera spin for corners and turns. 1 

Being able to make video map more stable. 1 

Why not add the tactile belt? 1 

Would like to flip the screens so map point is on top. 1 

Display B (toggle)

Did not like this one at all - very difficult to see hand signals, navigate and see your 

position on the map with the lag time factor. 

1 

Did not like this system at all. Felt disorientated and slow. 1 

Don't use it. One can easily miss hand signals using this display. 1 

Eliminate this method, having to toggle takes more time, senses, and it temporarily 

takes the solider out of the fight. That split second could mean life or death. 

1 

Have both screens. It added to frustration and the time of the navigation. 1 

Having the switch on the key instead of the pen. 1 

I don't really know what to say for improvement but this method seemed to be the 

least effective because you miss things jumping between screens. 

1 

I really did not like the flip screen. It took away from concentration as well as waste 

valuable time flipping back and forth between screens. 

1 

It was time consuming flipping to both screens. 1 

Make a secondary toggle switch on the controller (joystick). 2 

Maybe make the toggle some sort of switch you can hold in your hand. 1 

Being able to make video map more stable. 1 

Place arrow in corner of screen, arrow pointing toward objective. 1 
 

45 

Comments No. of Responses

Probably just don't use the toggle. Takes a moment to shift back and forth and gain a 

good understanding of navigation, could pose a hazard in a combat zone to 

soldiers. 

1 

Video with higher resolution. Bigger Image on the screen. 1 

The flip screen made it impossible to stay situationally aware at all times. If the 

processor were fast enough to allow for instant flipping it could work. As far as 

finger flashing in front of the camera though, they still would be a high chance that 

they would be missed. If you had the ability to record every time you flip the 

screen then you would know that you would not be missing anything, however the 

cost of such an option is something that I am not aware of. 

1 

The switching/flipping back and forth distracts from mission performance and 

accomplishment. 

1 

They both can be effective if trained correctly. The toggle can be built into a smaller 

unit and would be lighter. Make the GPS corrections quicker. 

1 

Display C (tactile)

Excellent. 1 

As with each display, It seemed that the camera had trouble with it's resolution over 

rough ground. It seems as though the pixels were blinking out. The problem 

corrected itself in a second or so. 

1 

It was still harder to know that you reached the way-point even with the signals. I 

got the signal but with just once I still questioned if I missed it being my first time. 

1 

Lag time creates inefficiencies. 1 

Make the electrode pads slightly larger to compensate for larger operators. 1 

Being able to make video map more stable. 1 

Place arrow in corner of screen, arrow pointing toward objective. 1 

Possibly having the split screen and the tactile method together would be a really 

efficient way of accomplishing the task. That way the sensors can guide you but 

you also have the map readily accessible to look at if you feel like you are getting 

off course. 

1 

Possibly integrate it with the split-screen. The only negative aspect was that you 

still had to toggle to the GPS. 

1 

The 6 o'clock signal was sometimes faint because of the shape of my back. This 

signal should be made stronger. 

1 

The belt could produce a higher output for a better feel of direction and it needs a 

faster return of data so operator does not get confused on actual direction needing 

to go. 

1 

Video with higher resolution. Bigger Image on the screen. 1 

The camera display could be larger. 1 

The problem I had with Tactile is that you needed to switch over to view the map. 1 
 

46 

Comments No. of Responses

The tactile display may be more efficient if the belt were used with the split screen 

display. 

1 

Why not add the split screen? I know the object of this research is to create the most 

compact and mobile system, but considering you are bringing a large robot with 

you, what difference does another few inches of a screen with (hopefully) better 

resolution make. 

1 

The tactile type was irritating at times because of the vibrations that were constantly 

present. It is a good system but the intensity was possibly too much for just putting 

on the screen. 

1 

Get rid of it. 1 

The vibrations are only helpful if other screens are unavailable. 1 

6. Please provide suggestions on ways to improve the driving lanes that you negotiated? 

I thought the lanes were set up well. 2 

I believe the terrain for the lanes were ok. Driving lanes should never be perfect. 1 

They were pretty good for reality wise because you never know what terrain you 

will come up against. So practicing in a more rough terrain is probably the best 

way. 

1 

Perhaps more (smaller but impassable) obstructions could be used. 1 

Add more obstacles. 1 

Color bushes and other objects that may harm the robot so navigation can be more 

accurate around obstacles instead of just running them over. 

1 

Don't run the lanes in the rain unless robot is further developed to handle wet 

conditions to prevent inaction during handling. 

1 

If the course requires that a soldier stand in front of ravines in order to tell you when 

to halt the robot, that should tell us that the camera display isn't picking up 

sufficient terrain detail for real-world use. 

1 

Improve on stabilizing the camera better while the robot travels over various terrain. 1 

Making sure that the paths are clear from major obstacles. Ran into a dead tree in 

the middle of one of the runs. 

1 

More variation other than the standard engineering tape and concertina wire. Maybe 

make a makeshift wall or two to simulate driving around buildings. 

1 

One test should be done on smooth or pavement area to see the full capacity of the 

robot. 

1 

Possibly make the engineering tape higher up. This would make picking a point and 

driving towards it easier, as you can see the lane further out. 

1 
 

47 

Comments No. of Responses

The tents that were provided were not very strong in the conditions here at Fort 

Benning. If better tents were provided it could have helped in the process. The 

wind was extremely rough today. 

1 

Use bright orange tape. 1 

Use red tape to identify obstacles, as the yellow tape was difficult to see because of 

poor screen resolution. 

1 

With the taller grass and brush it was sometimes hard to see the lines they came up 

quickly sometimes. I had to try and drive more carefully. 

1 

7. What are your comments on information needed for teleoperating a vehicle? 

A truly remarkable system that is very user friendly. Some navigation systems are 

definitely easier than others as noted previously. 

1 

Good. 1 

Operators must be given a driving course that focuses on compensating for on left / 

right steering delay. 

1 

The vehicle overheats often. 1 
 

48 

Acronyms 

2-D two-dimensional 

3-D three-dimensional 

ANOVA analysis of variance 

ARL U.S. Army Research Laboratory 

GPS global positioning system 

MRT Multiple Resource Theory 

NASA National Aeronautics and Space Administration 

OCS Officer Candidate School 

OCU operator control units 

SA situation awareness 

TARDEC Tank and Automotive Research, Development and Engineering Center 

TLX Task Load Index 

UAV unmanned aerial vehicle 

UCF University of Central Florida 

XGA extended graphics array 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION

49 

 1 DEFENSE TECHNICAL 

 (PDF INFORMATION CTR 

 only) DTIC OCA 

 8725 JOHN J KINGMAN RD 

 STE 0944 

 FORT BELVOIR VA 22060-6218 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 IMNE ALC IMS 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK TL 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK PE 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR ML J MARTIN 

 MYER CENTER RM 2D311 

 FT MONMOUTH NJ 07703-5601 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MZ A DAVISON 

 320 MANSCEN LOOP STE 115 

 FT LEONARD WOOD MO 65473 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MD T DAVIS 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 COMMANDANT USAADASCH 

 ATSA CD 

 AMSRD ARL HR ME DR HAWLEY 

 5800 CARTER RD 

 FT BLISS TX 79916-3802 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MM DR V J RICE 

 BLDG 4011 RM 217 

 1750 GREELEY RD 

 FT SAM HOUSTON TX 78234-5002 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MG R SPINE 

 BLDG 333 

 PICATINNY ARSENAL NJ 07806-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ARMC FIELD ELEMENTT 

 AMSRD ARL HR MH C BURNS 

 THIRD AVE BLDG 1467B RM 336 

 FT KNOX KY 40121 

 1 ARMY RSCH LABORATORY - HRED 

 AWC FIELD ELEMENT 

 AMSRD ARL HR MJ D DURBIN 

 BLDG 4506 (DCD) RM 107 

 FT RUCKER AL 36362-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MK MR J REINHART 

 10125 KINGMAN RD 

 FT BELVOIR VA 22060-5828 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MY M BARNES 

 2520 HEALY AVE 

 STE 1172 BLDG 51005 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MP D UNGVARSKY 

 POPE HALL BLDG 4709 

 BCBL 806 HARRISON DR 

 FT LEAVENWORTH KS 66027-2302 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MJF J HANSBERGER 

 JFCOM JOINT EXPERIMENTATION J9 

 JOINT FUTURES LAB 

 115 LAKEVIEW PKWY STE B 

 SUFFOLK VA 23435 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MQ M R FLETCHER 

 NATICK SOLDIER CTR 

 AMSRD NSC WS E BLDG 3 RM 343 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY-HRED 

 AMSRD ARL HR MT J CHEN 

 12423 RESEARCH PKWY 

 ORLANDO FL 32826 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION

50 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MT C KORTENHAUS 

 12350 RESEARCH PKWY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MS C MANASCO 

 SIGNAL TOWERS 

 BLDG 29808A RM 303 

 FT GORDON GA 30905-5233 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MU 

 6501 E 11 MILE RD MS 284 

 BLDG 200A 2ND FL RM 2104 

 WARREN MI 48397-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MF C HERNANDEZ 

 2421 NW AUSTIN RD STE 220 

 FT SILL OK 73503-9042 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MV 

 S MIDDLEBROOK 

 91012 STATION AVE RM 348 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MN R SPENCER 

 DCSFDI HF 

 HQ USASOC BLDG E2929 

 FT BRAGG NC 28310-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MW E REDDEN 

 BLDG 4 ROOM 332 

 FT BENNING GA 31905-5400 

 1 ARMY G1 

 (CD DAPE MR B KNAPP 

 only) 300 ARMY PENTAGON RM 2C489 

 WASHINGTON DC 20310-0300 

 1 US ARMY RSRCH DEV & ENGR 

 CMND 

 SYS OF SYS INTEGRATION 

 AMSRD SS T 

 6000 6TH ST STE 100 

 FT BELVOIR VA 22060-5608 

ABERDEEN PROVING GROUND

 1 DIR USARL 

 AMSRD ARL CI OK TP (BLDG 4600) 

 5 DIR USARL 

 AMSRD ARL CI OK TP 

 S FOPPIANO 

 AMSRD ARL HR 

 T LETOWSKI 

 AMSRD ARL HR MD 

 D HARRAH 

 AMSRD ARL HR MR 

 F PARAGALLO 

 AMSRD ARL HR SD 

 B AMREIN 

TOTAL: 33 (1 ELEC, 1 CD, 31 HCs)