Scalability of Robotic Controllers: 

An Evaluation of Controller Options 

by Rodger A. Pettitt, Elizabeth S. Redden, and Christian B. Carstens 

ARL-TR-4457 May 2008 

Approved for public release; distribution is unlimited.
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position 

unless so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or 

approval of the use thereof. 

DESTRUCTION NOTICE⎯Destroy this report when it is no longer needed. Do not return it to 

the originator. 
Army Research Laboratory 

Aberdeen Proving Ground, MD 21005-5425 

ARL-TR-4457 May 2008 

Scalability of Robotic Controllers: An Evaluation of 

Controller Options 

Rodger A. Pettitt, Elizabeth S. Redden, and Christian B. Carstens 

Human Research and Engineering Directorate, ARL 

Approved for public release; distribution is unlimited. 
ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no 

person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

 May 2008 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

 June through July 2007

5a. CONTRACT NUMBER 

5b. GRANT NUMBER

4. TITLE AND SUBTITLE 

 Scalability of Robotic Controllers: An Evaluation of Controller Options 

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

 622716H7099 

5e. TASK NUMBER 

6. AUTHOR(S) 

 Rodger A. Pettitt, Elizabeth S. Redden, and Christian B. Carstens (all of ARL) 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

 U.S. Army Research Laboratory 

 Human Research and Engineering Directorate 

 Aberdeen Proving Ground, MD 21005-5425

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-4457 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 

11. SPONSOR/MONITOR’S REPORT NUMBERS 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

 Approved for public release; distribution is unlimited.. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

This study, conducted at Fort Benning, Georgia, was an operational investigation of tele-operation control performance with the

use of three different robotic control devices. Twelve Soldiers from the Officers’ Candidate School and three Soldiers from 

Headquarters Company, 1st Battalion, 11th Infantry Regiment served as participants. Before any training, Soldiers provided an 

initial evaluation of the intuitiveness of controller features. After training in the operation of the IRobot PackBot Robot system, 

each Soldier completed a driving course using three different controller types. Controller A was the largest of the three 

controllers and each control manipulation had a single function. Both controller A and controller B had a similar number of 

single-function controls; however, controller B’s controls were laid in a different configuration and were smaller than controller 

A’s. Controller C had the fewest controls and the controls were multi-functional. Soldiers were tasked to drive the robot and to 

perform operations such as surveillance using the robotic arm. We measured workload for each controller was measured by 

having the Soldiers complete the NASA (National Aeronautics and Space Administration) Task Load Index survey after they 

used each controller type. Controller type and usability were evaluated, based on objective performance data, data collector 

observations, and Soldier questionnaires. The multifunctional controller was reported to be more difficult to learn and use than 

the controller with reduced control sizes because switching between functions was time consuming and confusing. This 

difficulty increased perceived workload. Soldiers also found that several robotic control functions (e.g., raising the control arm 

while turning the sensor head) could not be performed simultaneously with the multifunction controller. This necessitated 

sequential operation which was time consuming and difficult. Findings indicate that reducing the size of the individual controls 

shows promise as a valid approach for reducing overall controller size. 

15. SUBJECT TERMS

 control scalability; control size; dismounted Soldiers; robotic control; tele-operation 

16. SECURITY CLASSIFICATION OF: 

19a. NAME OF RESPONSIBLE PERSON

 Rodger A. Pettitt 

a. REPORT 

UNCLASSIFIED 

b. ABSTRACT 

UNCLASSIFIED

c. THIS PAGE 

UNCLASSIFIED

17. LIMITATION 

OF ABSTRACT 

SAR 

18. NUMBER 

OF PAGES 

85 19b. TELEPHONE NUMBER (Include area code) 

 (706) 545-9142

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18
iii 

Contents 

List of Figures v

List of Tables v

1. Introduction 1

1.1 Statement of the Problem ................................................................................................1 

1.2 Overview of Experiment .................................................................................................2 

1.3 Objectives........................................................................................................................3 

2. Method 3

2.1 Participants ......................................................................................................................3 

2.1.1 Pre-test Orientation and Volunteer Agreement ...................................................3 

2.1.2 Medical Review and Screening...........................................................................3 

2.2 Instruments and Apparatus..............................................................................................4 

2.2.1 IRobot PackBot Explorer Robot..........................................................................4 

2.2.2 Robotic Vehicle Controllers................................................................................4 

2.2.3 Control Intuitiveness Test ...................................................................................6 

2.2.4 Robotic Course....................................................................................................6 

2.2.5 NASA-TLX .........................................................................................................9 

2.2.6 Questionnaires .....................................................................................................9 

2.3 Procedures and Methodology..........................................................................................9 

2.3.1 Soldier Orientation ..............................................................................................9 

2.3.2 Control Intuitiveness Station.............................................................................10 

2.3.3 Training Station.................................................................................................10 

2.3.4 Robotic Driving Course.....................................................................................11 

2.3.5 End of Iteration Questionnaire Administration Station.....................................11 

2.4 Experimental Design .....................................................................................................12 

2.4.1 Independent Variable ........................................................................................12 

2.4.2 Dependent Variables .........................................................................................12 

2.5 Data Analysis ................................................................................................................12 

3. Results 12

3.1 Demographics................................................................................................................12 

3.2 Control Intuitiveness .....................................................................................................13 

3.2.1 Controller A.......................................................................................................13 

3.2.2 Controller B.......................................................................................................14 
iv 

3.2.3 Controller C.......................................................................................................15 

3.2.4 Summary ...........................................................................................................16 

3.3 Training .........................................................................................................................17 

3.4 Robotic Course..............................................................................................................17 

3.5 NASA-TLX...................................................................................................................20 

3.6 End of Experiment Questionnaire Results ....................................................................22 

4. Conclusions 22

5. References 24

Appendix A. Soldier Questionnaire Results 27

Appendix B. Control Intuitiveness Test Results 51

Distribution List 77 
v 

List of Figures 

Figure 1. IRobot PackBot explorer robot....................................................................................... 4 

Figure 2. Controller A.................................................................................................................... 5 

Figure 3. Controller B.................................................................................................................... 5 

Figure 4. Controller C.................................................................................................................... 6 

Figure 5. Robotic driving course. .................................................................................................. 8 

Figure 6. Controller A with labels. .............................................................................................. 13 

Figure 7. Controller B with labels................................................................................................ 14 

Figure 8. Controller C with labels................................................................................................ 15 

Figure 9. Proportion of correct responses.................................................................................... 17 

Figure 10. Mean course completion times (seconds)................................................................... 18 

Figure 11. Mean NASA TLX scale scores. ................................................................................. 21 

List of Tables 

Table 1. Controller data. ................................................................................................................ 4 

Table 2. Order of treatments. ....................................................................................................... 10 

Table 3. Robotic course stations. ................................................................................................. 11 

Table 4. Controller A weighted scores. ....................................................................................... 14 

Table 5. Controller B weighted scores......................................................................................... 15 

Table 6. Controller C weighted scores......................................................................................... 16 

Table 7. Proportion of correct responses. .................................................................................... 16 

Table 8. Mean course completion times. ..................................................................................... 18 

Table 9. Follow-on comparisons, course completion times. ....................................................... 19 

Table 10. Mean major off-course errors. ..................................................................................... 19

Table 11. Mean minor off-course errors...................................................................................... 19

Table 12. Mean driving errors. .................................................................................................... 19 

Table 13. Weighted means, NASA workload scales. .................................................................. 20 

Table 14. ANOVA summary, NASA TLX scales....................................................................... 21 

Table 15. Follow-on comparisons, total NASA TLX.................................................................. 21 
vi 

INTENTIONALLY LEFT BLANK
1 

1. Introduction 

1.1 Statement of the Problem 

Several streams of research are under way to delineate the degree and manner in which robotic 

controllers and displays can be scaled to the needs of Army Soldiers (Barnes, Knapp, Tillman, 

Walters, & Velicki, 2000; Barnes, Everett, & Rudakevych, 2005; Chen, Haas, & Barnes, 2007; 

Chen, Haas, Pillalamarri, & Jacobson, 2006; Renfro, Merlo, Duley, Gilson, & Hancock, 2007; 

Stafford, Jingjing, Merlo, & Hancock, 2007; Stafford, Hancock, Graham, & Merlo, 2007). This 

experiment is the second in a series of experiments designed to investigate current and future 

options for scaling robotic controls and displays specifically for use by dismounted Soldiers. 

The term “scalability” encompasses the various ways in which devices can be made more operationally effective, not only in terms of size and weight but also in terms of more intuitive display 

of information, enhancement of training transfer among other Army controls and displays, and 

minimization of information overload. For example, one operational definition of scalability is 

“The transmission of critical information to the Soldier tailored for each level of combat to ensure 

mission success while maximizing survivability by minimizing equipment requirements; minimizing multitasking workload, maximizing situation understanding; and maximizing aerial and 

ground robotic mission effectiveness” (Merlo, 2006). A similar definition is “The tailored 

reception and transmission of mission-essential information at the appropriate level for the 

Soldier, to ensure mission success while maximizing the survivability and lethality through the 

synergistic interaction of equipment requirements, appropriate cognitive workload, situation 

awareness and understanding for oneself and others connectivity of distributed intelligent agents” 

(Barnes, 2006). 

In our stream of research, we focus on size, weight, and ease of use for dismounted Soldier 

applications. Soldiers operate in a large range of environments, from the relatively stable and 

spacious environment of a tactical operations center (TOC) to the cramped environment of a 

vehicle to the rugged and challenging environment of the dismounted Soldier. All these environments have an impact on the effective size and configuration of the robotic interface. A 

dismounted Soldier cannot carry the relatively large controller that can be used in a TOC. Small 

input devices are quite different from larger ones and can potentially impact the complexity of 

operation, the speed of operation, and the accuracy of input. Designing for the “optimum” input 

device size may seem to be a good idea because the presentation of the interface may seem to be 

controlled. However, if an input device is designed that is only practical in one environment, it 

may be completely unusable in another. 
2 

Certainly, smaller devices that weigh less are usually preferred by dismounted Soldiers. The 

question is how best do we accomplish controller size reduction? Tele-operation of robotic 

vehicles is a difficult task because robots are complex machines, typically consisting of a large 

number of components. Many of the robotic components provide feedback to the operator and 

require input from the operator. Typically, the controller consists of at least two joysticks or 

comparable controls for robotic navigation and camera control. In addition, there are many 

buttons and switches and one or more video screens. Space to incorporate controls on small 

controllers is very limited. 

Miniaturizing individual controls is one option for reducing the size of controllers, but Soldiers 

have to be able to operate controllers while wearing gloves without accidentally activating adjacent 

controls. One benefit of the miniaturization approach is that if we simply make existing controllers smaller, the similarity of controls should enhance transfer of training. However, how small 

can we go before the buttons and spacing are too difficult for operational performance? Another 

option is to reduce the number of individual controls on controller devices by incorporating 

multifunction controls (controls that can perform different functions, depending on the current 

mode of operation). For example, a driving joystick can become a camera pan-and-tilt controller 

when the system is placed in another mode of operation. However, multifunction controls may 

increase control activation time and cognitive complexity. The end result could be that a controller 

with multifunction controls may require many hours of training before a user is competent and 

comfortable enough to use the system during a real mission. Other creative approaches to controller size reduction that have been developed are sketch interfaces (Skubic, Bailey & Chronis, 2003; 

Setalaphruk, Ueno, Kume, & Kono, 2003), voice recognition and synthesis systems (Chen, Haas, 

Pillalamarri, & Jacobson, 2006), and hands-free systems (Veronka & Nestor, 2001), but these 

novel controls often present problems of their own and they are still being refined. 

In the meantime, there is an immediate requirement for dismounted Soldier robotic tele-operation 

controllers and not much empirical data about the impact of size reduction. In this experiment, we 

chose three different types of controllers and demonstrated their potential for use by a dismounted 

Soldier to control a small robotic platform and its sensor payload. One of the controllers was the 

default controller for the robot used in this study. We also included two controllers that reflected 

near-term options for scaling controllers for the dismounted Soldier. All controllers were compared on the same set of measures. Pre-test experimentation was performed to ensure that the 

functional mapping of each of the controllers was effective as possible. The trade-offs between 

interface size and weight with these controllers versus input speed, accuracy, training time, and 

cognitive load were examined. 

1.2 Overview of Experiment 

This study was an investigation of the effect of controller size scalability on robotic control. It 

took place at Fort Benning, Georgia, with 12 Soldiers from the Officers’ Candidate School 

(OCS) and three Soldiers from Headquarters Company 1st Battalion 11th Infantry Regiment as 
3 

participants. Before any training, an intuitiveness evaluation was conducted on each controller. 

After training in the operation of the IRobot1 PackBot1 Robot system, each Soldier completed a 

driving course using three different controller types. The terrain and hazards were counterbalanced along with the controller type to control for the effect of learning. Workload for each 

controller was measured by having the Soldiers complete the National Aeronautics and Space 

Administration (NASA) Task Load Index (TLX) survey after they completed the driving course. 

Controller type and usability were evaluated, based on objective performance data, data collector 

observations, and Soldier questionnaires. 

1.3 Objectives 

There were two objectives in this experiment. The primary objective was to determine whether 

differences among the controller design approaches had any effect on the operation of a small 

robot. The second objective was to conduct an intuition test of the controller design and function 

mapping of each controller. 

2. Method 

2.1 Participants 

Twelve Soldiers from the OCS and three Soldiers from Headquarters Company 1st Battalion 11th 

Infantry Regiment participated in the study. 

2.1.1 Pre-test Orientation and Volunteer Agreement 

The Soldiers were given an orientation about the purpose of the study and their participation. 

They were briefed about the objectives and procedures, as well as the robot. They were also told 

how the results would be used and the benefits the military could expect from this investigation. 

Any questions the subjects had regarding the study were answered. It was made clear that Soldier 

participation in the experimentation was voluntary. The volunteer agreement affidavit was explained and its contents verbally presented. The Soldiers were given the volunteer agreement 

affidavit to read and sign if they decided to volunteer. 

2.1.2 Medical Review and Screening 

The Soldiers were given a medical status form to determine if any of them had a medical profile 

or history that would jeopardize them if they participated in the study. 

 1iRobot and PackBot are registered trademarks of iRobot Corporation, Burlington, Massachusetts.
4 

2.2 Instruments and Apparatus 

2.2.1 IRobot PackBot Explorer Robot 

The IRobot PackBot Explorer Robot (figure 1) is a portable small unmanned ground vehicle 

(SUGV) reconnaissance and tactical robot that can enter and secure areas that are inaccessible or 

too dangerous for humans. The IRobot PackBot Explorer Robot payload has a rotating pan-andtilt head equipped with multiple cameras. 

2.2.2 Robotic Vehicle Controllers 

The three controller types used during this experiment are listed in table 1. Controller A was the 

largest of the three controllers, and each control manipulation had a single function. Controllers A 

and B had the same number of single-function controls; however, controller B’s controls were 

placed in a different configuration and were smaller than controller A’s. Controller C attached to 

the bottom rail of the M4 in the same fashion as a forward hand grip. Controller C had the fewest 

controls, and the controls had multi-functionality. Controllers A and B were designed to be 

operated with both hands, while controller C was designed for single-handed operation. Pictures 

of the three controllers are shown in figures 2 through 4. 

Figure 1. IRobot PackBot explorer robot. 

Table 1. Controller data. 

Controller Label Controller Name No. of Possible 

Control 

Manipulations 

Controller Weight 

(lb) 

Controller 

Dimensions 

(LxWxH in.) 

A Legacy Controller 30 1.40 8.5 x 4.25 x 4.25 

B Logitech Controller 26 .56 6 x 4 x 2.38 

C Q3D Controller 11 .65 6 x 2.63 x 7 
5 

DOCU for Explorer: 

Mappings 

(Legacy Controller) 

Figure 2. Controller A. 

DOCU for Explorer: 

Mappings 

(Logitech2 Controller) 

Figure 3. Controller B. 

 2Logitech is a registered trademark of Logitech, Inc. 
6 

Figure 4. Controller C. 

2.2.3 Control Intuitiveness Test 

The control intuitiveness test performed in this experiment was roughly based on an icon 

intuitiveness study done by Nielsen and Sano (1994), which was performed during the usability 

evaluation of Sun Microsystems’ internal web. Nielsen and Sano presented images to users who 

were then asked to indicate what functionality they thought the icons represented. In the present 

study, we presented a task (i.e., raise the arm of the sensor head, raise the flippers, drive the robot 

forward, brake, pan the camera to the left) to the Soldiers who had no prior training or use of the 

controller and then asked them to perform the task. The administrator documented the number of 

correct and incorrect attempts to perform the task and noted the specific controls that were 

incorrectly activated. 

2.2.4 Robotic Course 

The experiment was conducted on a driving course approximately 200 meters long (figure 5). The 

driving lane on the course was approximately 1 meter wide and clearly marked with white engineer 

tape on the left and right sides. Along the course were seven stations where the operator conducted 

a specified maneuver or reconnaissance task. The seven stations were stair ascend/descend; window reconnaissance; target identification/tracking; hill climb; bunker reconnaissance; tunnel reconnaissance; and zigzag negotiation. Soldiers tele-operating the IRobot PackBot were situated at a 

remote stationary position inside a tent. The tent kept the operator and the controller equipment out 

of the elements. A description of each station and instructions for executing them follow: 
7 

• Station 1. Stairs (Maneuver). The stairs are made of wood. Five steps lead up to a 

platform and five steps lead down. In order to ascend and descend the stairs, the operator 

must correctly position the robot flippers to obtain the appropriate angle and control the 

robot’s speed throughout the maneuver. 

• Station 2. Window (Maneuver/Reconnaissance). The operator’s task at this station is to 

locate and identify the simulated improvised explosive device (IED) hidden behind the 

urban wall. Upon reaching the wall, the operator must back the robot into position below 

the window, extend the sensor head arm, and maneuver the sensor head in order to see 

through the window opening. The operator locates and identifies to the data collector a 

pipe bomb, ammunition can bomb, or soda can bomb. 

• Station 3. Target Tracking (Maneuver/Reconnaissance). The operator positions the robot 

at the row of sandbags and locates the human target. The operator zooms in on the human 

target and identifies the object he is holding as a sledge hammer, M4 carbine, shovel, or 

box. The human target is then cued by the data collector to start walking perpendicular to 

the robot. At set intervals along the route, the human target discards the object and 

replaces it with a different object. The operator is required to track the target and report 

to the data collector what the target is carrying at all times throughout the exercise. 

• Station 4. Hill (Maneuver). The hill is approximately 10 m long, 1 m wide, and 3 m tall 

with a 30-degree sloped incline and decline. The operator must control the robot’s 

direction and speed throughout the maneuver.

• Station 5. Bunker Reconnaissance (Foxhole) (Maneuver/Reconnaissance). The operator’s 

task at this station is to locate and identify the simulated IED marked with an alphanumeric 

label hidden in a partially covered foxhole. Upon reaching the foxhole, the operator must 

back the robot into position, extend the sensor head arm, and maneuver the sensor head in 

order to see through the foxhole opening. After the operator locates the IED, s/he must use 

the camera zoom feature and report the alphanumeric label to the data collector.

• Station 6. Tunnel (Maneuver). The tunnel is a culvert pipe 6 m long by 1 m in diameter 

and made of corrugated steel. The operator aligns the robot with the tunnel opening and 

maneuvers the robot through the pipe. 

• Station 7. Zigzag (Maneuver). The zigzag is 1.6 m tall, 14 m long, and approximately 1 m 

wide. It consists of three turns (approximately 90 degrees each) within the lane. The framework is constructed of wood with mesh wire installed between the two lanes and on the 

outside framework of each lane. The operator maneuvers the robot through the zigzag, 

controlling both speed and direction.
8 

Figure 5. Robotic driving course. 
9 

2.2.5 NASA TLX 

The NASA TLX requires the user to rate the workload of a device on a number of different scales 

and to assign an importance weight to each scale. The scores on the workload scales (mental, 

physical, temporal, performance, effort, and frustration) can be combined in an overall workload 

score (Hart & Staveland, 1988). 

2.2.6 Questionnaires 

The questionnaires were designed to elicit Soldiers’ opinions about their performance and 

experiences with each of the controller systems. The questionnaires asked the Soldiers to rate 

the devices on a 7-point Likert scale ranging from “extremely bad/difficult” to “extremely 

good/easy”. Questionnaires were administered to each Soldier at the end of each iteration and at 

the end of the experiment. Questionnaires were also used to gather information concerning the 

Soldiers’ demographic data, robotic experience, and physical characteristics that might affect their 

ability to operate the robot. 

2.3 Procedures and Methodology 

2.3.1 Soldier Orientation 

The experiment Soldiers reported in groups of three for one day each, 0800 to 1800 daily. They 

then received a roster number, which was used to identify them throughout the evaluation. After 

their pre-test orientation, the volunteer agreement affidavit was explained and its contents verbally 

presented. Next, the Soldiers were given the volunteer agreement affidavit to read and sign if they 

decided to volunteer. The volunteers were then asked to complete the medical review and screening form. Demographic data, as well as data concerning their Army and robotic experience and 

physical characteristics, were also taken for each Soldier. Next, Soldiers received an orientation 

of the robotics driving course. During the orientation, trainers demonstrated the tasks to be 

performed at each station. The intuitive test and the training course were also explained and any 

questions the Soldiers had concerning the experiment were answered. 

Soldiers completed three iterations of the four stations (one with each controller type) according 

to the matrix in table 2. They started at station 1 (the intuitiveness exercise), then moved to 

station 2 (the training course), then moved to station 3 (the robotic course), and finally moved 

to station 4 (the questionnaire station). After the third iteration, they completed an end of experiment questionnaire. 
10 

Table 2. Order of treatments. 

Roster (Soldier) Iteration

1 2 3 

1 A B C 

2 B C A 

3 C A B 

4 C B A 

5 B A C 

6 A C B 

7 A B C 

8 B C A 

9 C A B 

10 C B A 

11 B A C 

12 A C B 

13 A B C 

14 B C A 

15 C A B 

2.3.2 Control Intuitiveness Station 

At the intuitiveness station, Soldiers were given a controller with all labels covered or removed 

and were asked to activate the control that they thought would most likely cause the robot to 

perform a specified task. Soldiers were then asked why they selected the control. Five tasks 

were selected for the test: raise the arm of the sensor head, raise the flippers, drive the robot 

forward, brake, and pan the camera to the left. Soldiers were given three attempts to select the 

correct control and were asked after each selection why they selected it. If a Soldier selected the 

correct control, s/he moved to the next task. The administrator documented the number of incorrect attempts to perform the task, noted the specific controls that were incorrectly activated, and 

recorded the Soldier’s feedback concerning why each control input was chosen. After the Soldiers 

completed the tasks with all three controllers, they were asked to complete a questionnaire 

designed to elicit information about the intuitiveness of each controller’s design. 

2.3.3 Training Station 

A representative from iRobot trained the Soldiers to use the IRobot PackBot. Soldiers practiced 

tele-operating the robot on the actual course used during the experiment. They were trained on 

each controller just before executing the robotic course with that controller. The course required 

driving, obstacle negotiation, sensor head manipulation, target identification, and moving target 

tracking. It also involved training in the graphical user interface (GUI), use of the video cameras, 

and use of the robotic flippers to negotiate obstacles such as stairs and low walls. Soldiers were 

trained on each controller for 30 minutes. During pilot testing, it was found that 30 minutes of 

hands-on training with each controller was adequate for Soldiers to become proficient in performing the various tasks required to complete the driving course. Upon completion of the 

training, the Soldiers negotiated the robotic driving course. 
11 

2.3.4 Robotic Driving Course 

Soldiers negotiated the robotic driving course by tele-operating the robot using the three controller 

types. The course required driving, obstacle negotiation, sensor head manipulation, target identification, and moving target tracking. 

The robotics course was approximately 200 m long and had seven stations along its route (see 

table 3). The Soldiers were told to maneuver the robot along the course marked with engineer tape 

and to conduct a reconnaissance at key locations along the route. At each station, the operator was 

instructed to perform a specified maneuver or reconnaissance task. A data collector following the 

robot recorded the time required for the Soldier to complete each station, driving errors, and the 

number of times the robot was driven off course. A driving error was defined as running into 

objects on the course or failing to properly position the robot to perform the task required at a 

station. Off course was defined as major off course or minor off course. A major off course was 

recorded if the robot went completely outside the engineer tape and a minor off course was recorded when only a portion of the robot crossed the tape. An additional data collector was colocated with the Soldier to observe and record the Soldier’s responses and to record overall course 

completion times. 

Table 3. Robotic course stations. 

Station Description Task 

Station 1 Stair Ascend/Descend Maneuver 

Station 2 Window Target Maneuver/Reconnaissance 

Station 3 Area Scan/moving target Maneuver/Reconnaissance 

Station 4 Hill Maneuver 

Station 5 Bunker Reconnaissance (Foxhole) Maneuver/Reconnaissance 

Station 6 Tunnel Maneuver 

Station 7 ZigZag Maneuver 

Upon completion of the course, the Soldiers were given a questionnaire designed to assess their 

performance and experiences with each of the control systems and to assess their perception of 

the training adequacy. Questions about the amount of practice time given, the level of detail 

presented, and the adequacy of training aids were included. 

2.3.5 End of Iteration Questionnaire Administration Station 

End of iteration questionnaires and the NASA-TLX were administered to each Soldier at the end 

of each iteration. After completing the course with each of the three controller types, the Soldiers 

completed an end of experiment questionnaire that compared each of the controller types for a 

number of characteristics. They also completed questionnaires concerning the information 

requirements for tele-operating the robot. 
12 

2.4 Experimental Design 

This experiment used a within-subject design, with treatments counterbalanced to control for 

sequence effects. 

2.4.1 Independent Variable 

• Controller type

2.4.2 Dependent Variables 

• NASA TLX workload scores, 

• Robotic course completion time, 

• Time to complete each station on the course, 

• The number of major times the driver went off course, 

• The number of minor times the driver went off course, 

• The number of driving errors on the course, 

• Number and type of errors committed during each task of the intuition test, 

• Data collector comments, and 

• Questionnaire responses. 

2.5 Data Analysis 

All objective data were analyzed with a repeated measures analysis of variance (ANOVA). 

Follow-on pair-wise comparisons were done with Holm’s Bonferroni procedures to control for 

family-wise error rates. Partial η2

 (η2

p), an index of effect size, was computed for each ANOVA. 

Iteration effects were controlled through the counterbalanced order of the experimental design. 

Soldier questionnaire data were analyzed with the use of descriptive statistics on the subjective 

ratings. 

3. Results 

3.1 Demographics 

The rank of the enlisted Soldiers ranged from E-3 to E-7, with one chief warrant officer in the 

rank of CW3. Their average time in service was 69 months. Two of the Soldiers wrote with 

their left hand and all 15 Soldiers fired their weapons right handed. Nine of the Soldiers had 
13 

driven a remote controlled car, but none had ever operated a robotic vehicle. All but three of the 

Soldiers played video games on a regular basis. Detailed results are presented in appendix A. 

3.2 Control Intuitiveness 

The results from the intuitiveness test were weighted based on the number of attempts to find the 

correct control. Three points were given for the first selection, two for the second, and one for 

the third. Therefore, the higher weights represent the most intuitive control mapping and the 

lower weights represent the least intuitive. 

The following three figures show each controller with the controls labeled with numbers used to 

identify which control the Soldiers selected during the intuitiveness test. The following three 

tables show the tasks, the control with the highest intuitiveness weight, and the weight given for 

the control currently mapped to perform the task. The complete list of responses (including those 

that did not receive the highest weightings) are presented in appendix B. 

3.2.1 Controller A 

The controls on controller A labeled 2 through 17 are actuated by buttons; 1A through 1F and 18A 

through 18F are actuated by toggle manipulations (figure 6). Results for controller A (table 4) 

show that the current controller mappings for raising the arm of the sensor head (18E) and raising 

the flippers (1F) were not selected as the most intuitive control manipulations to perform these 

tasks. Soldiers justified their selections, based on the assumption that the toggles were mapped for 

important functions such as driving and camera manipulations. The current mapping for driving 

the robot forward, applying brakes, and panning the camera left were weighted as the most 

intuitive control manipulations for these tasks. Detailed results are presented in appendix B. 

Figure 6. Controller A with labels. 
14 

Table 4. Controller A weighted scores. 

Task Control Selected 

(figure 6) 

Weighted Score 

2 13 

18A 13 

1. Raise the arm of the sensor head. 

18E* 2* 

2. Raise the flippers 2 11 

1F* 3* 

3. Drive the robot forward. 1A* 37* 

4. Brake 14* 12* 

5. Pan the camera left 18C* 17* 

* = Currently mapped control 

3.2.2 Controller B 

The controls on controller B (figure 7) labeled 1 through 8 and 11 through 18 are actuated by 

buttons; 9A through 9D and 10A through 10D are actuated by toggle manipulations. Results for 

controller B (table 5) show that the current controller mappings for raising the arm of the sensor 

head (2) and applying brakes (7) were not selected as the most intuitive control manipulations to 

perform these tasks. Soldiers based their selection of 10A for raising the sensor head on the 

assumption that the toggles should control the camera and 14 for braking based on the assumption 

that the directional pad should control the robot’s speed. The current mappings for driving the 

robot forward, raising the flippers, and panning the camera left were weighted as the most intuitive 

control manipulations for these tasks. 

Figure 7. Controller B with labels. 
15 

Table 5. Controller B weighted scores. 

Task Control Selected 

(figure 7) 

Weighted Score 

10A 16 

2* 12* 

1. Raise the arm of the sensor head. 

17* 22* 

2. Raise the flippers 9A* 36* 

14 18 

3. Drive the robot forward. 7* 5* 

4. Brake 10C* 21* 

5. Pan the camera left 10A 16 

* = Currently mapped control 

3.2.3 Controller C 

The controls labeled 2 through 5 on controller C (figure 8) are buttons; 1A through 1E are 

actuated by a toggle, and 6A and B are actuated when the hand grip on the control is twisted. 

Results for controller C (table 6) show that the current controller mapping for raising the arm of 

the sensor head (1D) was not selected as the most intuitive control manipulation to perform the 

task. Soldiers stated that raising and lowering the arm of the sensor head should be accomplished 

by pushing up or down on the toggle. The current mappings for raising the flippers, driving the 

robot forward, applying brakes, and panning the camera left were weighted as the most intuitive 

control manipulations for these tasks. 

Figure 8. Controller C with labels. 
16 

Table 6. Controller C weighted scores. 

Task Control Selected 

(figure 8) 

Weighted Score 

1A 34 

1D* 2* 

1. Raise the arm of the sensor head. 

1A* 21* 

2. Raise the flippers 1A* 39* 

5* 18* 

3. Drive the robot forward. 1C* 43* 

4. Brake 1A 34 

5. Pan the camera left 1D* 2* 

* = Currently mapped control 

3.2.4 Summary 

Table 7 and figure 9 show the proportion of correct responses (choice of the currently mapped 

control) to each command for the three controllers. A correct response is defined as the response 

being correct on the first, second, or third choice. The differences in proportion of total correct 

responses between the three controllers is statistically significant: χ

2

 = 9.90, df = 2, p < .01. One 

reason for the higher proportion of correct responses with the C controller is that there are only 

11 possible responses with the C device, as opposed to 30 possible responses with the A 

controller and 26 possible responses with the B controller. 

Table 7. Proportion of correct responses. 

Controller (percent) 

Task A B C 

1. Raise the arm of the sensor head 3 16 3 

2. Raise the flippers 3 26 29 

3. Drive the robot forward. 74 52 100 

4. Brake 21 7 25 

5. Pan the camera left 28 30 82 
17 

Figure 9. Proportion of correct responses. 

3.3 Training 

The Soldiers’ overall ratings of the training course were between good and extremely good for 

all three controllers. Most test Soldiers rated the C controller as more difficult to learn than the 

A and B controllers primarily because they had to learn the different functions (modes) for each 

control. Controlling the camera and adjusting to the controllers’ sensitivities were identified as 

tasks that were difficult to learn for all three controllers. Soldiers commented that the 30 minutes 

allowed to train with each controller was adequate time for learning to negotiate the course. 

Detailed training questionnaire results are presented in appendix A. 

3.4 Robotic Course 

Table 8 and figure 10 show the mean course completion times for the three controllers. The error 

bars in the graph represent 95% confidence intervals. A repeated measures ANOVA indicated that 

there was a significant difference among the means: F(2,24) = 7.07, p = .004, η2

p = .371. Followon pair-wise comparisons were conducted with Holm’s Bonferroni sequential correction for 

family-wise error rates. As shown in table 9, the average course completion time for controller C 

was significantly slower than the average times for controllers A and B. The time required to 

switch function modes and the limited ability to perform simultaneous tasks when Soldiers were 

tele-operating the robot using controller C were identified as the primary reasons for its slower 

completion times. Performing some tasks simultaneously with controller C, such as driving the 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

 Raise arm

of sensor

head

Raise

flippers

Drive 

robot

forward

Brake Pan

camera left

Command

A

B

C
18 

robot forward while raising the sensor arm, is not possible because Soldiers have to switch modes 

of the multifunction controls. 

Table 8. Mean course completion times. 

Controller Mean SD 

A 16:17 3:44 

B 16:07 5:49 

C 20:35 3:37 

A BC

700

800

900

1,000

1,100

1,200

1,300

1,400

95% CI

Figure 10. Mean course completion times (seconds). 
19 

Table 9. Follow-on comparisons, course completion times. 

Pair T df obtained p required p

A vs. B 0.22 14 0.827 0.05 

A vs. C -3.84 12 0.002* 0.0167 

B vs. C -2.84 12 0.015* 0.025 

* p < .05, 2-tailed 

Table 10 shows the mean number of major off-course errors. There were too few major errors to 

conduct a repeated measures ANOVA. 

Table 10. Mean major off-course errors. 

Controller Mean SD 

A 2.00 0 

B 1.67 0.52 

C 1.67 0.58 

The mean numbers of minor off-course errors for the three controllers are shown in table 11. 

There was no statistically significant difference among the means: F(2,14) < 1.00, p = .639, 

η2

p = .062. 

Table 11. Mean minor off-course errors. 

Controller Mean SD 

A 4.13 4.64 

B 4.88 2.70 

C 3.38 1.51 

The average numbers of driving errors between stations for each controller are shown in table 12. 

There was no statistically significant difference among the means: F(2,16) = 1.17, p = .336, 

η2

p = .128. 

Table 12. Mean driving errors. 

Controller Mean SD 

A 4.11 5.04 

B 6.22 2.82 

C 4.00 2.18 

Soldiers were able to successfully perform all the driving and reconnaissance tasks presented on the 

robotic driving course using all three controllers. Upon completion of each iteration, Soldiers were 

asked to rate the characteristics of the controller they had used and their ability to perform various 

tasks while maneuvering through the driving course. When asked to rate the characteristics of each 

controller, Soldiers rated controller B highest for size, weight, shape, comfort of use, and location 

of individual controls. Several Soldiers stated that controlling the robot’s rate of speed and 
20 

direction using the B controller was difficult because of the sensitivity of its driving controls. 

Soldiers stated that the large size of controller A’s controls allowed them to be labeled and made 

finding the correct control easy. 

Performing multiple tasks simultaneously (i.e., raising the control arm while tilting the sensor 

head) was rated as very difficult with the C controller. Controlling more than one robotic task 

simultaneously with one hand requires more dexterity than when the tasks are performed with 

both hands. Several Soldiers reported that they experienced wrist, arm, hand, and finger strain 

when operating the C controller. Supporting the weight of the weapon while manipulating the 

controller and performing all tasks with one hand were identified as the primary reasons Soldiers 

were experiencing strain. Detailed Soldier questionnaire results for the driving course are 

presented in appendix A. 

3.5 NASA TLX 

Table 13 shows the weighted means for each of the TLX scales, as well as the weighted means 

for the total workload score. The scale score means are illustrated in figure 11. The graph 

clearly indicates that the mental activities formed the greatest portion of task workload. The C 

controller has a slightly higher mean workload rating than controllers A or B in terms of mental 

demands, effort, and frustration. 

Table 13. Weighted means, NASA workload scales. 

Mental Physical Temporal Performance 

Controller Mean SD Mean SD Mean SD Mean SD 

A 13.2 9.2 0.5 0.9 4.1 2.6 8.0 7.2 

B 13.3 7.6 1.2 3.1 6.2 7.0 6.8 6.6 

C 17.3 9.1 2.2 2.9 6.2 6.2 5.8 5.3 

Effort Frustration Total 

Controller Mean SD Mean SD Mean SD 

A 6.5 5.3 4.3 5.3 36.6 13.5 

B 5.3 4.7 7.0 9.1 39.7 16.1 

C 8.7 5.8 9.5 10.6 49.7 23.8 
21 

0

2

4

6

8

10

12

14

16

18

20

Mental Physical Temporal Perf. Effort Frust.

Scale

Weighted Mean

A

B

C

Figure 11. Mean NASA TLX scale scores. 

Repeated measures ANOVAs were used to compare the weighted means for each of the workload 

scales. The results are summarized in table 14. The difference among the controller means was 

significant only for the total scale. 

Table 14. ANOVA summary, NASA TLX scales. 

Scale df F p η2

p

Mental 2,28 2.77 0.080 0.165 

Physical 2,28 3.24 0.054 0.188 

Temporal 2,28 1.27 0.297 0.083 

Effort 2,28 < 1.00 0.638 0.032 

Performance 2,28 1.92 0.165 0.121 

Frustration 2,28 1.97 0.158 0.123 

Total 2,28 3.93 0.031* 0.219 

* p < .05 

The follow-on pair-wise comparisons for the total scale are summarized in table 15. Although 

none of the pair-wise differences were statistically significant, the difference between controllers 

A and C approached statistical significance. 

Table 15. Follow-on comparisons, total NASA-TLX. 

Pair t df obtained p required p

A vs. B -1.09 14 0.296 0.05 

A vs. C -2.62 14 0.020 0.0167 

B vs. C -1.62 14 0.127 0.025 
22 

3.6 End of Experiment Questionnaire Results 

Most Soldiers felt that controller A was the quickest and easiest to use because of its size and 

control layout. Controller B was selected by most Soldiers as the most comfortable controller to 

use because of its size, weight, and design. Controller C was selected as the easiest to carry on a 

mission since it attaches directly to the weapon. Soldiers commented that they would like the 

ability to set the sensitivity level for each controller, based on the difficulty they experienced 

controlling the robot’s rate of speed and problems encountered with over-steering when 

maneuvering. 

Soldiers felt that reducing a controller size by shrinking the size of the individual controls would 

impact performance less than reducing its size by reducing the number of controls. Soldiers 

indicated that the multifunction controller was more difficult to learn and use than the controller 

with reduced control sizes and that switching between functions was time consuming and 

confusing. Detailed end of experiment questionnaire results are presented in appendix A. 

4. Conclusions 

The primary objective of this study was to evaluate controller devices that differ in features 

relevant to scaling robotic control devices for the dismounted Soldier. Soldiers were able to 

successfully perform all the driving and reconnaissance tasks presented on the robotic driving 

course using all three controllers. The results provide an indication that reducing a controller 

size by shrinking the size of the individual controls (controller B) had less adverse impact on 

performance than reducing its size by reducing the number of controls (controller C). However, 

when one is interpreting the results of this experiment, it is important to understand that there were 

limitations in the availability of controllers that were identical in all aspects (control mappings, 

sensitivity, etc.). Thus, uncontrolled variables could have played a role in the outcomes. However, 

several findings that support this conclusion should not have been impacted by the uncontrolled 

variables. First, Soldiers reported that the multifunction controller was more difficult to learn and 

use than the controller with reduced control sizes because switching between functions was time 

consuming and confusing. This difficulty increased perceived workload, which is a particularly 

important finding for dismounted operations because Sterling and Perala (2007) found that robotic 

controllers supporting infantry units had higher workloads and stress than controllers supporting 

other types of units. Second, Soldiers found that several robotic control functions could not be 

performed simultaneously (e.g., raise the control arm while turning the sensor head) with the 

multifunction controller. This necessitated sequential operation which was time consuming and 

difficult. Therefore, despite the uncontrolled variables, the conclusion that miniaturization provides 

less adverse impact on performance reduction of the number of controls appears to be a valid one. 
23 

The intuitiveness test used during this experiment to evaluate the control mapping performed on 

the controllers shows promise as an evaluation technique. It would be especially useful as a quick 

and easy way to use target audience Soldiers to evaluate different mappings on the same controller. 
24 

5. References 

Barnes, M.J. Personal correspondence, U.S. Army Research Laboratory: Fort Huachuca, AZ, 

2006. 

Barnes, M.J.; Everett, H. R.; Rudakevych, P. ThrowBot: Design Considerations for a Manportable Throwable Robot. Proceedings of SPIE Proc. 5804: Unmanned Ground Vehicle 

Technology VII. (http://www .nosc.mil /robots/pubs/SPIE5804-59.pdf), 2005. 

Barnes, M.J.; Knapp, B.G.; Tillman, B.W.; Walters, B.A.; Velicki, D. Crew Systems Analysis of 

Unmanned Aerial Vehicle (UAV) Future Job and Tasking Environments; ARL-TR-2081; 

U.S. Army Research Laboratory: Aberdeen Proving Ground, MD, 2000. 

Chen, J.Y.C.; Haas, E.; Barnes, M. Human Performance Issues and User Interface Design for 

Teleoperated Robots. IEEE Transactions on Systems, Man, and Cybernetics 2007, 37, 1231-

1245. 

Chen, J.Y.C.; Haas, E.; Pillalamarri, K; Jacobson, C. Human-Robot Interface: Issues in 

Operator Performance, Interface Design, and Technologies; ARL-TR-3834; U.S. Army 

Research Laboratory: Aberdeen Proving Ground, MD, 2006. 

Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of Empirical 

and Theoretical Research. In P. A. Hancock, & N. Meshkati (Eds.), Human Mental 

Workload (pp. 139-183). Amsterdam: North-Holland, 1988. 

Merlo, J. Personal correspondence, U.S. Army: West Point, NY, 2006. 

Nielsen, J.; Sano, D. SunWeb: User Interface Design for Sun Microsystem’s Internal Web. In 

Proc. 2nd World Wide Web Conf. ‘94: Mosaic and the Web. Chicago, IL, pp. 547.557. 

Also available in URL http://archive.ncsa.uiuc.edu/SDG/IT94/Proceedings/HCI/nielsen/ 

sunweb.ht%m, 1994. 

Renfro, M. B.; Merlo, J. L.; Duley, A.; Gilson, R.; Hancock, P. A. A Multimodal Approach to 

Unihabited Vehicle Operations in Dynamic Environments. Division 19/21 Midyear 

Conference of the American Psychological Association, Fairfax Virginia, 2007. 

Setalaphruk, V.; Ueno, A.; Kume, I.; Kono, Y. Robot Navigation in Corridor Environments 

Using a Sketch Floor Map. Proceeding of the 2003 IEEE International Symposium on 

Computation Intelligence in Robotics and Automation, Kobe, Japan (pp. 552-557). 

Retrieved September 23, 2005, from http://ai-www.naist.jp/papers/setalav/EX/CIRA2003/Paper/cira2003-setalav. pdf, 2003. 
25 

Skubic, M.; Bailey, C.; Chronis, G. A Sketch Interface for Mobile Robots. Proceedings of the 

2003 IEEE International Conference on Systems, Man, and Cybernetics, (pp. 919-924), 

2003. 

Stafford, S.C.; Jingjing, W.; Merlo, J.; Hancock, P. A. Soldier Opinions of Future Military 

Display Technologies. Proceedings of Human Performance in Extreme Environments, 

Baltimore, Maryland, 2007. 

Stafford, S.C.; Hancock, P. A.; Graham, J.; Merlo, J. Equipment to Meet the Cognitive and 

Physical Requirements of the Soldier. Proceedings of Human Performance in Extreme 

Environments, Baltimore, Maryland, 2007. 

Sterling, B.S.; Perala, C.H. Workload, Stress, and Situation Awareness of Soldiers Who Are 

Controlling Unmanned Vehicles in Future Urban Operations; ARL-TR-4071; U. S. Army 

Research Laboratory: Aberdeen Proving Ground, MD, 2007. 

Veronka, N.; Nestor, T. Integrated Head-Mounted Display Interface for Hands-Free Control; 

SBIR report No. ADB264384; 2001. 
26 

INTENTIONALLY LEFT BLANK
27 

Appendix A. Soldier Questionnaire Results 

DEMOGRAPHICS 

SAMPLE SIZE = 15 

 RANK MOS DUTY POSITION AGE

 

E-3 - 1 11B - 1 68W - 2 Rifleman – 1 28 years 

(mean) 

E-4 - 3 13F - 1 92A - 1 OCS – 4 

E-5 - 6 15H - 1 98G - 1 Instructor Pilot – 1 

E-6 - 1 25S - 1 OCS - 2 Medical PSG – 1 

E-7 - 3 25W - 1 NR - 3 S-1 Clerk – 1 

CW3 - 1 42-F - 1 S-3 Clerk – 1 

 S-4 – 1 

 NR – 5 

 

1. How long have you served in the military? 69 months (mean) 

2. How long have you had an infantry-related job? 18 months (mean) 

3. How long have you been a fire team leader? 2 months (mean) 

4. How long have you been a squad leader? 12 months (mean) 

5. How long have you been deployed overseas? 15 months (mean) 

6. How long have you been deployed in a combat area? 7 months (mean) 

7. Corrected Visual Acuity: 14 20/20 both eyes 0 20/20 in one eye 1 other: 20/60

8. Please list any visual problems which you may have. None (other than corrective lenses). 

9. Height: 69 inches (range is 60-76) 

10. Weight: 169 pounds (range is 143-210) 

11. With which hand do you most often write? 13 Right 2 Left 

12. With which hand do you most often fire a weapon ? 15 Right 0 Left 

13. a. Do you wear prescription lenses? 9 No 6 Yes 

 b. If so, which? 4 Glasses 2 Contact 

Lenses 

14. Do you wear glasses or contacts while firing a 

weapon? 

 9 No 6 Yes 

15. Which is your dominant eye? 11 Right 4 Left 

16. Have you ever driven a robotic vehicle? If yes, 

describe. 

 15 No 0 Yes 

17. Have you ever driven a remote control car? 4 No 9 Yes 2 NR 
28 

18. Using the scale below, please rate your level of experience with the following computer 

software and computer related activities. 

1 2 3 4 5 6 7 

No 

experience 

Below 

average 

Slightly 

below 

average 

Average Slightly above 

average 

Above 

average 

Expert 

MEAN RESPONSE 

a. Microsoft Windows 98, 2000, XP, etc. 4.80 

b. Computer based games 4.27 

c. Army digital systems (e.g. FBCB2) 2.53 

d. I would self rate my computer skills as: 4.73 

19. Using the scale below, please rate the following Self-rating of Knowledge, Skills, and 

Abilities (KSAs) related to Infantry duties. 

1 2 3 4 5 6 7 

No 

experience 

Below 

average 

Slightly 

below 

average 

Average Slightly above 

average 

Above 

average 

Expert 

MEAN RESPONSE 

a. Knowledge of tactics, techniques, and procedures (TTP). 3.87 

b. Knowledge of map reading and orientation in field setting. 4.53 

c. Knowledge of reconnaissance, surveillance, and target 

acquisition procedures. 

2.87 

d. Knowledge relating to communications equipment and 

communications procedures. 

3.73 

e. Communication skills (ability to use communications equipment 

and face-to-face communications to enhance mission 

accomplishment). 

4.27 

20. How often do you play video games? 

 3 Never 5 1-11 times per year 3 1-3 times per month 2 1-6 times per week 2 

Daily 

21. List the games you play regularly from 1 (most frequent) to 5 (least frequent). 

1. Counter strike, John Madden, Halo (2), Medieval Total War, Rise of Nations, Tetris 

2. Day of Defeat, NCAA Football, Project Gotham Racing 3, WarCraft, Halo, 

Command/Conquer2, Gran Trurismo 

3. Halo, NCAA Basketball, Sudoku, Age of Empires II, Solitaire, Command/Conquer Generals, 

Grand Theft Auto 

4. MLB Baseball, Brain Age, Morrowind, Dominoes, Oblivion 

5. NHL Hockey, Everquest, Chess, Lego Starwares 2 
29 

POST ITERATION 

SAMPLE SIZE = 15 

1. Using the scale below, please rate the training that you received on the controller in the 

following areas: 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN RESPONSE 

TRAINING TOPICS A B C 

Explanation of how to drive the robot 6.53 6.47 6.20 

Time provided to practice driving the robot 6.47 6.40 5.93 

Explanation of how to operate the controller 6.67 6.67 6.13 

Time provided for explanation of the controller 6.67 6.53 6.07 

Difficulty of learning the controller 6.40 5.67 4.40 

Explanation of how to complete each station of the course 6.87 6.53 6.67 

Evaluation of the practice lane 6.67 6.47 6.33 

How well did you expect to perform on the actual course after 

your training was complete? 

5.33 5.27 5.20 

Overall evaluation of the training course 6.27 6.20 5.20 

2. What were the easiest and hardest training tasks to learn? 

Comments No. of Responses

Easiest: 

A

Overall control. 1 

Learning how to operate. 1 

What each control did. 1 

Climbing the stairs and hill. 1 

Driving. 3 

Steering. 1 

Using the direction controller. 1 

Using the controller. 2 

Easy layout. 1 

Camera control. 1 

Camera movement. 1 

Moving the robot. 1 

Speed control. 1 
30 

Comments No. of Responses

Multi-tasking. 1 

Using the selection screen. 1 

B

Self-explanatory. 1 

Learning the controller. 1 

Buttons easy to remember. 3 

Forward movement. 1 

Movement and maneuver. 1 

Utilizing camera and display. 1 

Flippers. 1 

Controlling head. 1 

Control of robot. 1 

Directional control. 1 

Controls very well organized. 1 

Stationary task (view stationary targets). 1 

Use of control sticks for movement. 1 

C

Learning the controller. 1 

Driving. 2 

Driving over the hill. 1 

Taking corners. 1 

Steering the control. 2 

Automatic drive mode. 1 

Moving the body. 1 

Thumb stick. 2 

Camera. 2 

Learning what each button does. 1 

Nothing was easy. 1 

Hardest: 

A

Following and focusing on moving targets. 1 

Learning how the orientation icon was oriented. Sometimes it was facing me, 

sometimes away. 

1 

Performing each step. 1 

Adapting to the display’s lag. 1 

Movement (turning). 1 

Neck movement. 1 

Flipper and neck control. 1 

The maze. 1 

Tracking eye movement. 1 
31 

Comments No. of Responses

Camera maneuver. 1 

Azimuth. 1 

Using zoom. 1 

Tracking targets. 1 

The scrolling of the camera on moving object. 1 

B

Steering. 1 

Controlling the speed. 2 

Sensitivity. 4 

Compensating for lag time on display. 2 

Camera. 1 

Camera angle. 1 

Control turning. 1 

All the buttons. 1 

No Y axis inversion. 1 

Limited tracking ability. 1 

Tracking a moving person. 1 

The cross of the flipper control and the raising and lowering of arm. The flipper 

control should be on the side of the drive and the arm control should be on the 

side of the camera. 

1 

C

Backing up the robot into a spot where the camera has the most visibility. 1 

Controller hard to move. 1 

Camera modes. 1 

Camera switching. 1 

Controlling the camera. 1 

Camera toggle (select). 2 

Narrow zigzag hallway. 1 

Holding on to the controller. 1 

Toggling between different control functions. 1 

Learning and switching between modes. 2 

Tracking moving targets. 1 

GUI. 1 

3. What are your comments on the training course? 

A

Excellent. 1 

Very simple to understand and learn. 2 

Very well set-up and built course. 3 

Suitable for real life scenarios. 1 
32 

Comments No. of Responses

Adequate difficulty. 1 

Maybe incorporate more use of the camera on moving objects. 1 

Put the robot through good range of maneuvers. 1 

Misc. tasks allow for different aspects of the robot being able to be used and 

learned. 

1 

Could use more obstacles. 1 

The lag between control and actual is very disorienting though with time a person 

can adjust. 

1 

B

Very good. 1 

Lots of fun! 1 

Objectives were clear, instructions kept simple, and tasks useful for real life 

situations. 

1 

Good course; covered most, if not all, areas that robot would be used in. 2 

Consistent and enables you to use all the features. 1 

Good maneuvers for the robot and used all controls. 1 

Testing my ability on basic tasks. 1 

Much easier after one training period. 1 

Needs a little more practice (operator) before testing. 1 

Joystick was too sensitive to movement. 1 

Incorporate more camera movement tasks. 1 

C

Overall good instruction. 1 

Very good course. 4 

Very realistic on everyday combat fields. 1 

Very easy to get a working knowledge of this system. 1 

Good for learning controls. 1 

Used a variety of tasks on the robot. 1 

Evaluated basic tasks; taught very well. 1 

Could be more challenging. 1 

The control was fairly simple, but having it on the weapon was not practical. 1 

Need a little more training time in the foxhole. 1 

The course was more difficult and extremely time consuming. 1 

Controller was slippery and failed to go R and L. 1 

More tracking of moving targets. 1 

4. What did you think about this controller after completing the training course? 

A

Great controller. 1 

I liked how the controller was easy to function and easy to learn and operate. 3 

Practical setup. 1 

Everything is at your fingertips. 1 

Very easy to use. 6 
33 

Comments No. of Responses

Easy to multi-task with. 1 

Sensitive. 1 

Overall, good controller. 1 

Good control. 2 

Well labeled. 1 

Excellent, but I had azimuth issues. 1 

Interface was good and the buttons were under utilized. If some of the buttons 

were used instead of generic arrows/select, it could be more efficient, i.e., zoom 

in/out. 

1 

Adequate. 1 

Big, bulky. 3 

Needs a few improvements. 1 

It works, yet the program seemed too overbearing on the computer. 1 

Control sticks could be more sensitive. 1 

Could not multifunction enough, i.e., flippers and driving. 1 

B

Easy and quick to use. 2 

Easiest to use by far. 1 

Good size. 1 

Easy to reach all controls. 1 

Very simple; almost like common gaming controls. 1 

The controller was familiar, making the learning curve a little easier. 1 

Key placement was intuitive, making learning simple. 2 

I liked the ergonomics of it a lot. Makes more sense than the other two. 1 

It was OK. 1 

I like it, but I would change the drive button. 1 

Better fine control (but not inherent in the controller, more in calibration). 1 

Limited multifunction. 1 

Why waste a whole button on brake? Could be used as a default/home button. 

For example, I want to go to drive mode with drive camera from anywhere. 

1 

Acceleration issues with thumb force controller. 1 

Way too sensitive. 5 

Having the zoom button and the toggle to “track” ,both on the right side, made it 

difficult to do both tasks simultaneously. 

1 

Tactically, wouldn’t have positive control of your weapon. 1 

Too many unlabeled buttons. The number sequence made it more difficult to 

remember what each was meant for. 

1 

Hard for me to control camera (L, R, up, down). 1 

C

Easy to use. 2 

Very easy to learn and move. 1 

By far the best maneuvering control. 1 

Very smooth. 1 
34 

Comments No. of Responses

Practical. 1 

Ergonomic. Don’t change a thing with the controller. 1 

Multifunction button with computer screen makes it simple to navigate. 1 

Button placement is easy to work with. 1 

The one positive is the system being attached to the weapon, but the equipment 

has a greater chance of getting damaged. 

1 

It’s better as just a hand-held device. 1 

Very complicated. 1 

Need some grips on the controller so your hand won’t slip. 1 

I can’t see the thumb stick working well when the operator is sweaty. I kept slipping 

off. 

1 

Takes too much time to switch between modes. If I was in a hurry, it would have 

been hard to execute. 

1 

Camera controller should be inverted. 1 

May be some steering bugs. 1 

Did not like. 1 

Not very good. 1 

Sticky and doesn’t offer sensitive controls. 1 

Speed is not consistent. 1 

This controller is terrible, unreliable, and difficult to maneuver. 1 

5. Using the scale below, please rate your ability to perform each of the following control tasks

based on your experience with the controller that you just used. 

1 2 3 4 5 6 7 

Extremely 

difficult 

Very difficult Difficult Neutral Easy Very easy Extremely 

easy 

MEAN RESPONSE 

CONTROL TASKS A B C 

Ascend stairs 5.60 5.80 5.40 

Descend stairs 5.93 5.80 5.60 

Position robot to look through a window 5.67 5.13 4.33 

Scan for targets 5.40 5.33 4.40 

Track a target 4.60 4.40 4.53 

Zoom in and out with the camera 5.73 5.87 4.53 

Maintain control over uneven terrain 5.93 5.20 5.27 

Position robot to look in a bunker 5.60 5.53 4.20 

Drive through a tunnel 6.33 6.20 5.43 

Drive through a confined space (zigzag) 5.07 4.80 4.73 

Stay within course limits (engineer tape) 5.13 4.53 5.20 

Control robots rate of speed 4.87 3.40 4.40 

Raise robot arm 6.40 6.27 4.86 

Tilt sensor head 6.00 5.73 4.87 

(continued on next page) 
35 

(continued) 

MEAN RESPONSE 

CONTROL TASKS A B C 

Rotate/pan sensor head 6.20 5.93 4.93 

Steer robot while moving forward 4.73 4.13 4.80 

Steer robot while moving in reverse 4.64 4.36 4.43 

Position robot flippers 6.40 6.40 5.00 

Perform multiple tasks simultaneously (i.e. raise arm while 

tilting sensor head) 

5.14 4.71 2.54 

Overall ability to perform driving tasks 5.87 5.13 4.87 

Comments No. of Responses

A

Excellent. 1 

Easy to operate. 2 

Much easier to use. 1 

User friendly. 1 

Great control on speed. 1 

Easy to do the task. I could drive, raise the head, and change cameras all together. 1 

This controller made minute changes possible and easy. 1 

Size is too bulky. 1 

Due to the lag time, there was difficulty adjusting to the drift. 1 

Difficult to make slight turns. 2 

I had some difficulty with overcorrecting on right and left steering. 1 

Never understood how to “tilt” the camera head. The view ended up uneven when 

trying to track a target. Difficulty keeping it level. 

1 

Hardest part was trying to do multiple tasks, i.e., tracking targets while tilting head 

up and down. 

The falling of the flippers and having to raise them over and over again while 

traveling over flat ground detracted from concentration on the movement. 

1 

B

Very responsive to the controls, but lag made it easy to overcompensate. 1 

Difficult to steer while moving forward, otherwise it was good. 1 

Hard to adjust drifting without overcompensating. 1 

Turning right tends to pull backward. 1 

Steering functions were impaired due to the sensitivity of the analog sticks. 1 

Speed control was an issue for me. The controller is very sensitive. 3 

Seemed like it moved like a button instead of a positional switch. 1 

To control the robot with this specific controller was more difficult and took more 

time to maneuver. 

1 

Moving tasks are much more difficult than stationary. With practice, task can 

become easier to perform. 

1 
36 

Comments No. of Responses

One control stick should control the forward movement the other should control left 

and right movement. Also, movement should be controlled by right hand. 

1 

C

Relatively easy to work with limited exposure to device. 1 

Much easier to use for me. 1 

Driving and turning at the same time was very easy. 1 

Sensitivity of thumb stick was good because it helped get rid of over correction. 1 

Not being able to do multiple tasks at the same time. 3 

Not being able to perform multiple tasks, as with zoom in or out while tracking 

made it difficult to pinpoint what an object is. More concentration was needed to 

switch to the modes making the task at hand more time consuming. 

1 

Nearly impossible to perform tasks simultaneously, but it would not take long to 

learn to perform functions quickly. 

1 

Having to toggle between different stations is really frustrating. 2 

The need to switch modes made me unable to zoom out while tracking my target. 1 

I think the flippers and sensor head should switch positions on the controller. 1 

One or two more hours using the controller would’ve made a difference. 1 

The camera toggle sticks or jumps. 1 

6. During this trial, did you experience any of the following? 

Number of Responses 

A B C 

Wrist strain 0 0 1 

Arm strain 0 0 4 

Hand strain 0 0 5 

Finger Strain 0 0 4 

Nausea 0 0 0 

Fatigue 0 0 0 

Disorientation 0 1 0 

Dizziness 0 0 0 

Other? 0 0 0 

C

Fingers slipping. 1 

Got tired from arm and hand strain. 1 

Wrist strain; soreness/CT. 1 

Arm strain holding the weapon. 1 

Arm got tired. 1 

Hand strain having to use fingertips. 2 

Finger strain using toggle with thumb. 2 

Could be physically straining if had to perform for long periods of time. 1 
37 

7. Using the scale below, please rate the following characteristics of the controller that you just 

used: 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN RESPONSE 

CONTROLLER CHARACTERISTICS A B C 

Size of controller 4.00 6.33 5.00 

Size of individual controls 4.93 6.33 5.07 

Location of individual controls 5.67 6.07 4.67 

Comfort while holding the controller 4.64 6.40 4.53 

Weight of controller 4.07 6.40 4.80 

Shape of controller 4.33 6.27 4.87 

Mapping of controller functions 5.60 5.87 3.67 

8. Using the scale below, what is your overall rating of the controller that you used this iteration? 

1 2 3 4 5 6 7 

Extremely bad Very bad Bad Neutral Good Very good Extremely good 

MEAN RESPONSE 

A B C 

5.36 5.67 4.50 

Comments No. of Responses

A

Positive. 1 

Good controller. 1 

Easy to use and learn. 2 

User friendly. 1 

Controllability is good. 1 

Very good control over speed of vehicle. 1 

Operates very smooth. 1 

Very well labeled and layout was ideal. 1 

Good sensitivity on camera adjustments. 1 

A large controller offers more functions and ease of use. In a situation where size 

didn’t matter, this would be a great controller. 

1 

If I had to use as hand-held, instead of placing on a table, I’m sure it would be 

extremely difficult to maneuver. 

1 

Negatives outweighs the positives. 1 

Make it more ergonomic to allow for ease of operation while holding. 1 

Should weigh more. 1 
38 

Comments No. of Responses

A bit large. 1 

Needs to be a little more sensitive with the driving controls. 1 

B

The size, feel, and location of buttons is awesome. 1 

Easy to learn. 1 

Very light; fits comfortably in my hand. 1 

Shape, weight, familiarity of design was good. 1 

Control buttons ideally placed. 1 

I like it; very ergonomic. 1 

I liked it as it was very responsive. 1 

It would help if the controller could be configured to the operator, i.e., right-handed 

person drives with right toggle. 

1 

Takes getting used to for the time lag. 1 

Not best for tactical environment, but otherwise outstanding. 1 

Complicated. 1 

The mapping of the button functions needs work. 1 

C

Very easy to use. 1 

I can’t think of a better place on an M4 or in a combat environment to have this 

controller than where it is now. 

1 

The thumb stick should have a depression in it for your thumb. 1 

Controller needs a textured thumb controller to avoid slipping in a very hot 

environment. 

1 

Less sensitive than B and allows for more precise movements that are easily 

controlled. 

1 

Cycling through a main menu can be complicated. 1 

Requires too much time overall. 1 

Would like to see multiple task options. 1 

Just too hard to multitask. You need to be able to easily transition between modes. 

Maybe eliminate some. 

1 

Difficult and frustrating to keep switching between functions to make it perform. 1 

Don’t like one-hand control. 1 
39 

END OF EXPERIMENT 

SAMPLE SIZE = 15 

1. Using the scale below, please rank order the controllers in the order of your preference on the 

following characteristics by placing a “1” next to your favorite, a “2” next to your second choice, etc. 

1 2 3 

First choice Second choice Last choice 

MEAN RESPONSE 

CHARACTERISTICS A B C 

a. Comfort in the hand 2.33 1.33 2.33 

b. Ease of use 1.60 1.73 2.67 

c. Quickest to use 1.53 1.67 2.80 

d. Ease of carrying with you on a mission 2.80 1.73 1.47 

e. Best to take with you in battle 2.47 1.73 1.80 

Comments No. of Responses

Comfort in the hand

A

Most rugged. 1 

Better controllability. 2 

Still solid with both hands working. 1 

A familiar shape, but awkward. 1 

Too large; bulky. 3 

If smaller, it would be better. 1 

Very difficult to hold. 1 

B

Fit well in my hands. 2 

Hand-held controller. 1 

Very comfortable. 1 

Lightest. 1 

Had better ergonomics and access to buttons. 2 

Very natural. 1 

Most practical. Easily used while holding. 1 

Easier to function. 1 

Mapped ideally and clearly labeled. 1 

Allows multi-functions using thumbs and fingers. 1 

It feels like a game controller. 1 

Too sensitive. 2 

Hard to control speed. 1 
40 

Comments No. of Responses

C

It fits in palm of one hand. 2 

Easier for me to control rate of speed and azimuth simultaneously. 1 

Ergonomic and easy to use. 1 

Awkward, but convenient. 1 

Very tricky to change commands. 1 

Very difficult. 1 

Too complex for simple tasks. 1 

Limited to one hand which can experience fatigue. 1 

Cumbersome. 1 

Too much with the weapon. 1 

Ease of use

A

Very basic and large, making it easy to use. 5 

Simple. 2 

Controls are very practical. 1 

Less sensitive so lag wasn’t difficult to adjust. 1 

Targeting is rather easy on all. 1 

Less buttons and all were labeled. 2 

Not as well mapped/labeled as B. 1 

Too bulky. 2 

Joysticks multi-functioning a poor idea. 1 

Almost everything is done with two controls on this one. 1 

B

Game controller. 1 

Easy to use. 3 

Clearly marked; no confusion; natural. 1 

Every button could be reached easily. Your hands never had to leave the 

controller. 

1 

Light and ready to use with functions at fingertips. 1 

Small, but more buttons. 1 

Little to no switching between modes. 1 

A close second. 1 

Targeting is rather easy on all. 1 

Very difficult to use. 1 

C

Easier for me to control rate of speed and azimuth simultaneously. 1 

Targeting is rather easy on all. 1 

4 buttons. 1 

Didn’t function right. 1 

Very difficult to use. 1 

Display system much more complicated than others; time consuming. 1 

Not as fluid. 1 
41 

Comments No. of Responses

Difficult to find the control you need. 1 

A lot of hit and miss to change modes. 1 

Takes a few minutes to get used to. 1 

Quickest to use

A

Every use written on keypad/controller. 2 

Big, labeled buttons. 2 

Very natural. 1 

No multi-tasking. 1 

I could have easily figured out the controls without any guidance. 1 

All controls could be reached from one position, never requiring much movement. 1 

B

More single function buttons. 2 

Closest to the controllers I’m used to, so it was the easiest to just pick up and use. 1 

Most familiar to me. 1 

Most sensitive. 1 

Limited multi-tasking. 1 

Too many buttons for functions. 1 

Required some movement. 1 

Set up much like a gaming controller. 1 

C

All multi-function buttons. 

Modes are a work around for limited inputs, but having more inputs is essential for 

multi-function. 

1 

Controllers were easy to use. I could toggle through and select tasks. 1 

Very difficult to switch modes. 1 

Had too much mode switching. 1 

Slow because you have to cycle thru everything. 1 

Required almost constant movement to change modes. 1 

Nothing is labeled and takes a little getting used too. 1 

Ease of carrying with you on a mission

This is a size issue. 1 

A

Fit as a natural piece of the weapon. 1 

Smallest/lightest. 1 

Too big, bulky. 3 

B

Lightest. 4 

Easy and small. 5 

Small, but still is extra component. 1 

C

Just attaches to your weapon and you don’t have to think about it. 4 
42 

Comments No. of Responses

Device C if it was not attached to the weapon. 1 

Would be right on the rifle. 1 

Very practical since it may attach to the RAS. 1 

Too large. 2 

Obvious here. 1 

Size of controller matters for packing in ruck. 1 

Will work on an M-4, but at what cost to the warrior? 1 

Best to take with you in battle

This is a size issue. 1 

A

Fit as a natural piece of the weapon. 1 

Smallest, lightest. 1 

Easy to use. 1 

Actually works. 1 

Too big, bulky. 2 

B

Second best since it is so small. 3 

Durable. 1 

Light. 2 

Easiest to use without looking once functions are learned. 1 

Small, but still is extra component. 2 

C

Best since it is on your weapon. 1 

Already on weapon and ready to use. 1 

Attaches to the weapon and you can maintain a hand on the grip of the weapon. 

But since you are so involved in the operation of the robot, you won’t be able to 

effectively use your weapon. 

1 

Convenient, but in a bad place for battery to break off or get damaged. 1 

Too large. 2 

Looks like PAQ-4C and front M-4 handle. 1 

Will work on an M-4, but at what cost to the warrior? 1 

Compacting equipment is always practical. 1 

This would not be something a rifleman should have. 1 

2. Please provide suggestions for improvement in the following areas: 

 a. Suggestions for ways to increase the effectiveness of the controllers. 

A

Smaller design. 3 

Easier to handle. 1 

Use the buttons for multi-function tasks. 1 
43 

Comments No. of Responses

Lets users map their own controls. 1 

No malfunction with joysticks. 1 

Be able to select the sensitivity of the thumb sticks!!! 1 

If it is a two-handed controller, make it where it can be tailored to the person, i.e., 

left or right-handed. 

1 

Decrease lag. 1 

Hand-held separate controller. 1 

Less sensitivity of controllers. 2 

Zoom control on the twist axis of the controllers. 1 

B

Attach to weapon somehow. 1 

Have both thumb sticks on the control movement. 1 

Make the two-handed controller where you can adjust the sensitivity. 1 

Make just a hair more rugged. 1 

Make more user friendly. 1 

Use face buttons for 3 separate cameras and last for select. 1 

Less switching between modes the better. 1 

Reduce sensitivity of steering. 1 

Minimize camera delay. 1 

Remove superfluous buttons. 1 

Wireless. 1 

C

Small joystick on it. 1 

Easier. 1 

Faster to use. 1 

When zoomed in, sensitivity goes down so you don’t over shoot. 1 

Make the one-handed controller able to be multi-functioning. 1 

Make less sensitive to overcome lag. 1 

Input into a HUD. 1 

Create a function to rotate 180 degrees. 1 

Smaller. 1 

A textured grip. 1 

Able to adjust rate of acceleration. 1 

Not really a good idea. 1 

Move brake to another place (or make it automatic). 1 

Use bottom button for drive mode default (a home button). 1 

Remap buttons; awkward to select with pinky and ring finger.. 1 

Add buttons; using display navigation is slow. 1 

Tighten the toggle for less exaggerating of maneuvers. 1 
44 

Comments No. of Responses

 b. Suggestions on ways to improve the training course. 

Course was great. 3 

Did a great job of simulating the needs of the equipment. 1 

Shorter day. 1 

Multi-testing by all testers. 1 

More tasks that require multi-tasking. 1 

More tracking of targets. 1 

Track a moving target over uneven terrain. 1 

Track a target while moving. 1 

Going under obstacles, like cars. 1 

Raise the engineer tape to six inches. 1 

Controls and different maneuvers. 1 

Make course harder. 1 

More obstacles. 2 

Add IED or other training robot. 1 

Reduce training time to 30, then 25, then, 20. 1 

Vary course a little. Can be repetitive. Go thru the other way. 1 

Multiple robots at once. 1 

Use tactical situations. 1 

Increase difficulty. 1 

 c. Suggestions on ways to improve the driving lanes that you negotiated 

No need to change; accomplished. 1 

Desired mission. 1 

Rooms (urban terrain). 1 

Make one longer to see how people adjust for slight drifting in forward movement. 1 

Make driving lane more defined. 1 

Make lanes rougher to negotiate. 3 

Maneuver more turns. 1 

Drive with lights on (dark). 1 

More terrain obstacles (i.e., wet ground, mud, rubble). 3 

Longer straight-aways. 1 

Make some with overhead cover. 1 

 d. Which would you prefer? 

 4 One-handed control operation 

 11 Two-handed control operation 
45 

Comments No. of Responses

One-handed

Allows free use of other hand for tactical environments, et. 2 

Easier. 1 

Needs to be multi-taskable. 1 

Two-handed

Allows for more ease of control and makes for an overall less complicated system. 1 

Keeping maneuvers to the left and specific tasks on the right worked well. 1 

To change modes. 1 

Means I can do more at one time. 1 

Feels more stable. 1 

Requires less mode changes which can complicate/slow down process. 1 

Opens up more options for functions. 1 

 e. Did you have any difficulty operating the controls on the Q3D with your left hand? 

 4 Yes 

 11 No 

No. In fact, I used my left hand, but only because I felt like I had to have my firing 

hand on my weapon. 

1 

Didn’t require dominant-arm-strength/coordination. Controls were simple enough to 

use a non-dominant hand. 

1 

1 

I’m used to firing my weapon left-handed so it made it easier. 3 

Just changing menus was challenging. 1 

Too much switching between modes. 1 

I would have to have a lot of practice with this device. 1 

Got fatigued quickly. 1 

Very uncomfortable. 1 

3. Using the scale below, please rank the following features in terms of importance in a robotic 

hand controller by placing a “1” next to your favorite, a “2” next to your second choice, etc. 

1 2 3 4 5 6 

First choice Second choice Third choice Fourth choice Fifth choice Last choice 

MEAN 

RESPONSE 

Comfort in hand 2.79 

Size 3.07 

Weight 4.21 

Ruggedness 4.57 

Maintenance free 3.57 

Intuitiveness of controls 2.60 
46 

4. Please answer the following questions on the entire controller system (processor, antennae, 

cables, etc.) 

 a. Would you want to carry/wear the controller system? 

 10 Yes 

 4 No 

 1 Don’t know 

Comments No. of Responses

No, Solders carry enough already. 2 

It’s a laptop and controller. It would weigh less than the radios. 1 

Why would we wear it? Keep it in a case with the robot. 1 

Did not experience the weight of the robot itself. If light enough, this piece of 

equipment will serve a great purpose to any Solder. 

1 

Too fragile for combat. 1 

 b. Please provide as many adjectives as you can think of describing only the hardware of the 

controller system (processor, antennae, cables, display, hand controller, etc.) 

Expensive. 1 

Durable. 1 

Easy. 2 

Basic. 1 

Small. 2 

Compact. 1 

Lightweight. 1 

Efficient. 1 

Laggy. 1 

Rugged. 2 

Rough. 1 

Cool. 1 

Agile. 1 

Versatile. 1 

Impressive picture. 1 

Programming bugs. 1 

Heavy. 1 

Need to be condensed. 1 

Too many wires hanging out. 1 

Not too rugged looking. 1 

A bit much with a full combat load with body armor. 1 

Simple, yet unreliable. 2 

 c. If you could name the robot you’ve been controlling, what would you name it? 

Threat Negator. 1 

Stalker. 1 

Bob. 1 

Johnny 5. 4 

Besides Johnny 5? How about Chuck. I’ve never known a Chuck, but this robot 

sure acts like one. 

1 

Roger. 1 
47 

Comments No. of Responses

Urban surveillance electronic robot (USER). 1 

Recon ranger. 1 

The Mole. 1 

Breland-Bot. 1 

Red Max; Red Hot. 1 

6. Using the scale below, please rate the importance of the following driving tasks that are needed 

for teleoperating a robot. 

1 2 3 4 5 6 7 

Extremely 

Unimportant 

Very 

Unimportant Unimportant Neutral Important

Very 

Important 

Extremely 

Important 

DRIVING TASKS MEAN RESPONSE

a. Heading information 5.20 

b. Relative distance to obstacles 5.93 

c. Depth of pot holes 5.67 

d. Information on the navigability of down slopes (tilt meter) 5.53 

e. Information on the navigability of side slopes (tilt meter) 5.40 

f. Identification of other terrain features that might have an adverse 

effect on the ability of the robot to maneuver through the terrain 

(describe) 

5.43 

g. Ability to determine where the sides of the vehicle are located. 

(sides are in the camera field of view or are marked on the screen) 

5.60 

h. Ability to determine where the front of the vehicle is located 5.73 

i. Ability to determine where the back of the vehicle is located 4.93 

j. Information concerning whether the ground clearance of the 

vehicle will k. allow negotiation of rugged terrain 

5.27 

l. Indication of the turn radius of the vehicle 5.00 

m. Information on the terrain closer than 6 inches in front of the 

vehicle 

5.57 

n. Information on the terrain five to 15 feet in front of the vehicle 5.13 

o. Information on the terrain greater than 15 feet in front of the 

vehicle 

4.67 

p. Information on the current speed of the vehicle (speedometer) 4.20 

q. Information on vehicle RPMs 3.27 

r. Information on the way the vehicle motor sounds 4.40 

s. Information on the other noises present in the vehicle’s 

environment 

5.13 

t. Information concerning the color of objects in the environment 

(color vs. black and white camera) 

5.20 

u. Information concerning objects on the side of the vehicle (side 

facing cameras) 

5.00 

v. Information concerning objects behind the vehicle (back facing 

camera) 

5.13 

w. Information concerning the condition of the vehicle tires/tracks. 5.00 

(continued on next page) 
48 

 (cont) 

DRIVING TASKS MEAN RESPONSE

x. Information concerning the temperature of the vehicle 4.60 

y. Feedback on the ruggedness of terrain. 4.67 

z. Information concerning any delay between the time the operator 

sends a vehicle command and the time that the vehicle responds 

(i.e., how long after a stop command is sent before the vehicle 

actually stops) 

5.73 

aa. Feedback concerning whether or not the vehicle correctly 

received a command 

4.87 

bb. Battery status 6.20 

cc. Fuel status 6.07 

dd. Vehicle temperature 5.27 

Comments No. of Responses

f. sand/muck, rocket, clearance under objects. 1 

Most important is lag time info. 1 

I think 360 degree awareness around the robot is important because someone 

could easily sneak up on or sneak by otherwise. Also, many threats could go 

undetected without 360 degree awareness. 

1 

You should know how loud the robot is being if it must be stealthy. 1 

Color is important because it can go to identification. 1 

The lag could pose challenges until it is the user adjusts to it. The B controller 

would be the best option regarding size and mapping of controls. 

1 

A HUD. 1 

7. The purpose of this experiment was to determine the feasibility of reducing the size of the 

robotic controller for use by dismounted Solders. Based upon what you’ve seen, is it feasible to 

reduce the size of the controller without adversely impacting performance by: 

 a. Shrinking the controller by shrinking it’s controls sizes? 

 10 Yes 

 5 No 

Shrink Device A. 1 

Device B was easiest to use and had the smallest learning curve. 1 

Device C is a very good size. 1 

Device C is perfect for combat. 1 

Shrinking controls seems relatively uncomplicated. 1 

Not worth shrinking anymore; all three are fairly small. 1 

The smaller, the more comfortable it was to use. 1 

Smaller controls don’t impact the performance. 1 

The sizes are correct for the size of the individual controls. 1 

If having to carry it, it would be useful if smaller. 1 
49 

 b. Shrinking the size of the controller by reducing the number of controls? 

 5 Yes 

 10 No 

Comments No. of Responses

For the first two. 1 

Will increase time spent performing control tasks. 1 

It is easier to control the robot and can more timely do so with more buttons. 1 

If done right, this could work, but this prototype wasn’t user friendly. 1 

Make more user friendly. 1 

To reduce much would adversely affect performance. 1 

Having to switch between modes to complete a maneuver wasted time. 1 

More controls are needed to make it faster and easier. 1 

Device C swapped user friendliness for less controls – bad! 1 

Having the press the same button over and over was annoying. 1 
50 

INTENTIONALLY LEFT BLANK
51 

Appendix B. Control Intuitiveness Test Results 

Controller A 

Controller A 

1. Raise the arm of the sensor head. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A 2 2 1 5 11 

1B 1 0 1 2 4 

1C 0 0 1 1 1 

1E 0 0 1 1 1 

2 3 2 0 5 13 

3 0 0 1 1 1 

5 1 1 4 6 9 

6 0 2 0 2 4 

11 0 0 1 1 1 

14 1 0 1 2 4 

18A 3 2 0 5 13 

18B 1 2 0 3 7 

18E* 0 1 0 1 2* 

18F 0 0 1 1 1 

2. Raise the flippers. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A 2 1 0 3 8 

1B 0 0 1 1 1 

1C 0 1 0 1 2 
52 

(cont) 

2. Raise the flippers. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1E 1 0 0 1 3 

1F* 1 0 0 1 3* 

2 3 0 2 5 11 

4 0 0 1 1 1 

5 0 2 1 3 5 

10 0 0 1 1 1 

11 1 0 0 1 3 

12 0 0 1 1 1 

13 0 1 0 1 2 

14 0 2 1 3 5 

17 2 1 2 5 10 

18A 0 2 0 2 4 

18B 0 1 1 2 3 

18F 2 0 0 2 8 

3. Drive the robot forward. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A* 9 5 0 14 37* 

1B 0 1 0 1 2 

18A 3 0 1 4 10 

4. Brake. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1B 2 1 0 3 7 

1C 0 0 1 1 1 

1E 0 0 1 1 1 

1G 1 0 0 1 3 

2 0 1 0 1 2 

3 0 0 1 1 1 

4 0 0 1 1 1 

5 1 0 0 1 3 

14* 2 2 2 6 12* 

15 2 1 0 3 8 

16 0 2 1 3 5 

17 3 0 1 2 10 

18B 1 2 1 4 8 

18G 0 1 0 1 2 

5. Pan the camera left. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1C 2 2 1 5 11 

1E 3 0 0 3 9 
53 

(cont) 

5. Pan the camera left. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

6 0 1 1 2 3 

10 1 1 0 2 5 

11 0 0 1 1 1 

14 1 0 0 1 3 

18C* 4 2 1 7 17* 

18E 1 2 1 4 8 

*Correct choice based on current mapping. 

Controller A

1. Raise the Arm 

 

First Choice 

Response Reason selected No. Responses

 

1A Direction; on left 1 

 Guess 1 

 

1B Intuitive; flight analogy 1 

2 Important function; should have easy access 1 

 Intuitive 1 

 

5 Guess 1 

 

14 Button is easy to reach and prefer to use joystick 1 

 

18A Assume left toggle for driving; camera second 

most important 1 

 Main function, so should be main control 1 

 Video game analogy 1 

 

18B Seems most logical; left stick for driving, right for 

camera 1 

Second Choice 

Response 

 

1A Directional motion 1 

 Easy to operate 1 

 
54 

No. of Responses

2 Each row could represent a different part of the 

robot 1 

 Guess 1 

 

5 Guess 1 

 

6 Process of elimination 1 

 Would want close to left hand 1 

 

18A Guess 1 

 Joystick seems probable 1 

 

18B Guess 1 

 Second most logical 1 

 

18E Process of elimination; joysticks appear for track so 

must be button 1 

Third Choice 

Response 

 

1A Most sensitive control 1 

 

1B Joystick natural for controlling direction 1 

 

1C Left or right motion makes sense for moving head 1 

 

1E Guess 1 

 

3 Top of the key pad equals up 1 

 

5 Closest button if driving with right hand 1 

 Easy access 1 

 Guess 1 

 

11 Could be arrow pad 1 

 

14 Guess 1 

 

18F Simple movement 1 

 

2. Raise the Flipper 

First Choice 

Response 

 

1A Move quickly 1 

 Up and down motion 1 
55 

No. of Responses

 

1E Two directional analog 1 

 

1F Turn for up or down 1 

 

2 Each row for different part of robot 1 

 Guess 1 

 Top of control 1 

 

11 Buttons may be arrow pad 1 

 

17 Easy to reach 1 

 Flippers located opposite of neck on robot so 

opposite on keyboard 1 

 

18F Guess 1 

 Joystick for movement 1 

 

Second Choice 

Response

 

1A Assigns direction 1 

 

1C Same as 1; hands already on stick for driving 1 

 

5 Guess 1 

 

13 Process of elimination 1 

 

14 Easy to reach with non driving hand 1 

 

17 Guess 1 

 

18B Direction easy to control 1 

 

Third Choice 

Response 

 

1B Guess 1 

 

2 Guess 2 

 

4 Process of elimination 1 

 
56 

No. of Responses

5 Assume would be upper button 1 

 

10 Same; process of elimination 1 

 

12 Process of elimination 1 

 

14 Easy to reach with thumb 1 

 

17 Easy to reach with driving hand 1 

 Guess 

 

18B Could use it while driving 1 

 

3. Drive the robot forward. 

First Choice 

Response 

 

1A Directional 1 

 Feels natural 1 

 Makes sense; easy to use 2 

 Left hand dominant 1 

 Most sensitive to direction; assume left 1 

 Usually left controls body 1 

 Guess 1 

 Two track should have two movements 1 

18A Drive with right hand 1 

 Steering easier with right hand 2 

 

Second Choice 

Response

 

1A Guess 2 

 Next best choice 1 

 Process of elimination 1 

 Second most logical 1 

 

1B Same as 1 but opposite direction 1 

 

Third Choice 

Response 

 

18A If not. two controls would drive with right hand 1 

 
57 

4. Brake.

 

First Choice 

Response 

 

No. of Responses

1B Could easily stop 1 

 Opposite of driving forward 1 

 

1G Brake analogy 1 

 

5 Won’t use often so furthest away 1 

 

14 Easy to reach quickly 1 

 Makes sense 1 

 

15 Guess 2 

 

17 Brake should be near drive control 1 

 If driving with right hand easiest to reach 1 

 Intuitive to use thumb for brake 1 

 

18B Opposite of driving forward 1 

 

Second Choice 

Response 

 

 

1B Opposite direction of motion 1 

 

2 Same as first 1 

 

14 Easiest to reach with other hand 1 

 Same as 1 1 

 

15 Close to drive; lower end of keypad 1 

 

16 Guess 2 

 

18B Same as 1 2 

 

18G Same as 1 1 

 
58 

Third Choice 

Response 

 

No. of Responses

1C Wouldn’t want to push button 1 

 

1E Twisting motion like pan motion 1 

 

3 Guess 1 

 

4 Guess 1 

 

14 Easily reachable from drive control 1 

 Same as 1 1 

 

16 Process of elimination 1 

 

17 Peripheral 1 

 

18B Pull back to stop 1 

 

5. Pan the camera left.

First Choice 

Response

 

1C If driving with right hand would want camera on the 

left 1 

 Left side 1 

 

1E Guess 1 

 Knob has left and right direction 1 

 Turn left with left hand 1 

 

10 Hadn’t used row yet 1 

 

14 Left side of keyboard; lower priority function 1 

 

18C Guess 1 

 If related to moving the neck should also move the 

camera 1 

 Intuitive 1 

 Sensitive control 1 

 

18E Same as 1 1 

 
59 

Second Choice 

Response 

No. of Responses

 

1C First person shooter game analogy 1 

 Left side 1 

 

6 Second most intuitive 1 

 

10 Process of elimination 1 

 

18C Feels most natural 1 

 Next best choice 1 

 

18E Guess 1 

 Same as 1 1 

 

Third Choice 

Response 

 

1C Guess 1 

6 

 Guess 1 

 

11 Process of elimination 1 

 

18C Easier 1 

 

18E If not for driving makes sense for camera control 1 
60 

Controller B 

Controller B 

1. Raise the arm of the sensor head. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1 0 1 0 1 2 

2* 2 2 2 6 12* 

3 3 1 3 7 14 

5 0 1 0 1 2 

9A 1 1 1 3 6 

9B 1 0 1 2 4 

10A 4 2 0 6 16 

10B 1 0 1 2 4 

16 0 2 2 4 6 

17 2 1 1 4 9 

18 1 1 0 2 5 
61 

2. Raise the flippers. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1 2 0 0 2 6 

2 3 0 1 4 10 

3 1 2 1 4 8 

7 0 0 1 1 1 

9A 1 1 1 3 6 

9D 0 0 1 1 1 

10A 1 2 0 3 7 

12 0 1 1 2 3 

16 2 1 1 4 9 

17* 5 3 1 9 22* 

18 0 1 0 1 2 

3. Drive the robot forward. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

2 0 0 1 1 1 

3 4 2 1 7 17 

9A* 8 5 2 15 36* 

9B 0 1 0 1 2 

10A 2 1 1 4 9 

16 0 1 0 1 2 

4. Brake. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1 0 1 0 1 2 

1B 1 0 0 1 3 

2 1 0 1 2 4 

5 1 0 0 1 3 

7* 1 0 2 3 5* 

8 0 2 1 3 5 

9B 1 1 0 2 5 

9E 0 0 1 1 1 

10E 1 1 0 2 5 

11 3 0 0 3 9 

12 0 1 1 2 3 

13 2 1 1 4 9 

14 3 3 3 9 18 

15 1 1 0 2 5 

17 0 2 1 3 5 

18 0 1 3 4 5 
62 

5. Pan the camera to the left. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1 1 0 0 1 3 

2 0 1 3 4 5 

3 0 1 0 1 2 

4 4 2 2 8 18 

9C 4 3 1 8 19 

10C* 4 3 3 10 21* 

10D 1 0 0 1 3 

*Current correct choice. 

Controller B

1. Raise the Arm 

 

First Choice 

Response Reason selected No. Responses

2 Guess 1 

 Makes sense 1 

 

3 Axis of movement appears similar to robot 1 

 Guess 1 

 Other direction controls assumed to be for other 

tasks 1 

 

9A Directional control 1 

 

9B Prefer to drive with right and use left for peripheral 1 

 

10A Assume left toggle for driving; camera second 

most important 1 

 Easy to reach and move 1 

 Pivot control; right handed 1 

 Right handed 1 

 

10B Seemed related to arm motion 1 

 

17 Digital analog of up and down 1 

 Makes sense 1 

 

18 Easy to reach 1 

 
63 

Second Choice 

Response 

No. of Responses

1 Same as 1 1 

 

2 Can press quickly 1 

 Same as 1 1 

 

3 Arrow up makes sense to move up; video game 

analogy 1 

 

5 Next closest to left hand; pull up analogy 1 

 

9A Direction 1 

 Same sort of control as first choice 1 

 

10A More comfortable; directional 1 

 Try different direction 1 

 

16 13-16 directional mapping 1 

 Right side preference 1 

 

17 Right handed 1 

 

18 Opposite orientation of first choice 1 

 

Third Choice 

Response 

 

2 Feels like up 1 

 Same as 1 1 

 

3 Arrow signifies up 1 

 Direction arrow 2 

 

9A Same as 2 1 

 

9B Seems to be toggle 1 

 

10B Only logical choice left 1 

 

16 Four button could represent axis of movement 1 

 Top equals up 1 

 

17 Same as 1 1 
64 

2. Raise the Flipper 

First Choice Response 

No. of Responses

 

1 Head top feet bottom 1 

 Convenient for up and down 1 

 

2 Looks like physical layout of the robot 1 

 Seems logical for up 1 

 Flippers independent 1 

 

9A One control for each flipper 1 

 

10A Right handed 1 

 

16 Neck control and flipper control should be parallel 1 

 Up direction 1 

 

17 Guess 2 

 Lower priority task; out of the way; don’t need high 

sensitivity 1 

 Makes sense 1 

 Seem to relate to up and down 1 

 

Second Choice 

Response 

 

3 Direction 1 

 Same as 1 1 

 

9A Direction makes sense 1 

 

10A Drive with right hand 1 

 Process of elimination 1 

 

12 Guess 1 

 

16 Up direction 1 

 

17 If left buttons for neck then right for flippers 1 

 Same as 1 2 

 

18 Same as 1 1 

 
65 

Third Choice 

Response 

No. of Responses

2 Top and bottom correspond to up and down; one 

set for each flipper 1 

 

3 Up arrow 1 

 

7 Guess 1 

 

9A Next logical choice 1 

 

9D Two flippers should have two controls 1 

 Looks like it would be up and the one below it down 1 

 Makes sense 1 

 Top button; quick raise 1 

 

3. Drive the robot forward. 

 

First Choice 

Response 

 

3 Direction arrow 1 

 Direction pad typical for forward movement; video 

game analogy 1 

 Forward arrow 1 

 Video game analogy 1 

 

9A Easier to maneuver 1 

 Easy to use with thumb; main function 1 

 Easy to use; video game function 1 

 Has swivel for direction; more freedom of 

movement 1 

 Most sensitive to direction 1 

 Control speed and direction 1 

 Direction; on for each track 1 

 Tank analogy 1 

 

10A Drive with right hand 1 

 Driving most important task; right handed; sensitive 

to pressure, gas pressure analogy 1 

 Right handed; analog stick 1 

 
66 

Second Choice 

Response 

No. of Responses

 

3 Forward arrow 1 

 Only other logical choice 1 

 

9A Next logical choice 1 

 Process of elimination; left hand dominant 1 

 Same as 1 1 

 Video game analogy 1 

 One control for each track 1 

 

9B Same as 1 but opposite direction 1 

 

10A Right handed; seems logical 1 

 

16 Right handed 1 

 

Third Choice 

Response 

 

2 Process of elimination 1 

 

3 Directional pad 1 

 

9A Easy to use analog stick 1 

 Process of elimination 1 

 

10A Guess 1 

 

4. Brake.

 

First Choice 

Response

 

1B Can slow while driving 1 

 

2 Close to forward button 1 

 

5 Pull back brake analogy 1 

 

7 Process of elimination 1 

 

9B Opposite of drive forward 1 

 
67 

No. of Responses

10E Driving control; brake analogy 1 

 

11 Closest to driving control 1 

 Low priority function; don’t want to accidentally 

press; video game pause button 

1 

 Process of elimination 1 

 

13 Analogy to video game 1 

 Video game analogy 1 

 

14 Closest to thumb; wouldn’t be directional 1 

 Should be a button type control; close to thumb is 

convenient 

2 

 

15 Should be a button type control 1 

 

Second Choice 

Response 

 

1 Hands off all other buttons 1 

 

8 Easy to reach; doesn’t seem like priority 1 

 Same as 1 1 

 

9B Backward motion to stop 1 

 

10E Intuitive 1 

 

12 Easy to reach with thumb 1 

 

13 Same as 1 1 

 

14 Drive with left and brake with right 1 

 Easy to reach with thumb 1 

 Feels natural to press for brake 1 

 

15 Next to first choice 1 

 

17 Out of the way of other controls 1 

 Same as 1; proximity to index finger makes in 

convenient 

1 

 

18 Guess 1 

 
68 

Third Choice 

Response 

No. of Responses

 

2 Easy to press quickly 1 

 

7 Not easily confused with other controls 1 

 Same as 2 1 

 

8 Guess 1 

 

9E Makes sense 1 

 

12 Same as 1 1 

 

13 Guess 1 

 

14 Down position 1 

 Process of elimination 1 

 Seems unrelated to other functions so off to the 

side 

1 

 

17 Guess 1 

 

18 Easy access 1 

 Easy to reach with fingers 1 

 Same as 1; process of elimination 1 

 

5. Pan the camera left.

 

First Choice 

Response

 

1 Guess 1 

 

4 Directional pad 2 

Easy to change 1 

Makes sense 1 

 

9C Easy to move with toggle control 1 

Joystick should move camera left and right and the 

neck up and down 

1 

Non dominant hand; high priority function; speed 

and direction 

1 

Right to drive; left for camera; video game analogy 1 

 
69 

No. of Responses

10C Affordance to turn 360 degrees; right handed 1 

If left toggle is drive the right should be for the 

camera 

1 

Logical choice; right control usually for head 1 

Sensitive control 1 

Point in direction of motion 1 

 

13 Buttons 13-16 create up-down, left -right layout 1 

 

Second Choice 

Response 

 

2 Left control; top button can be pressed quickly 1 

 

3 Left arrow 1 

 

4 Directional arrows 1 

Directional pad 1 

 

9C Left side and left direction 1 

Makes sense 1 

Next to first choice 1 

 

10C Directional; drive with left hand, control camera with 

right 

1 

If not for driving would make sense 1 

Prefer to use left hand for view 1 

 

13 Wouldn’t use toggle 1 

 

Third Choice 

Response

 

2 Could use simultaneously with other functions 1 

Easy access; on left side 1 

Guess 1 

 

4 Direction arrow 1 

Only directional control left 1 

 

9C Guess 1 

 

10C Process of elimination 1 

Same as 1 1 

Toggle direction 1 
70 

Controller C 

Controller C 

1. Raise the arm of the sensor head. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A 8 4 2 14 34 

1B 0 2 0 2 4 

1D* 0 0 1 1 2* 

2 4 5 0 9 22 

3 1 1 5 7 10 

4 0 0 3 3 3 

5 0 0 1 1 1 

6A 0 1 0 1 2 

2. Raise the flippers. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A* 4 3 3 10 21* 

2 3 3 0 6 15 
71 

(cont) 

2. Raise the flippers. 

3 1 2 5 8 12 

4 3 1 2 6 13 

5 1 1 0 2 5 

6A 1 0 0 1 3 

6B 0 1 0 1 2 

3. Drive the robot forward. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1A* 13 0 0 13 39* 

4. Brake. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1B 1 0 1 2 4 

1E 4 1 1 6 15 

2 2 4 0 6 14 

3 0 0 3 3 3 

4 1 0 0 1 3 

5* 4 3 0 7 18* 

6A 0 0 1 1 1 

6B 1 0 0 1 3 

7 0 1 0 1 2 

5. Pan the camera to the left. 

Control First Choice Second 

Choice 

Third 

Choice 

Total Weighted 

Totals 

1C* 10 5 3 18 43* 

2 0 1 0 1 2 

4 1 0 0 1 3 

5 0 0 1 1 1 

6B 1 0 0 1 3 

*Current correct choice. 

Controller C

1. Raise the Arm 

 

First Choice 

Response Reason selected No. Responses

 

1A Convenient 1 

 Directional pad; multipurpose 2 

 If in correct mode obvious direction 1 

 Only control with direction 1 

 Makes sense 1 
72 

No. of Responses

 Since only one direction control choose a button to 

specify mode 1 

 Buttons for different modes 1 

2 Easiest to reach 1 

 Primary function; top equals up 2 

 Starting from the top and work down 1 

3 Meant to hit top button; up goes with top button 1 

 

Second Choice 

Response 

 

1A Directional motion 1 

Logical 1 

Place in mode 1 

Guess 1 

 

1B Convenient 1 

Only directional control 1 

 

2 If not toggle, should be top button to equal up 1 

Main function so first button 1 

Top button means raise 1 

Toward top; more priority 1 

Up goes with higher located button 1 

 

3 Process of elimination 1 

 

6A Standard position of neck is down so turn grip to 

the right to raise 1 

 

Third Choice 

Response 

 

1A Process of elimination 1 

Guess 1 

 

1D Right and left could mean up and down 1 

 

3 Assume the first two buttons correspond to head 1 

Guess 2 

Next in line; primary function 1 

Process of elimination 1 

 
73 

No. of Responses

4 Buttons are two pairs of up and down 1 

Process of elimination 2 

 

5 Guess 1 

 

2. Raise the Flipper 

 

First Choice 

Response

 

1A Direction 1 

Directional control 1 

Logical 1 

Flippers lower priority 1 

 

2 Lower relative to other buttons 1 

Secondary priority so not the toggle 1 

Top button equals up 1 

 

3 Guess 1 

 

4 Feels right 1 

Flippers lower priority 1 

Guess 1 

 

5 Flippers lower priority 1 

 

6A Think toggle is for driving, so process of elimination 1 

 

Second Choice 

Response

 

1A Direction 1 

Guess 2 

 

2 Guess 1 

Convenient 1 

Top equals up 1 

 

3 Guess 1 

Middle location on robot 1 

 

4 Same as 1; process of elimination 1 

 

5 Order of priority; process of elimination 1 

 
74 

No. of Responses

6B Same as 1 1 

 

Third Choice 

Response 

 

1A If not for driving; directional control 1 

Guess 1 

Mode 1 

 

3 Guess 4 

Same as 1 1 

 

4 Guess 2 

 

3. Drive the robot forward. 

 

First Choice 

Response 

 

1A Directional control 5 

Driving most important task 1 

Feels natural 1 

Flexible to control movement 1 

Main function 1 

Most sensitive to direction 1 

Movement is natural for thumb; up equals forward 1 

Only one that makes sense 1 

Range of motion 1 

 

3.2 None. 

 

3.3 None. 

4. Brake.

 

First Choice 

Response

 

1B More control to move with thumb 1 

 

1E Brake analogy 1 

Easy if driving 1 

Guess 1 

Makes sense; standard in other controls 1 

 

2 Guess 1 

Closest to driving control 1 

 
75 

No. of Responses

4 Most convenient 1 

 

5 Furthest away prevents confusion 1 

Least frequent use so furthest away 1 

Low priority; out of the way 1 

Out of the way 1 

 

6B Feels like halt, pull back 1 

 

Second Choice 

Response 

 

1E Brake in car analogy 1 

 

2 Guess 1 

Close to drive 1 

Easy access 1 

Makes sense to press a button for brake 1 

 

5 Guess 2 

Very end; low priority 1 

 

7 If stop pressing controls should stop robot 1 

 

Third Choice 

Response 

 

1B Guess 1 

 

1E Guess 1 

 

3 Guess 1 

Process of elimination 2 

 

6A Easy to reach 1 

5. Pan the camera left.

 

First Choice 

Response

 

1C Directional 4 

Left direction 1 

More flexible than buttons 1 

Toggle is for direction 1 
76 

No. of Responses

Guess 1 

Next main function 1 

Feels comfortable 1 

 

4 Third button should be left and fourth right for 

camera 1 

 

6B Left handed 1 

Logical 1 

 

Second Choice 

Response 

 

1C Makes sense to use toggle to control view direction 1 

More access 1 

Easy 1 

Guess 2 

 

2 Same logic as one for first and second button 1 

 

Third Choice 

Response 

 

1C Guess 2 

Next in line 1 

 

5 Guess 1 
77 

NO. OF 

COPIES ORGANIZATION

 1 DEFENSE TECHNICAL 

 (PDF INFORMATION CTR 

 ONLY) DTIC OCA 

 8725 JOHN J KINGMAN RD 

 STE 0944 

 FORT BELVOIR VA 22060-6218 

 1 US ARMY RSRCH DEV & ENGRG CMD 

 SYSTEMS OF SYSTEMS INTEGRATION 

 AMSRD SS T 

 6000 6TH ST STE 100 

 FORT BELVOIR VA 22060-5608 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 IMNE ALC IMS 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK TL 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CS OK T 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR ML J MARTIN 

 MYER CENTER RM 2D311 

 FT MONMOUTH NJ 07703-5601 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MZ A DAVISON 

 320 MANSCEN LOOP STE 115 

 FT LEONARD WOOD MO 65473 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MD T COOK 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 COMMANDANT USAADASCH 

 ATTN ATSA CD 

 ATTN AMSRD ARL HR ME DR HAWLEY 

 5800 CARTER RD 

 FT BLISS TX 79916-3802 

NO. OF 

COPIES ORGANIZATION

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MM DR V J RICE 

 BLDG 4011 RM 217 

 1750 GREELEY RD 

 FT SAM HOUSTON TX 78234-5002 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MG R SPINE 

 BUILDING 333 

 PICATINNY ARSENAL NJ 07806-5000 

 1 ARL HRED ARMC FLD ELMT 

 ATTN AMSRD ARL HR MH C BURNS 

 BLDG 1467B ROOM 336 

 THIRD AVENUE 

 FT KNOX KY 40121 

 1 ARMY RSCH LABORATORY - HRED 

 AWC FIELD ELEMENT 

 ATTN AMSRD ARL HR MJ D DURBIN 

 BLDG 4506 (DCD) RM 107 

 FT RUCKER AL 36362-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MK MR J REINHART 

 10125 KINGMAN RD 

 FT BELVOIR VA 22060-5828 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MV HQ USAOTC 

 S MIDDLEBROOKS 

 91012 STATION AVE ROOM 348 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MY M BARNES 

 2520 HEALY AVE STE 1172 BLDG 51005 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MP D UNGVARSKY 

 POPE HALL BLDG 470 BCBL 

 806 HARRISON DR 

 FT LEAVENWORTH KS 66027-2302 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MJF J HANSBERGER 

 JFCOM JOINT EXPERIMENTATION J9 

 JOINT FUTURES LAB 

 115 LAKEVIEW PARKWAY SUITE B 

 SUFFOLK VA 23435 
78 

NO. OF 

COPIES ORGANIZATION

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MQ M R FLETCHER 

 US ARMY SBCCOM NATICK SOLDIER CTR 

 AMSRD NSC WS E BLDG 3 RM 343 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY-HRED 

 ATTN AMSRD ARL HR MT J CHEN 

 12423 RESEARCH PARKWAY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY-HRED 

 ATTN AMSRD ARL HR MT C KORTENHAUS 

 12350 RESEARCH PARKWAY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MS C MANASCO 

 SIGNAL TOWERS BLDG 29808A RM 303 

 FORT GORDON GA 30905-5233 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MU M SINGAPORE 

 6501 E 11 MILE RD MAIL STOP 284 

 BLDG 200A 2ND FL RM 2104 

 WARREN MI 48397-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MF C HERNANDEZ 

 2421 NW AUSTIN RD STE 220 

 FORT SILL OK 73503-9042 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MW E REDDEN 

 BLDG 4 ROOM 332 

 FT BENNING GA 31905-5400 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MN R SPENCER 

 DCSFDI HF 

 HQ USASOC BLDG E2929 

 FORT BRAGG NC 28310-5000 

 1 ARMY G1 

 ATTN DAPE MR B KNAPP 

 300 ARMY PENTAGON ROOM 2C489 

 WASHINGTON DC 20310-0300 

 1 US ARMY RDECOM TARDEC 

 ATTN AMSRD TAR R MS 264 TIERNEY 

 WARREN MI 48397-5000 

NO. OF 

COPIES ORGANIZATION

 1 SOLDIER BATTLE LAB 

 VIRTUAL/CONSTRUCTIVE SIM DIV 

 ATTN J WILLIAMS 

 6651 WAY STREET 

 FT BENNING GA 31905 

 1 UNIV OF CENTRAL FLORIDA 

 ATTN PHYC DEPT P HANCOCK 

 4000 CENTRAL FL BLVD 

 ORLANDO FL 32816 

 3 SHAWN STAFFORD 

 2584 CREEKVIEW CIRCLE 

 OVIEDO FL 32765 

 1 JAMES MERLO 

 1047 KELLY CREEK CIRCLE 

 OVIEDO FL 32765 

 1 iROBOT CORPORATION 

 ATTN MELISSA LEDOUX 

 8 CROSBY DRIVE 

 BEDFORD MA 01730 

 ABERDEEN PROVING GROUND

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL CI OK TECH LIB 

 BLDG 4600 

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL CI OK TP S FOPPIANO 

 BLDG 459 

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL HR MR F PARAGALLO 

 BLDG 459 

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL HR SC K COSENZO 

 BLDG 459