Comparison Methodology for Robotic Operator Control Units

Robert Bialczak Ph.D., Jonathan Nida, Brian Pettitt

Technology Service Associates, Inc.

Michelle Kalphat

US Army Simulation, Training, and Instrumentation Command

ABSTRACT

Operator controls and interfaces for military unmanned systems

are being developed and used in both laboratory and field research.

These controls are considered to be an efficient method of

controlling unmanned systems in tactical settings and scenarios.

Performance comparisons of different controls and interfaces in

differing tactical environments need further study. This paper

proposes a methodology of evaluating two selected controls in a

laboratory experiment.

The Advanced Robotics Simulations STO research being

conducted at the Technology Development Center (TDC) at

STRICOM and The Mounted Maneuver Battlespace Laboratory

(MMBL) in Fort Knox, Kentucky is using two different control

systems. Both are based upon the OneSAF Testbed Baseline

(OTB). The first is a version of the OTB that has been extensively

modified. Currently it is not instrumented and cannot log the

user’s actions to the button press level. Instrumenting the SAF

would be a relatively straightforward exercise. The second, known

as the Operator Control Unit (OCU), has been developed for

Future Combat system Experimentation, is built on top of the

OTB. The OCU has been instrumented to facilitate analysis. The

OCU has the ability to send DIS (Distributed Interactive

Simulation) PDUs (Package Data Units) to a logger.

Once fully instrumented, the two controllers would then be

run in an exercise. The TDC and the MMBL both have

capabilities where 20-30 OCUs and a comparable number of

OTBSAF control stations could be utilized. Using the information

from varying FCS scenarios an exercise will allow us the ability to

analyze the user’s actions and the corresponding results. Some of

the questions to be examined include:

1) Compare survivability of robots controlled from

different stations

2) Compare time to execute commands

3) Compare steps necessary to execute commands

4) When an interesting event happened (e.g.

firefights, robot destroyed, etc.), and what actions

preceded the event

This research would allow the user community to evaluate not

only the effectiveness of the control stations, but in the future

evaluate the effectiveness of the man-robot units and interactions.

KEYWORDS: Robot, Control, Compare, Evaluate

1. MEASURING ROBOTIC CONTROLS

There are many ways to measure human to robot controls in

tactical scenarios. First you must decide the degree of

fidelity for the robotic vehicles and their environment. This

ranges from complete immersion in a real world scenario

(the robot is in the field in an actual scenario e.g. war) to

total simulation (the robot and its environment are modeled

by computers.) The most accurate method would be to

always test in real situations. Unfortunately that is

prohibitive across numerous domains. The complexity of

this is also intensified because many robotic systems are

still in the design phase. For the purposes of this paper and

this experiment we choose to simulate both the robotic

vehicle and its environment with a modified version of

OTBSAF and a under development experimental FCS

OCU. This simulation tool is accurate enough in most

battlefield scenarios. We will strive to make the

measurements as objective as possible. Only certain aspects

of the robotic controls will be measured. One problem that

will not be addressed is the level of operator training.

Obviously, a poorly trained operator will limit the use of the

control unit, so we will assume that all the operators have

been sufficiently trained on their control unit.

2. TEST FACILITY

The STRICOM TDC and the Mounted Maneuver

Battlespace Laboratory in Fort Knox, Kentucky is the

proposed facilities to conduct these experiments. Currently,

both facilities have two ways of controlling simulated

robotic vehicles at their respective sites – the OTBSAF

controller and the FCS Operator Control Unit. Data will be

collected with a logger which will record all Distributed

Interactive Simulation messages including experimental

Package Data Units for button presses from either controller

and any communications between the Operator Control Unit

and robotic platforms.
2.1 OTBSAF Controller

The first is with an extensively modified version of

OTBSAF 1.0. The mechanism for controlling vehicles with

this tool has not been changed from the OTBSAF 1.0

baseline. Feedback from controlled vehicles is through a

two dimensional tactical map which displays friendly and

other known vehicles’ status. This version of OTBSAF has

not been fully instrumented for button presses, but doing so

will be necessary before it can be used for this exercise.

2.2 FCS Operator Control Unit

The second way for controlling robots is with the Operator

Control Unit developed for the Mounted Maneuver Battle

Laboratory. The Operator Control Unit is a further

modified version of OTBSAF 1.0. This tool was designed

to simplify many of the user’s choices during combat, but

still allow flexibility in the system. The Operator Control

Unit was also designed to control all robotic entities in the

Future Combat System. Feedback from controlled vehicles

is through a tactical map as before and with realistic images

from sensors on the robotic platform. There have been

exercises with more than twenty Operator Control Units in

use at the same time. It has been completely instrumented

down to the button press level.

3. EXERCISE QUESTIONS

The experiment to be conducted at the TDC and Mounted

Maneuver Battlespace Laboratory facilities is proposed to

be on up to thirty OCUs and a comparable number of

OTBSAF control stations. The same scenarios will be

performed with only the robotic control station being

different at the start. After the selected scenarios have been

completed the information collected by the data logger can

be analyzed.

3.1 Mission Success

The most basic question to ask is which robotic controller

had the highest success rate for its robots completing their

assigned missions.

This may be a difficult problem to solve because of the

varying mission types and characteristics of the robots.

Scout vehicles may be evaluated on the percentage of

enemy vehicles identified or the stealthy ness of how they

conducted their mission. Direct fire robots may be evaluated

on the amount of enemy vehicles disabled and or destroyed.

The mission success for other types of robots (re-supply,

engineer) can be assessed in comparable manner.

3.2 Platform Survivability

The next evaluation criteria are which robotic controller

allow for the highest survival rate for its robots. The

meaning of “survival” must be discussed. Does survival

mean the robot is 100% functional or that it still has some

use to the controller? A control station that consistently has

more platforms completely functional at the end of the

exercise must be given more credit than one that has more

robots partially disabled (movement impaired, sensors

destroyed, etc.)

3.3 Executing Commands

Several ways will be used to evaluate the tasking of vehicles

from the control stations. Since the control stations will be

completely instrumented, either the time it takes or the

number of steps required to complete the command will be

captured. Another interesting question currently being

posed is what will the number of platforms destroyed be

while the user was trying to do other tasks or had their

attention diverted to another task and was it significantly

different between types of control stations.

4. KNOWLEDGE GAINED

If the appropriate scenarios are conducted and the correct

questions are asked and evaluated, the effectiveness of the

robotic control requirements can be assessed and evaluated.

This method is not limited to the OTBSAF control station

and the Operator Control Unit. Other control stations can

be integrated into this environment. The Demo III Operator

Control Unit was previously used in the Mounted Maneuver

Battlespace Laboratory simulated environment to train

soldiers before controlling actual robots in the field. An

effort to make the messages of the current Operator Control

Unit both JAUGS and 4D-RCS compliant is being

discussed. This would enable integration of another JAUGS

or 4D-RCS compliant controllers with minimal effort. In

the future with a consistent method to evaluate measure and

asses robotic control stations, the encompassing problem of

evaluating the man-robot unit and their interactions can be

addressed.