The functions/structure model also shows MR
databases at the center of the MRS. These databases
are true “national assets” in the sense that they can
easily contain more mission information, from
multiple sources, packaged in a more useable
structure, than is available anywhere else. With
proper packaging of database contents, many users
can be served and many uses can be met with this
information. Using simulators, participating crews
and mission planners can conduct “what if?
exercises, practice difficult maneuvers, determine
weapon system limits (e.g., maximum comfortable
gross weight for a high altitude rotary wing
operation) and conduct individual and collective risk
assessments. Hard copy products, such as enhanced
photographs and maps can be generated in a simple
extension of the database development process. Such
products have been highly valued by planners and
mission participants. Videotapes are an additional
medium for exporting database information. This
medium provides a mission preview capability for
participants and decision makers who do not have the
luxury of physically going to the rehearsal system.
Finally, the training observation center and the
distributed interactive simulation node can connect

decision makers and mission participants to the
rehearsal environment and add a dynamic dimension
to the information available upon which to make
decisions.

Finally, Banathy emphasizes the importance of
feedback as part of the process model. An extensive
and diverse literature strongly indicates that mere
repetition is not likely to produce optimal
performance (see, for example, Ericsson, Krampe,
and Tesch-Romer, 1993). Rather, feedback is needed
to shape practice in ways that encourage creation of
improved strategies for task accomplishment. This
has relevance for both mission preparation, where
measures of mission effectiveness are needed to
provide the feedback needed to improve the plan and
the performance of participants. Similarly, feedback
is needed for the rehearsal system, to improve the
MR environments, MRS elements, and related
processes. Spiker and Nullmeyer (1995) developed
an initial set of effectiveness measures for the MRS.
These measures reflect both the context of the larger
mission preparation process, and input, process, and
output components of MR. Initial input measures
include completeness, timeliness, and relevance to
mission decisions and outcomes. Process measures
include efficiency and timeliness of database
development, and fidelity and relevance of the
resulting database. Measures of rehearsal output

10

include the ability of planners to create better plans,
crews having more accurate mental models of the
operating environment, and decision makers having
better bases for decisions. Finally, MR itself can
serve as a valuable external feedback mechanism for
training. Performance in the rehearsal environment
can provide insights into how well training is
preparing crews for their operational missions, and
can identify gaps and other problem areas. Lessons
learned at the 58th SOW can help other systems
mature as the “book” is still being written on MR in
three dimensional visualization simulation.

REFERENCES.

Banathy, B. H. (1992). A Systems View of Education.
Englewood Cliffs, New Jersey: Educational
Technology Publications.

Ericsson, K. A., Krampe, R. T., and Tesch-Romer, C.
(1993). The role of deliberate practice in the
acquisition of expert performance. Psychological
Review, 100, 363-406.

Nullmeyer, R. T., Bruce, P. D., Conquest, M. T., and
Reed, E. T. (1992). Using the MH-53J weapon
system trainer/mission rehearsal system: initial
assessments and lessons learned. Proceedings of the
14th Interservice-Industry Training Systems and
Education Conference. San Antonio, TX.

Reed, E.T. (1993). U.S. Air Force helicopter
training high fidelity simulation providing advanced
mission training & rehearsal. Proceedings of the
15th Interservice-Industry Training Systems and
Education Conference. Orlando, FL.

Spiker, A., and Nullmeyer, R.T. (1995). Measuring
the effectiveness of mission preparation in the special
operations forces. (AL/HR-TR-1994-0032). Mesa,
AZ: Armstrong Laboratory, Aircrew Training
Research Division.


SIMULATION-BASED MISSION REHEARSAL
AS A HUMAN ACTIVITY SYSTEM

Robert T. Nullmeyer
Aircrew Training Research Division
Armstrong Laboratory

Lt. Col. Edward T. Reed V. Alan Spiker
58th Special Operations Wing Anacapa Sciences, Inc.

ABSTRACT

Observations of the 58th Special Operations Wing (58 SOW) Weapon System Trainer/Mission Rehearsal System
(WST/MRS) during rehearsals have clearly shown that mission rehearsal (MR) effectiveness is influenced by the
activities of both the people who support the MRS and the people who use it. To explain this demonstrated
importance of the people involved in MR, a human activity system mode! of simulation-based rehearsal was
developed. It provides an integrated depiction of the MRS, and specifically addresses: (1) the context for rapid
database development and simulation-based MR (crisis action planning) and how the MRS fits into it; (2) MRS
components, functions and structure; and (3) processes that enhance rehearsal effectiveness. Implications of this
human activity system view are discussed, including: places in the mission preparation process where simulation
can benefit operations; the scope of human activities that are essential for successful simulation-based MR; the
potential value of MR to provide feedback concerning the adequacy of training to support mission requirements;
and the need for new procedures throughout this extended scope of players to accommodate both MRS
requirements and capabilities.

ABOUT THE AUTHORS

Robert Nullmeyer is a research psychologist with the Aircrew Training Research Division of the Armstrong
Laboratory at Mesa, AZ (formerly Williams AFB). He has conducted research on training system design and
evaluation, simulator effectiveness for training and rehearsal, and combat training requirements. He earned a Ph.D.
in Experimental Psychology from the university of New Mexico. He joined the Aircrew Training Division of the
Air Force Human Resources Laboratory in 1981.

Lt. Col. Edward T. Reed is Director, Aircrew Training Systems, 58th Operations Group, Kirtland Air Force Base
NM. He is responsible for the management and operations of both fixed and rotary wing aircrew training,
supporting special operations and air rescue training missions. Previous duties have included: Special Operations
program manager at HQ MAC, Scott AFB, IL and HQ USAF, Washington DC; Chief Combat Tactics, 1 SOW,
Hurlburt Field FL; MH-53H instructor pilot, 29th SOS, Hurlburt Field FL; and HH-53C Aircraft Commander, 41
ARRS, McClellan AFB, CA. Lt. Col. Reed holds a BA in Political Science from the Citadel and a Masters in Public
administration from Golden Gate University.

V. Alan Spiker is a principal scientist at Anacapa Sciences, Inc., Santa Barbara, CA. He is responsible for human
factors research associated with advanced technology and has conducted field studies on mission planning and
command and control systems for all branches of service. He has a Ph.D. in Experimental Psychology from the
University of New Mexico and has been with Anacapa Sciences since 1982.

ACKNOWLEDGMENTS: This model benefited greatly from the help of many individuals. Our friend and
colleague Phil Bruce introduced us to the concept of human activity systems and spent many hours discussing these
ideas with us. Dave Harris (58 SOW/OGU) contributed considerable time and effort to the development of this
model and was instrumental in specifying how the MRS needed to fit into SOF mission preparation. The
Lockheed-Martin contingent at Kirtland AFB, the people who built the MRS and make it work, provided numerous
essential inputs. Jack Kelly, Rex Corbin, Mike Phelps, Mike Scott, and others generously contributed their insights
and knowledge.



and brainstorming, realistic estimates of mission
success can be generated based on the latest and best
available information. As this information is
distilled, participating crews are "transformed" into a
truly mission-ready aircrew by rehearsing the
planned mission in the networked helicopter and C-
130 Aircrew Training Devices (ATDs).

Figure 4 adds MRS technology to the human activity
system. We represent equipment components and
information with dashed boxes/arrows and the human
* components with solid boxes (people) and arrows
(interpersonal communication). Interactions between
people and equipment are denoted by overlapping
their respective boxes. The MRS receives input from
the data providers in many forms, referred to
collectively as "all source data." Source data is
voluminous and includes topography,
meteorological, geodetic, communications, command
and control, threat, and ownship performance data.
The MRS is configured to receive most formats of
these data, including digital and paper charts and

maps, photography, imagery, sketches, diagrams, and
videotape.

These data enter the 58 SOW MRS through either
data analysis systems or electronic mission
planning/intelligence systems. Tactical analysts use
mission planning and intelligence systems to update
the all source data with real- or near-real time threat
location and order of battle data. The data analysis
systems are a constellation of computer
hardware/software that preview, enhance, modify,
and augment the all source data. The Database
Generation System (DBGS) is a vast array of
contractor-operated hardware and software that
creates the geo-specific visual database of the
mission area. Intelligence and tactical analysts
receive amplifying information from the data
providers and utilize the data analysis system to
extract as much information as possible from the all
source data. Meanwhile, contractor-supplied
database developers and terrain modelers begin
constructing the digital database the aircrews will fly
through during MR.

CI People Component

17 TH EqupmentComponent
2 MRS pure

te

re) MRS Outputs
People-Provided Information

— = Equipment Provided Information

Figure 4. Human and Equipment Components of a Simulation-Based Mission Rehearsal System


1Event | tnecohed!
1Occurs. te9-
Orgeniton ]
a tomes See wedea
Thos bor Problem 1 Develop? F TExecution I 1: 1
cINC Assessment) 1 Evaluate ! i] TEsecute I
Report ry coast OPORD |
Supporing i]
Commands “tT Sr!
advise! in een 1 ore’ Ale ] enone nn
cues 1 order 171 Order |{ O
a LCOAS_y | Co ge
r r--1 rea=|
NA Decide tol | Select 1 Decide i
Develop 1 COA! 4
12k tod Deditey

ro

a> Bventor Process

(J- rreduct

Figure 1: Overview of the Crisis Action Planning "Environment"

right of each group. Arrows show a notional
sequencing of these elements, although requirements
often force deviations from this template. Crisis
action planning begins with the occurrence of an
event having possible national security implications.
The theater commander reports the event and
provides his assessment to the National Command
Authorities (NCA) and the Chairman of the Joint
Chiefs of Staff (CICS). The NCA evaluate the
situation, determine whether a crisis exists, consider
military and non-military options, and decide
whether to prepare for a possible military course of
action (COA). The CJCS may issue a Warning Order
to the theater commander-in-chief (CINC). It defines
command relationships, the mission, and resource
requirements. Our focus for now will be on the
places where wing participation is likely. Figure 1
shows three areas of potential participation for
supporting commands (and their subordinate units) --
developing alternative courses of action, execution
planning, and mission execution. All three could
benefit from access to an MRS, as will be described
in subsequent discussions.

Figure 2 provides a more detailed look at one of
these three, execution planning. Square boxes depict

the activities of aircrews who would plan and execute
the mission. The wing establishes a planning cell
upon receipt of a mission_tasking in the form of a
warning, planning, or alert order. general_planning
begins with an initial briefing to the planning cell to
communicate planning to date, constraints,
intelligence data, and situation updates. The
planning cell then gathers additional information
such as charts and imagery, and performs multiple
analyses across various information categories such
as weather patterns, weight and balance data,
possible aerial refueling tracks, high altitude and low
level route options, objective area terrain, threat
coverage, and order of battle data to develop a
general plan that includes possible landing zones,
tactics, escape and evasion routes, and
communications procedures. An interim assessment
of plan viability is performed at the end of each
planning cycle. Simulation technology has several
uses at this early point in planning. Planners could
use even an unenhanced digital database to analyze
the terrain, assess the viability of basis tactics, and
study the terminal area to determine the number and
type of aircraft that can be inserted. Historically,
these tasks were performed using paper maps, table
look-ups, and rules of thumb estimates.



SIMULATION-BASED MISSION REHEARSAL
AS A HUMAN ACTIVITY SYSTEM

INTRODUCTION

Simulation-based mission rehearsal (MR) became a
reality for the Air Force in 1990 when the MH-53J
Weapon System Trainer and Mission Rehearsal
System (WST/MRS) was delivered to the 542nd Crew
Training Wing (now the 58th Special Operations
Wing, or 58 SOW) at Kirtland AFB, NM by a group
within General Electric (who have since become part
of Lockheed-Martin). It was the ability of this
government-owned, contractor-operated system to use
“real world” photo-texture derived from imagery,
combined with Defense Mapping Agency (DMA)
terrain and cultural feature data, that provided the
Special Operations Forces (SOF) and Combat Search
and Rescue (SAR) forces their first MR capability
(Reed, 1993). This use of simulation added a level of
realism to rehearsal that is not duplicated by other
media. Planners have access to three-dimensional,
computer-generated imagery based on Defense
Mapping Agency data enhanced with cultural features
and other details from a variety of sources, and mission
participants can interact with other players in
networked simulators. Demonstrated benefits include
man-in-the-loop validation of mission plans leading to
improved mission plans and tactics, heightened crew
confidence and preparation, and a clearer
understanding of mission dynamics and risks. (See,
for example, Nullmeyer, Bruce, Conquest, and Reed,
1992).

In numerous rehearsals for training exercises, we
have seen that MR is, first and foremost, a
cooperative venture among people to get
crewmembers maximally prepared for their mission,
and it is these varied people who will determine the
effectiveness of advanced simulation technology for
this purpose. Integrating simulation-based rehearsal
capabilities with the existing _ functionally,

geographically, and organizationally diverse SOF and
SAR mission preparation process is clearly a
challenge. A key element for meeting this challenge
was realizing that simulation-based _rehearsal
technology and its uses by people (both Air Force and
contractor personnel) must be viewed as an integrated
system that we will refer to as a human activity system.
This point of view provides a conceptual structure to
organize, understand, and facilitate the diversified
activities of the many participants involved.

Banathy (1992) asserted that no single model can
truly represent a complex human activity system. He
was addressing educational systems. We believe his
concepts are equally useful for understanding
simulation-based rehearsal. He proposed three
models to capture the essence of such systems: a
systems-environment model, a functions/structure
model, and a process model. The systems-
environment model describes the context in which
the system operates and provides the “big picture”
for the remaining models. The functions/structure
model describes the components or parts, their
functions, how they are organized, and how these
elements are integrated into a structured system.
Finally, the process model provides a “motion
picture” image of the system, addressing the receipt
and transformation of inputs and other dynamic
operations associated with the system, including a
feedback loop to support system management,
adjustment, and change. Each model portrays certain
critical characteristics that must be overlaid upon
each other to paint a comprehensive picture of the
system as a whole. We will employ these “lenses”
to illustrate the importance of people in effective
simulation-based MR, using the 58 SOW MRS as the

prototype.

A HUMAN ACTIVITY SYSTEM MODEL OF SIMULATION-BASED MR

The Environment - Crisis Action Planning

Because simulation-based MRSs are designed to
support short-suspense missions, we chose to address
the MRS within context of crisis action planning.
Crisis action planning is a coordinated sequence of

events involving people, procedures,
communications, and support systems that leads to a
military response to a time-sensitive situation. The
primary participating organizations are listed on the
left side of Figure 1. The major events and products
for these organizations are depicted to the


Tasking Ni

Agenoy

MRS

Analysts

x

H
'
1
—
MRS
Database
Developers

Mission
Planners
and
Participants

Decision
Makers

Figure 3. Human Components of a Simulation-Based Mission Rehearsal System

(hardcopy, videotape), and authorize various data
providers to release mission information so that
database development may proceed. The quality of
the rehearsal environment is critically dependent on
Data Providers such as planners, weather officers,
and intelligence officers.

th ti re Ai Trainin; item
‘vision ron intelli
personnel provide tasking and mission information
to the MRS tactical analysts on the contractor team.
These analysts study the tasking along with annotated
maps, imagery, and threat information and begin
prioritizing areas to receive high- and mid-levels of
detailed database development. Clear lines of
communication between MRS personnel and the
supported organization are vital to insure that the
latest and highest fidelity source imagery is available,
the most recent routes are conveyed, and the general
tactical plan is relayed. Otherwise, precious time is
spent building databases for areas that are not part of
the operation, less accurate geographic information is
incorporated into the database, and fewer tactically
relevant features are modeled.

MRS_database developers use state-of-the-art
software and hardware to scan maps, warp imagery,
extract elevation data, manipulate contour lines, and
construct object models to create a realistic database
of the mission area. Information is exchanged among
analysts and programmers using a mix of verbal,
written, and electronic means. The quality of the

source data provided to the developers—be it target
imagery, hand held photos, or terrain information—is
a major determinant of the accuracy and realism of
the resulting digital database.

Mission _planners and_participants (planning cell
staff, aircrews, liaisons) use the rehearsal database to
revise, refine, and update their mission plan. Mission
information can be delivered in a variety of forms,
such as hardcopies of enhanced imagery, views of
partially modeled mission areas on work stations, and
completed versions of the database in a simulator.
Access to the MRS by planners can benefit both
planning and database development as summarized
on the right hand side of Figure 2. Early mission
planing decisions can help the database generation
process by identifying the areas where database
developers need to focus their efforts.

Senior Decision-makers (wing, squadron, and
mission commanders; executive and action officers;
and flight leaders) are clearly pivotal participants in
the mission preparation process, and are another
essential element of the MR human activity system.
They can watch actual rehearsals via large repeater
displays in a training observation center (TOC),
communicate directly with the Participating
Airerews over the intercom networks, and review
printouts of threat avoidance, navigation accuracy,
timing, and other indices via the Instructor Operator
Station (IOS). Using a combination of viewed
simulations, tactical judgment, electronic what-iffing,



ALJHR-TR-1996-0114

. SIMULATION-BASED MISSION REHEARSAL AS A
HUMAN ACTIVITY SYSTEM

Robert T. Nullmeyer

HUMAN RESOURCES DIRECTORATE
AIRCREW TRAINING RESEARCH DIVISION
6001 South Power Road, Building 558
Mesa, Arizona 85206-0904

Edward T. Reed, Lt. Col., USAF

58th Special Operations Wing/DOU
4249 Hercules Way, Bidg. 948
Kirtland AFB, NM 87117-5861

901 Olive Street
Santa Barbara, CA 93101

HUMAN RESOURCES DIRECTORATE
AIRCREW TRAINING RESEARCH DIVISION
6001 South Power Road, Building 558
Mesa, Arizona 85206-0904

September 1996

Final Technical Report for Period June 1994 - December 1995

Approved for public release; distribution is unlimited.

<DOAPAOWFr AZOAAVHSAY

AIR FORCE MATERIEL COMMAND
BROOKS AIR FORCE BASE, TEXAS

wnmenm 19961223 049



MR Preparation ]

Plan & Database’
Refinement

(Data analysis)

Rehearsal
Database Validated Plan|

Integrate & Certif

Figure 5. Functions and Products in the Mission Rehearsal Process.

and stored in frame buffers for a consistent display
rate. During each step, modelers and analysts must
work closely so that the most important features find
their way into the database.

The visual database is the most obvious, but by no
means, only digital product for MR. Other products
are the digital record of flight planning data, photos
and images digitized for subsequent manipulation
(annotation, mensuration, warping), and scanned
maps and charts. After the database is developed,
government and contractor personnel pre-fly the
database to certify its spatial and content integrity.
58 SOW personnel can fly the database at the
headings, altitudes, and airspeeds corresponding to
the mission plan. In Dry Run tests, the ATDs will be
integrated with the EW environment and wired for
full communication with the TOC.

MR. Upon arriving at the 58 SOW, mission
participants will be in-briefed on MR objectives and
schedule of events, the contents of the database, and
importantly, what has been omitted from the database
due to visual system limits, time constraints, and lack
of available source data. The "maturity" of the
participant's plan will dictate the activities required
for refining the mission plan and associated
database. Participants "trial fly" mission segments
in an ATD, trying out different entry points into the
area of operations, landing zones and launch points,

and approach headings and profiles. With the initial
planning complete, the rest of the time is spent
performing tasks associated with the primary
function, rehearsal. The type of operation
(feasibility assessment, plan validation, plan
verification, joint mission practice) will determine
the content and extent of the activities performed
here. Typically, the TOC is "up" to monitor network
activities, aircrew communications, status of the EW
environment, and coordinate MR activities amongst
the various _ participants. Time _ permitting,
participants will fly ingress routes into the target area
multiple times, examining the impact of different
turning radii, approach angles, altitudes, and multi-
ship formations. The primary output of MR is the
creation of a mission ready crew. Observers in the
TOC, including senior decision makers, can listen to
the audio on the network and evaluate the integrity of
the visual database and the performance of the EW
displays, and thus validate the mission plan while
interacting with the rehearsing crews in real time.

Post-MR. Most MR operations will conclude with a
hot wash in which mission participants and facility
personnel perform self-critiques of the benefits and
negative aspects to the MR just concluded. These hot
washes have been instrumental in making
constructive improvements to the MR process, both
from an efficiency of utilization and outcome
effectiveness standpoint. 58 SOW personnel may use


The major products of the database generation
system (DBGS) are the multiple digital databases
reflecting different environmental or sensor
conditions. The visual out-the-window (OTW)
database can be viewed through night vision goggles
(NVGs) and processed at different moon illumination
levels to simulate OTW night scenes. Other
databases are created to mimic the FLIR, radar, and
EW environments. Five different "portals" into the
Digital Databases are denoted by exiting arrows. The
arrow at the top leads to the ATDs. Hardcopy
printouts of any part of the visual scene
(photographs, maps, combat folders, navigation logs)
that can be taken on the aircraft during mission
execution. A third link is the TOC, in which large
repeater displays let senior decision-makers observe
the participating aircrews fly through the database in
the ATDs. Other outputs include a commercial-
quality ~

audio-visual recording studio that can record the
route flown through the database; a standalone work
station that lets aircrews "joystick" through the
Database decoupled from the aero-vehicle model;
and a distributed interactive simulation (DIS) node.

Mission planners and participants can derive benefit
from the Databases by taking hardcopy products
‘onboard the aircraft, flying through the database in
an ATD, and by using the Audio-Visual Recording
System (AVRS) to create a video recording of the
aircraft's rqute through the database and record crew
comments during flight. Senior decision makers can
view the mission unfold by watching videotapes from
the AVRS, the display monitors in the TOC, or
multi-group interactions over the DIS node. Inputs to
mission execution come from critiques by senior
decision-makers, the knowledge and experience
gained by mission planners and_ participating
aircrews,

Simulation-Based Rehearsal Processes

The process model describes the.dynamic nature of
the system. While each MR is unique, a core set of
functions must be performed to ensure that the
requisite materials, information, support equipment,
and people have been coordinated. Figure 5 divides
these critical functions into three phases—MR
preparation, MR, and post-MR. Although the
functions are sequenced temporally within each
phase, we have not overlaid an absolute timeline
since that depends on the complexity of the operation
and whether an existing database can be modified.

MR Preparation. Before mission participants arrive
at the 58 SOW facility, an intensive series of
preparation activities occur. An MR begins with a
tasking. A 58 OG/OGU representative is contacted
to determine if a particular operation can be
supported within a given time frame. - During initial
discussions, a 58 OG representative determines
which organizations will be providing route and
targeting data as well as transmitting source data.
Internally, the OG representative will work with the
designated MR Team Chief to line up the requisite
assets to support an MR, such as simulator time,
AVRS and IOS operators, data packs, Image
Generators (IGs) for the proposed mission, and
intersimulator networking. To facilitate data analysis
and database development, the 58 OG recommends
what and who the supported organizations should
send ahead and/or bring with them to the training and
rehearsal complex. The "what" entails identifying
necessary maps, charts, imagery, and intelligence
data in the region of interest. The "who" are specific
crewmembers, intelligence and logistics personnel,
and associated ground customers who should be on-
site during the MR.

Database development in the MRS is an extremely
involved function that can be organized into eight
steps: 1) digital terrain elevation data and digitized
source data are combined to create an underlying
terrain grid of elevation values; 2) cultural features
(e.g., roads, rivers, trees) are taken from digital
feature analysis data, photos, and images, and added
to the terrain; 3) the terrain is triangulated into
polygons by positioning a given image to an exact
latitude/longitude (warping) and elevation
(orthorectification); 4) terrain is textured to appear
more realistic by applying texture models coupled
with recent real- world photos of the mission area; 5)
two- and three-dimensional object models from a
customized library and added to the database; 6) the
database is compiled, processed, and constructed
with concentric levels of resolution. (Areas around a
target or landing zone having the highest resolution,
areas surrounding key landmarks having moderate
resolution, and ingress/egress corridors having a
lower resolution); 7) the database is formatted into
multiple layers—OTW/visual, radar, FLIR, EW, and
NAVAID—and then correlated so they can be
displayed from a common viewing point and 8) the
database is processed through the IG in which scene
objects are given geometrical perspective, data are
transformed into a display raster format, pixels are
given individualized color and brightness values,



Interim Assessment

Mission Tasking }+ General Planning

TenainAnaysis:
ai ane Stu

PlanR efinem ent
| Mission Preview
Plan Evaluation!Rehearsal
Rehearsal —
Bret Back | | Final Preparation |

Figure 2: An Overview of Wing and Squadron Activities in the Execution Planning Phase.

An aircrew briefing transitions the general plan to
squadron crewmembers who will complete the
planning details. During this handover, the crews are
briefed on the mission tasking, intelligence
information, predicted weather, and available
logistics. During detailed planning, crews insert
specific elements into the general plan and make final
decisions concerning key aspects of the mission.
Activities are laborious and include completing high-
altitude and low-level routes - specifying waypoints,
initial points, and other navigation data; finalizing
tanker track and ground-based refueling station
locations; selecting landing zones and holding areas;
determining terminal area tactics and procedures;
calculating weight & balance and fuel flow;
determining procedures to avoid threats, populated
areas, and lines of communication; and completing
the communication matrix, load plan, and mission
execution checklist. Simulation can support many of
these functions, such as deconflicting routes among
multiple aircraft, determining suitable sites for
landing zones, identifying methods for optimizing
aircraft performance, honing tactics, and coordinating
practice on perishable tasks like threat avoidance.

During evaluation/rehearsal, plan details are fine-
tuned and assessed under various “what-if?
contingencies. Commonly planned-for contingencies
include the appearance of new threats, take-off or
landing with reduced engines, bad weather, loss of

communications, loss of aircraft, and key equipment
failure. Here, simulation allows man-in-the-loop
validation of mission plans and tactics, which
heightens crew confidence in themselves and the plan.
Based on assessments during rehearsals and crew
“brief backs,” commanders can better estimate the
probability of mission success. In final preparation,
crews coordinate with other participants; finalize the
mission execution checklist; incorporate the latest
mission, weather, and threat data; and complete
aircraft configuration and flight procedures
immediately prior to executing the mission.

MRS Components and Their Functional
Relationships

Following the human activity system point of view,
the people, their functions, and their organization are
the system of interest. Planning and rehearsal
technologies are tools that support these people.
Figure 3 depicts the people who are essential for
effective simulation-based MR. Solid arrows reflect
person-to-person communication, and dashed arrows
depict equipment-mediated information flow. It is
important to note that this model extends beyond the
local 58 SOW and MR contractor organizations. The
tasking agency must provide the geographic
boundaries to be modeled, timeline, available source
data, routes to be flown, initial products desired


PREFACE

The work documented in this report was presented at the 17th Interservice/Industry
Training Systems and Education Conference (/ITSEC), held 13-16 November 1995 in
Albuquerque, New Mexico.

The report documents work conducted under Work Unit 1123-B3-01, Special
Operations Forces (SOF) Aircrew Training and Mission Preparation Research. The

.Laboratory Work Unit Monitor is Dr Robert T. Nullmeyer, AL/HRAU. This effort is part

of an Armstrong Laboratory, Human Resources Directorate, Aircrew Training Research
(AL/HRA) program to provide behavioral research support for the Air Force Special
Operations Forces (SOF) community. The AL/HRA program has two major thrusts: The
first is to specify measures for effectiveness and principles of technology utilization in
preparing for real-world missions. The second is to develop strategies for using advanced
simulators and measuring their effectiveness in combat mission training. The results of
this project services both of these thrusts. °



Form Approved
REPORT DOCUMENTATION PAGE Ca Nee 188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering
‘and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of
‘information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for information Operations and Reports, 1215 Jefferson Davis Highway, Suite
1204, Arlington, VA. ‘Sexo aso, ‘and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), ington, DC 20503.
4. AGENCY USE ONLY (Leave blank) | 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED
September 1996 Final - June 1994 to December 1995.
4, TITLE AND SUBTITLE 5. FUNDING NUMBERS
Simulation-Based Mission Rehearsal as a Human Activity System re - 622 05F
6. AUTHOR(S, -
Rober Nulimeyer a BS
Edward T. Reed
V. Alan Spiker
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Armstrong Laboratory
Human Resources Directorate
Aircrew Training Research Division
6001 South Power Road, Building 558 .
Mesa, AZ 85206-0904 AL/HR-TR-1996-0114
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING

41. SUPPLEMENTARY NOTES

Armstrong Laboratory Technical Monitor: Dr. Robert T. Nullmeyer (602) 988-6561
This technical report has been published in the Proceedings of the 17th Interservice/Industry Training Systems and Education
Conference, held 13-16 November 1995 in Albuquerque, NM.

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE

Approved for public release; distribution is unlimited.

13. ABSTRACT (Maximum 200 words)

Observations of the 58th Special Operations Wing (58 SOW) Weapon System Trainer/Mission Rehearsal System (WST/MRS)
during rehearsals have clearly shown that mission rehearsal (MR) effectiveness is influenced by the activities of both the
people who support the MRS and the people who use it. To explain this demonstrated importance of the people involved in -
MR, a human activity system model of simulation-based rehearsal was developed. It provides an integrated depiction of the
MRS, and specifically addresses: (1) the context for rapid database development and simulation-based MR (crisis action
planning) and how the MRS fits into it; (2) MRS components, functions and structure; and (3) processes that enhance rehearsal
effectiveness. Implications of this human activity system view are discussed, including: places in the mission preparation
process where simulation can benefit operations; the scope of human activities that are essential for successfull
simulation-based MR; the potential value of MR to provide feedback concerning the adequacy of training to support mission
requirements; and the need for new procedures throughout this extended scope of players to accommodate both MRS
requirements and capabilities.

14, SUBJECT TERMS . _. _. . 15. NUMBER OF PAGES
Flight simulation; Flight simulators; Flight training; Human activity system; Mission 15
preparation; Mission rehearsal; MR. . 16. PRICE CODE
77. SECURITY CLASSIFICATION | 18. SECURITY CLASSIFICATION [19. SECURITY CLASSIFICATION | 20. LIMITATION OF
OF REPORT OF THIS PAGE OF ABSTRACT ABSTRACT
Unclassified Unclassified Unclassified UL
NSN 7540-01-280-5500 Standard Form 208 (Rev 2-89) Prescribed by ANSI Std 2-30-16

298-102 COMPUTER GENERATED



NOTICES

When Government drawings, specifications, or other data are used for
any purpose other than in connection with a definitely Government-related
procurement, the United States Government incurs no responsibility or any
obligation whatsoever. The fact that the Government may have formulated or
in any way supplied the said drawings, specifications, or other data, is not to
be regarded by implication, or otherwise in any manner construed, as licensing
the holder, or any other person or corporation; or as conveying any rights or
permission to manufacture, use, or sell any patented invention that may in any
way be related thereto.

The Office of Public Affairs has reviewed this report, and it is releasable
to the National Technical Information Service, where it will be available to the
general public, including foreign nationals.

This report has been reviewed and is approved for publication.

Fobrctd hrepe—~—— earntatnr nbech

ROBERT T. NULLMEYER ELIZABETH L. MARTIN
Project Scientist Technical Director

A. Ci OLL, Colonel, USAF

lef, Aircrew Training Research Division

Please notify AL/HRPP, 7909 Lindbergh Drive, Brooks AFB, TX 78235-5352, if your
address changes, or if you no longer want to receive our technical reports. You may
write or call the STINFO Office at DSN 240-3877 or commercial (210) 536-3877.



the AVRS to produce a videotape of the database
flown at the headings, routes, altitudes, and airspeeds
planned for by the crew. With full cockpit audio, the
videotape can be edited to include additional footage
(e.g., 360 degree scans of target areas from different
perspectives, such as ground level) to be viewed by
aircrews and decision-makers at forward operating
locations. Aircrews or the studio operator can also

provide explanatory voice overs. Finally, mission
data and databases are archived in the event the plan
is subsequently refined, re-flown, or updated. Since
many operations take place in the same geographic
region, the visual database becomes a highly useful
means to store and summarize previous tactical
analyses.

IMPLICATIONS AND LESSONS LEARNED

The importance of people in complex systems is
often acknowledged. However, it is not always easy
to translate this concept into reality, as evidenced by
advanced technologies being delivered with poor
user documentation, or with little or no training for
users. We believe the human activity system “lenses”
provide useful templates to facilitate this translation.
In this final section, we will explore a few
implications of these templates when applied to the
58 SOW MRS.

First, the context lens of crisis action planning shows
that the value of MR can extend well beyond the
traditional view of practice and validation
immediately preceding mission execution. Figure 1
shows the supporting command involved at three
points within the crisis action planning process: (1)
developing and evaluating alternative military
courses of action, (2) developing the detailed
operations plans, and (3) executing the plan given
receipt of an execute order. All three tasks can
benefit from access to a digital database representing
the area of operation. Early in the planning process,
alternative courses of action and other planning
activities can be supported with “man-in-the’ loop”
queries. Similarly, simulation can provide a dynamic
environment for risk analyses and decision making
during mission execution. Other participants in crisis
action planning could also benefit from simulation.

Second, the functions/structure lens shows that the
human system that must be in place for effective
simulation-based MR clearly extends beyond the
locally-controlled MRS. We have seen, for example,
that the quality of the MR database is limited by the
quality of input data from outside sources. MR
effectiveness is also determined in large part by user
(participants and decision makers) knowledge of
MRS capabilities, data requirements, capabilities, and
utilization strategies. Of some note, early users of
simulation-based MR uniformly praised the
capabilities of the technology and people, while

lamenting the lack of a strategy to incorporate this
capability into the mission preparation process
(Nullmeyer, et. al., 1992). Predictably, it has been
easier to improve processes within the MRS than to
coordinate activities of the geographically and
organizationally diverse team comprising the larger
mission preparation system, . However, effective
simulation-based MR requires the larger system to be
functioning as an integrated whole.

Some specific examples may help illustrate this
point. 1) The MRS has a photo overlay capability.
Annotations were traditionally placed on photographs
to add important and relevant information, but now,
these annotations add work for database developers,
who must remove them during database construction.
For a photo-based MRS, it is clearly better for
database developers to have both an annotated and an
unannotated photograph. 2) Experience has taught
that MR database responsiveness and quality is
enhanced when the user is directly involved during
database development. Today’s technology does not
allow all things to be modeled perfectly, so database
developers attempt to find the “best fit” between
MRS capabilities and mission needs. A mission
participant or planner can provide valuable guidance
to focus system resources on the most critical needs.
This user representative can also serve as an
interpreter for the other rehearsal participants to
describe where the simulation is accurate and where
it is not. In both examples, MR efficiency and
effectiveness are impacted by people who are not
part of the MRS per se.