Technical Report 1075

Observations About Defining Collective Training

Requirements

A White Paper Prepared in Support of the ARMS Program

Kenneth D. Cross

Bayview Research

John A. Dohme and William R. Howse

U.S. Army Research Institute

June 1998 19980721 047

United States Army Research Institute

for the Behavioral and Social Sciences

Approved for public release; distribution is unlimited.

•0I1C qptftf* ^
U.S. Army Research Institute

for the Behavioral and Social Sciences

A Directorate of the U.S. Total Army Personnel Command

EDGAR M. JOHNSON

Director

Research accomplished under contract

for the Department of the Army

Bayview Research

Technical Review by

John E. Stewart

Robert H. Wright

NOTICES

DISTRIBUTION: Primary distribution of this Technical Report has been made by ARI.

Please address correspondence concerning distribution ofreports to: U.S. Army

Research Institute for the Behavioral and Social Sciences, Attn: TAPC-ARI-PO, 5001

Eisenhower Ave., Alexandria, VA 22333-5600.

FINAL DISPOSITION: This Technical Report may be destroyed when it is no longer

needed. Please do not return it to the U.S. Army Research for the Behavioral and Social

Sciences.

NOTE: The findings in this Technical Report are not to be construed as an official

Department of the Army position, unless so designated by other authorized documents.
REPORT DOCUMENTATION PAGE

1. REPORT DATE (dd-mm-yy)

June 1998

2. REPORT TYPE

Interim

4. TITLE AND SUBTITLE Observations About Defining Collective Training

Requirements: A White Paper Prepared in Support of the ARMS Program

6. AUTHOR(S)

Kenneth D. Cross, Bayview Research

John A. Dohme,ARI

William R.Howse,ARI

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Bayview Research, 117 Lisa Marie Place, Shalimar, FL 32579

Anacapa Sciences, Inc., 2000 N. Pantops Drive, Charlottesville, VA 22911

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

U.S. Army Research Institute

Rotary Wing Aviation Research Unit

ATTN: TAPC-ARI-IR, Building 5100

FortRucker, AL 36362-5354

3. DATES COVERED (from... to)

October 1996-April 1997

5a CONTRACT OR GRANT NUMBER

Mr>AQm-q3-C-016l

5b. PROGRAM ELEMENT NUMBER 0622785A

5c. PROJECT NUMBER

.ÄZ2J

5d. TASK NUMBER

_22U

5e. WORK UNIT NUMBER

CQ2

8. PERFORMING ORGANIZATION REPORT NUMBER

ASI-BR 105813-97-1

10. MONITOR ACRONYM

ARI

11. MONITOR REPORT NUMBER

Technical Report 1075

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT (Maximum 200 words):

In the past, collective training requirements have been defined in terms of the mission, mission segments, or broad functions that an

Army aviation unit must learn to perform as a group. It is impossible to infer from these broad task descriptions the specific knowledge

and skills that can be acquired only through collective training. Without a clear understanding of these requisite knowledge and skills, it

is impossible to make prudent decisions about the level of realism that is required for each component of a collective training simulator.

This white paper describes the nature of the problem and describes the authors' views about the unique knowledge and skills that can

be acquired and sustained only through collective training exercises. All comments are aimed at the training requirements for the

Aviation Reconfigurable Manned Simulator (ARMS) and developed by the U.S. Army National Guard Bureau (USANGB).

15. SUBJECT TERMS

Collective Training, Collective Training Requirements, Defining Collective Training Requirements, Aviation Reconfigurable Manned Simulator (ARMS),

Army Aviator Training, Combined Arms Training

16. REPORT

Unclassified

17. ABSTRACT

Unclassified

18. THIS PAGE

Unclassified

I 19. LIMITATION OF

ABSTRACT

Unlimited

20. NUMBER

OF PAGES

36

21. RESPONSIBLE PERSON

(Name and Telephone Number)

Dennis C. Wightman, Ph.D.

(334) 255-2834/9091
Technical Report 1075

Observations About Defining Collective Training

Requirements

A White Paper Prepared in Support of the ARMS Program

Kenneth D. Cross

Bayview Research

John A. Dohme and William R. Howse

U.S. Army Research Institute

Rotary-Wing Aviation Research Unit

Dennis C. Wightman, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

June 1998

Army Project Number Education and Training Technology

20262785A791

Approved for public release; distribution is unlimited.

m
FOREWORD

This paper was written as part of an effort by U.S. Army Research Institute for the

Behavioral and Social Sciences (ARI) Rotary Wing Aviation Research Unit (RWARU)

to support the U.S. Army National Guard Bureau (USANGB). Specifically, the

USANGB initiated a project to design and implement an Aviation Reconfigurable

Manned Simulator (ARMS). The ARMS project is a pioneering effort to develop a

simulation system to train and sustain "collective" task performance. Collective task

refers to a class ofmission-oriented activities that depend upon coordinated interactions

among multiple crews. The USANGB requested that ARI RWARU develop

procurement support documentation to assure attainment oftheir training objectives.

The documentation produced included a Front End Analysis, Functional Requirements

Document, and Training Effectiveness Analysis. The development and integration of

these three documents made it apparent that additional clarification was needed to

support the USANGB. The design of a simulator to train collective tasks within aviation

is a relatively new undertaking. This paper therefore addresses the definition of

collective training and provides guidance to engineers, training developers and trainers

who endeavor to meet the challenge of ever increasing training requirements and

decreasing training assets. Training requirements are considered in terms ofindividual

and shared knowledge and skills required for performance of collective tasks and which

can only be attained and sustained through collective training.

ZITAM. SIMUTIS

Technical Director
OBSERVATIONS ABOUT DEFINING COLLECTIVE TRAINING

REQUIREMENTS: A White Paper Prepared in Support Of the ARMS Program

EXECUTIVE SUMMARY

Research Requirement:

Collective training is considered an essential element ofArmy aviation units'

training programs. Heretofore, collective training has been accomplished through field

training exercises. For a host ofreasons, it is becoming less feasible to accomplish

effective collective training through field training exercises. Among the most important

reasons are the high cost of planning and conducting field training exercises and the

scarcity ofgeographical areas where realistic exercises can be conducted. These

problems have caused Army officials to assess the feasibility of conducting collective

training in networked simulators capable of operating in a common virtual environment.

Systems currently being assessed include the Aviation Reconfigurable Manned Simulator

(ARMS) and the Aviation Combined Arms Tactical Trainer (AVCATT). The ARMS

and AVCATT are intended for use by reserve component (RC) aviators and active

component (AC) aviators, respectively.

In the past, collective training requirements have been defined in terms ofthe

missions, mission segments, or broad functions that Army aviation unit personnel must

learn to accomplish as a team. This approach is not adequate for defining training

requirements for a virtual collective training system. It is impossible to infer from these

broad function and task descriptions the specific knowledge and skills that can be

acquired only through collective training. Without a clear understanding ofthe requisite

knowledge and skills, it is impossible to make prudent decisions about the level of

realism required for each component of a virtual collective training system.

For the reasons discussed above, there is a pressing need to define collective

training requirements in terms ofthe fundamental knowledge and skills that Army

aviation unit personnel can acquire only through collective training exercises. The

requisite knowledge and skills must be defined for each individual who participates in

one or more collective tasks performed during mission planning or mission execution.

Procedure:

This paper presents preliminary thoughts about the types ofknowledge and skills

that can be acquired only through collective training. Although the comments focus on

the training requirements for the ARMS, most are equally applicable to other training

systems designed wholly or in part to accomplish collective training.

The ideas presented in this paper were influenced by information drawn from a

variety ofsources. Many useful insights were gained from the direct observation ofthe

planning, execution, and review of collective exercises conducted in the Aviation Test

Bed (AVTB) located at the U. S. Army Aviation Center (USAAVNC), Fort Rucker,

Vll
Alabama. Observations were made ofthe collective exercises performed as part ofthe

Focused Dispatch (FD) Advanced Warfighting Experiment (AWE), which was conducted

in August 1995 (Cross & Bierbaum, 1995). Subsequently, observations were made of

collective exercises performed during two other studies conducted in the AVTB: an

evaluation ofthe ARMS proof of principle device and the digital training exercise

(DTX).

A great deal ofuseful information was found in reports that describe lessons

learned from distributed interactive simulation (DIS) training exercises that involved a

mounted brigade (e.g., Koger, et al., 1996; Winsch, Garth, Ainslie, and Castleberry,

1996) and a multi-service task force (e.g., Bell, et al., 1996). Useful information also was

found in reports that summarized the research literature on team training (e.g., Prince and

Salas, 1993), cockpit resource management (CRM) training (e.g., Wiener, Kanki, and

Helmreich, 1993), situational awareness training (e.g., Garland and Endsley, 1995;

Gilson, Garland, and Koonce 1994), and training on practical thinking in battle (e.g.,

Fallesen, Michel, Lussier, and Pounds, 1996).

Finally, the ideas presented below were influenced by a general knowledge about

(a) human perception and cognition and (b) Army rotary-wing aviator training methods

and devices. .

Findings:

Discussed briefly below are functions for which the acquisition of adequate

knowledge and skill wasjudged to be impossible without some amount of collective

training. For all ofthese functions, individual and crew training is necessary but not

sufficient to develop the level ofknowledge and skill that is required. Important mission

planning functions and mission execution functions are discussed in turn.

Collective mission planning exercises were judged essential for acquiring the

knowledge and skills needed to perform the six broad mission planning functions

summarized below.

• Resource Management. Collective training is needed for the unit

commanders to (a) acquire in-depth knowledge about the specific

capabilities and limitations of each member ofthe planning staff and

(b) develop the skills needed to assign responsibilities in a manner that

maximizes the collective productivity ofthe planning staff. Collective

training also is needed for commanders to learn to improve inter-staff

communication through constructive critique, delivered in a positive

manner, with clearly stated alternative courses of action.

• Use ofDigital Equipment. Collective training is needed to acquire the

knowledge and skills needed to fully utilize the digital equipment that

will be used for planning future missions. First, collective training is

needed to enable both the commander and the equipment operators to

acquire comprehensive knowledge about the capabilities and

Vlll
limitations ofthe digital equipment used for mission planning.

Second, collective training is required for digital equipment operators

to develop their knowledge and skills to the level of mastery that is

needed.

• Terrain Analysis. Collective training is needed to enable commanders

to learn to develop a terrain analysis team consisting ofindividuals

who are capable and willing to articulate their perceptions and

conclusions and, when appropriate, take issue with another team

member's perceptions and inferences.

• Plan Formulation/Refinement. Collective training is needed to enable

the commander to determine the organization and processes required

for the planning staffto synthesize the planning components into a

unified mission plan.

• Common Cognitive Model. Collective training is needed for the commander

and members ofthe planning staffto learn methods for developing a common

cognitive model ofthe situation and the mission.

The training benefits of collective mission execution exercises are heavily

dependent on the type and number ofbattlefield elements that participate. An exercise

that involves only the aircraft crews provides no opportunity for training on many

collective functions and tasks. Of particular importance, limiting the exercises to aircraft

crews precludes training on interactions among aircraft crews and the personnel who

occupy the airborne unit's tactical operations center (TOC). In fact, it can be argued that

the need for collective training is greater for the ground-based battle staff members than

for the aircraft crews. Accordingly, the benefits summarized below reflect the

assumption that the participants in a collective mission execution exercise include both

multiple aircraft crews (e.g., an attack company) and the personnel who occupy the

supporting TOC (e.g., company or battalion TOC).

Collective mission execution exercises are considered essential for developing the

knowledge and skills needed to perform the seven functions listed below. It is

recognized that some ofthe functions overlap.

• Skill Deficiencies. Collective mission execution exercises in a virtual

environment provide a unique opportunity to identify and eliminate

important knowledge and skill deficiencies. In addition to providing

important feedback on proficiency, the exercises enable the

commander and his subordinates to acquire knowledge about the best

performance assessment techniques and the best methods for using the

simulation system's capabilities to eliminate key knowledge and skill

deficiencies.

• Situation Awareness. Establishing and maintaining an adequate level

ofsituation awareness is truly a collective task. Situation awareness is

most critical for the commander ofthe airborne unit and for the

commander ofthe TOC. However, some level ofsituation awareness

is essential for every participant. The knowledge and skills required to

IX
establish and maintain situation awareness probably cannot be

acquired in any way other than collective mission execution exercises.

• Information Needs ofOthers. An important skill that can be acquired

best through collective mission execution exercises is the skill required

to recognize the information needs of other members ofthe friendly

force. This skill is closely related to the skills required to establish and

maintain situation awareness.

• Communications. Realistic mission execution exercises are essential

for developing key communication skills. Such exercises enable

participants to acquire the knowledge and skills needed to select the

most suitable communication mode, establish priorities when message

traffic is high, communicate effectively under high workload

conditions, and determine where and when different types of

information should be communicated.

• Workload Management. Acquiring the knowledge and skills needed

to manage excessive workload is an important and unique benefit of

collective mission execution exercises. Such exercises enable every

participant to learn to recognize when workload is excessive and to

learn methods for coping effectively with excessive workload.

• Team Resource Management. Although team resource management

skills are poorly understood, it is reasonable to assume that important

problems and inefficiencies result from the lack ofsuch skills.

Collective mission execution exercises may be the only way to identify

the team resource management problems that degrade mission

effectiveness and to develop methods for eliminating the skill

deficiencies that cause such problems.

• Tactical Decision Making. Army personnel presently receive a

substantial amount oftraining on battlefield tactics. However,

collective mission execution exercises are necessary for personnel to

acquire much ofthe knowledge and skills needed to make prudent

tactical decisions in a timely manner.

Utilization ofFindings:

The findings ofthis effort will be published as part of a report on the ARMS

front-end analysis. Two purposes are served by also reporting the findings in a separate

ARI report. First, it is hoped that the findings will influence the development oftraining

requirements and design specifications for other collective training devices, especially

those developed to train Army unit personnel. Second, it is hoped that the findings will

stimulate more research and discussions aimed at developing more detailed and

comprehensive definitions ofthe knowledge and skills that can be acquired only through

collective training.
OBSERVATIONS ABOUT DEFINING COLLECTIVE TRAINING

REQUIREMENTS A WHITE PAPER PREPARED IN SUPPORT OF THE ARMS PROGRAM

CONTENTS

Page

INTRODUCTION 1

THE NEED 2

PURPOSE 3

INFORMATION SOURCES 4

UNIQUE BENEFITS OF COLLECTIVE TRAINING 5

Collective Mission Planning Exercises 5

Resource Management 6

Allocation ofresponsibilities • -'—6

Recognizing and coping with resource management

problems • 7

Use ofDigital Equipment 7

Capabilities and limitations of equipment 7

Equipment operation 8

Terrain Analysis 8

Intelligence Information 9

Plan Formulation/Refinement 10

Common Cognitive Model 10

Collective Mission Execution Exercises 10

Skill Deficiencies 12

Situational Awareness ....12

Information Needs of Others 14

Communications 14

Workload Management 16

Team Resource Management • .....16

Tactical Decision Making • 17

KEY QUESTIONS 18

RECOMMENDED ACTIONS 21

Mission and Task Analysis 21

Review NTC Lessons Learned Database 21

Review Latest Literature on Situational Awareness 22

Identify/Describe Collective Task Knowledge/Skills 22

Review Performance Measurement Literature 22

REFERENCES 25

APPENDIX A

A

"!

XI
OBSERVATIONS ABOUT DEFINING COLLECTIVE TRAINING REQUIREMENTS

A White Paper Prepared in Support of the ARMS Program

Introduction

Eventually, decisions will have to be made about each ofthe scores of design parameters

that will comprise the Aviation Reconfigurable Manned Simulator (ARMS). An essential first

step in making such decisions is to establish a clear understanding ofthe knowledge and skills to

be acquired and sustained through ARMS training exercises. Unfortunately, the ARMS

documentation contains only general information about the training requirements that must, in

turn, determine the ARMS functional and design requirements.

It is clear from existing documents that the primary impetus for the ARMS acquisition is

the lack of Training Aids, Devices, Simulators, and Simulations (TADSS) that are capable of

training collective tasks in a combined arms and joint environment (U.S. Army National Guard

Bureau [USANGB], 1995a, 1995c, 1995d). It also is clear that ARMS must be designed to train

rotary wing aviators at the company level (USANGB, 1995b, p D-2) to perform their units'

assigned collective tasks. * However, no documents have been located that contain a clear

description ofthe knowledge and skills that should be acquired and/or sustained as a result of

participating in ARMS collective training exercises.

Only very general training requirements can be inferred from the information contained

in such key documentation as the Army National Guard Aviation Simulation and Training

Strategy (USANGB, 1995b) and the Combined Arms Training Strategy (CATS) Task

Templates. Appendix D ofthe draft Army National Guard Aviation Simulation and Training

Strategy (USANGB, 1995b) contains a list of "representative collective tasks" (see Table D-l, pp

D5-D6). Although referred to as collective tasks, this list consists of a collection ofmissions

(e.g., Conduct Deliberate Attack), mission segments (e.g., Perform Passage ofLines), and

functions (e.g., Perform Formation Flight). This list of "collective tasks" has some value in

defining the types and scope oftraining that must be conducted in ARMS, but it provides little

insight about the essential knowledge and skills that can be acquired only through a collective

training exercise conducted in ARMS.

The CATS documentation includes "templates" that list key missions and Army Training

and Evaluation Program (ARTEP) subtasks for which various types offield training exercises

must be developed and implemented by aviation unit commanders. Some useful insights about

collective training requirements can be gained from a careful study ofthe CATS templates.

However, as was true for the "collective task" list discussed above, the CATS templates provide

no specific information about the unique knowledge and skill deficiencies that ARMS must be

designed to eliminate.

It is understandable that Army training documentation contains little useful information

about the specific knowledge and skills that can be acquired only through collective training.

1 Brief mention is made in the Army National Guard Aviation Simulation and Training Strategy (USANGB, 1995b)

of a requirement for ARMS to train fixed wing aviators. However, the requirement to train fixed wing aviators was

eliminated after the training strategy documentation was first released.
Because collective training traditionally has been conducted through field training exercises, it

has not been necessary to articulate the unique benefits of collective training. Army training

officials (training developers and training managers) have made the perfectly reasonable

assumption that realistic field exercises will reveal collective task deficiencies and that the

performance deficiencies can be eliminated through (a) repetitive exercises and (b) the

knowledge acquired during after action reviews (AARs).

It may have been adequate to define field training requirements only in terms ofthe types

of collective field training exercises that must be performed. However, defining ARMS training

requirements only in terms ofthe collective exercises that must be performed is not adequate.

Such a definition ignores the issue ofrealism2or, more specifically, selective realism. Without a

clear understanding ofspecific training requirements, it can only be assumed that the ARMS

virtual environment must have sufficient realism to enable every participant to perform every

task in a highly realistic manner. Uniformly high realism ofthis type probably is technologically

unfeasible at this time and most certainly is neither necessary nor affordable.

The Need

The performance of a complex collective training exercise (e.g., Conduct a Deliberate

Attack) requires each participant to perform a large number of different tasks. The knowledge

and skills needed to perform many ofthe tasks can (and should) be acquired through individual

training or crew training prior to participation in a collective training exercise. For ARMS, only

limited training benefits are realized from the resources expended to achieve realism in tasks for

which there is no knowledge or skill deficiency or tasks that can be trained more effectively in

another way. Other things being equal, resources are best spent to achieve realism in the tasks

for which the requisite knowledge and skills can be acquired and sustained only through

collective training in ARMS. Hence, it is necessary that training requirements be defined in a

manner that leads to meaningful deliberations about the tasks for which realism contributes

directly to training effectiveness vs. the tasks for which realism contributes only indirectly to

training effectiveness, or not at all.

For purposes ofthis discussion, the composite set oftasks required to perform a complex

collective training exercise can be subdivided into the following three categories.

• Category I—the set ofindividual tasks and crew tasks for which skills can be

acquired and sustained through individual or crew training (assuming the

resources are available to do so). This category includes most aircraft control

tasks (e.g., perform takeoff, fly nap ofthe earth [NOE], perform autorotation)

and most aircraft equipment operation tasks (e.g., operate radios, operate

aircraft survivability equipment [ASE]).

2

The term "realism" is used here to refer to the similarity between the virtual environment and the real world

environment that it emulates. The terms realism and fidelity have similar meanings. In this context, the term

realism is preferred because fidelity is generally used in discussing the attributes of a specific simulator component

(e.g., cockpit instruments, equations of motion).
• Category II--the set of collective tasks3

that can be performed adequately

without collective training, by virtue oftheir simplicity or because they can be

trained adequately in another way. The establishment of a unit standing

operating procedure (SOP) may be adequate to insure the performance ofsome

collective tasks (e.g., tactical communications, battlefield surveillance).

Similarly, the effective performance of other collective tasks may be highly

dependent on knowledge oftactics that can be acquired in the classroom (e.g.,

tasks requiring the application oftactical principles). Such tasks would be

classified as Category II tasks.

• Category IH--the set of collective tasks for which knowledge and skills can be

acquired and sustained only through the collective training ofthe individuals or

crews who must contribute in order for the tasks to be accomplished

successfully. The tasks that should be classified into Category III remain a

matter ofspeculation. However, the authors' views about Category III tasks are

reflected in the following sections ofthis white paper.

It seems self-evident that Category III should be weighted most heavily in defining

ARMS training requirements. However, Category III tasks do not, in themselves, constitute

training requirements. Rather, training requirements must be defined in terms ofthe knowledge

and skills that are required to perform the Category III tasks.

In summary, there is a need to define ARMS training requirements in terms ofthe

fundamental knowledge and skills that are required to perform Category III tasks effectively.

The requisite knowledge and skills must be defined for each individual who must contribute to

the accomplishment ofthe collective task.

Purpose

The main purpose ofthis white paper is to present some preliminary thoughts about the

types of knowledge and skills that can be acquired only through collective training. Some

reviewers ofthis white paper have argued with the claim that there are some types of knowledge

and skill that can be acquired and sustained adequately only through collective training. They

have argued, correctly, that the performance of every task draws upon enabling knowledge and

skills that are acquired through individual or crew training. Although true, this argument does

not invalidate the claim that, for some tasks, the requisite knowledge and skills can be acquired

adequately only through collective training. Indeed, the fundamental premise underlying this

white paper is that the knowledge and skills that can acquired through individual or crew training

are necessary but not sufficient to ensure an acceptable level of performance on some tasks at an

acceptable cost. Collective training may be essential for any one or more ofthe following

reasons:

3

The term "collective task" is used here to refer to a task for which success or failure depends on more than one

individual or more than one crew. By definition, a collective task consists oftwo or more individual tasks; however,

an individual or crew task is not necessarily a component of a collective task. It follows that collective training is

training (on a collective task) in which all ofthe collective-task participants are involved.
• an adequate level of existing knowledge and skill cannot be achieved without

collective training,

• some knowledge and skills are drawn upon only when functioning in a team context,

and

• requisite knowledge and skills can be acquired through individual or collective

training, but the training costs are far greater than acquiring the same level of

knowledge and skill through collective training.

Reviewers also have argued that it is possible that the types ofknowledge and skills

discussed in the following pages could be acquired with a computer-based training system in

which a computer served as a surrogate for one or more team members. Although this

possibility cannot be denied, replacing a team member with a computer surrogate does not

change the fact that collective training is being accomplished. In short, the possibility of using

computer surrogates for team members does not invalidate the claim that some knowledge and

skills can be acquired effectively only through collective training.

For a variety ofreasons, the knowledge and skills that can be acquired only through

collective training are poorly understood. One reason, as suggested above, is that effective

(collective) field-training exercises can be designed and conducted without a clear understanding

ofthe specific knowledge and skill deficiencies that are eliminated as a result ofthese exercises.

(The same can be said for most individual and crew tasks.) A second reason is that most ofthe

knowledge and skills that can be acquired only through collective training are largely cognitive,

and, therefore, difficult to define in other than very general terms (e.g., tactical decision making

skills, information processing skills).

A second purpose ofthis paper is to identify some ofthe key questions that should be

considered by the group ofsubject matter experts (SMEs) whose ultimate responsibility is to

make final decisions about ARMS training requirements, functional requirements, and design

requirements. The third purpose is to recommend actions that are likely to provide at least

partial answers to the questions posed.

This white paper is not intended to be comprehensive in its discussion ofthe knowledge

and skills that can be acquired only through collective training or its listing of key issues.

Rather, it is intended to serve as a stimulus for further discussions about how best to proceed in

formulating and documenting ARMS training requirements, functional requirements, and design

requirements.

Although this white paper focuses on training requirements for ARMS, most ofthe

comments are equally applicable to other ongoing training system development efforts such as

the Aviation Combined Arms Tactical Trainer (AVCATT).

Information Sources

The ideas presented below were influenced by information drawn from a variety of

sources. Many useful insights were gained from the direct observation ofthe planning,

execution, and review of collective exercises conducted in the Aviation Test Bed (AVTB)
located at the U. S. Army Aviation Center (USAAVNC), Fort Rucker, Alabama. Observations

were made ofthe collective exercises performed as part ofthe Focused Dispatch (FD) Advanced

Warfighting Experiment (AWE), which was conducted in August 1995 (Cross & Bierbaum,

1995). Subsequently, observations were made of collective exercises performed during two

other studies conducted in the AVTB: an evaluation ofthe ARMS proof of principle device and

the digital training exercise (DTX).

A great deal ofuseful information was found in reports that describe lessons learned from

distributed interactive simulation (DIS) training exercises that involved a mounted brigade (e.g.,

Elliott, Sander and Quinkert, 1996; Koger, et al., 1996; Winsch, Garth, Ainslie, and Castleberry,

1996) and a multi-service task force (e.g., Bell, et al., 1996). Useful information also was found

in reports that summarized the research literature on team training (e.g., Prince and Salas, 1993),

cockpit resource management (CRM) training (e.g., Wiener, Kanki, and Helmreich, 1993),

situational awareness training (e.g., Garland and Endsley, 1995; Gilson, Garland, and Koonce

1994), and training on practical thinking in battle (e.g., Fallesen, Michel, Lussier, and Pounds,

1996).

Finally, the ideas presented below were influenced by the authors' general knowledge

about (a) human perception and cognition and (b) Army rotary-wing aviator training methods

and devices.

Unique Benefits of Collective Training

The ARMS tasking documents suggest that ARMS must be capable of providing

collective training on both mission planning tasks and mission execution tasks. Accordingly, the

following subsections discuss, first, the unique benefits4of collective mission planning exercises

and, second, the unique benefits of collective mission execution exercises.

Collective Mission Planning Exercises

In some instances, mission planning5at the company level may consist oflittle more than

refining a detailed mission plan developed by the battalion staff. However, aviation company

staff members must be capable of planning complex missions with no more input from the

battalion staffthan a description of(a) the commander's intent, (b) the concept of operation, and

(c) the tasks to units. Some ofthe fundamental skills required to plan complex missions can be

and should be acquired through individual training. However, many highly critical mission

planning skills simply cannot be acquired in any way other than through collective mission

planning exercises. The following comments address the ways in which collective mission

planning exercises contribute uniquely to the acquisition ofthe knowledge and skills that are

needed for the detailed planning of complex missions. Although the focus is on mission

planning at the company level, virtually all ofthe comments are equally applicable to mission

planning at the battalion level.

4

For ease of exposition, the term "unique benefits" of collective training is used to refer to the knowledge and skills

that can be acquired and sustained only through collective training.

5

The term mission planning is used here in its broadest sense to refer to the planning of any type of military

operation and the preparation ofthe requisite planning documents and overlays.
It is essential that company personnel learn to accomplish mission planning as a

collective effort. Under realistic conditions, insufficient time is available for one or two persons

to accomplish all the tasks that must be performed to plan a mission and to produce the various

products that are needed (e.g., operational orders, overlays). The need to plan missions as a

collective effort becomes more apparent as mission complexity increases and as time constraints

become more severe. Indeed, the skills needed to plan missions under severe time constraints

are certain to be among the most important skills acquired through collective mission planning

exercises.

Although few would argue with the claim that mission planning must be accomplished as

a collective effort, it is necessary to establish the need for collective training on mission

planning. Specifically, it is necessary to establish that some ofthe essential knowledge and skills

can be acquired only through collective mission planning (training) exercises. That is one intent

ofthe following comments.

Resource Management

The effectiveness ofmission planning is influenced greatly by the effectiveness with

which members ofthe planning staff manage the resources available to them. Discussed below

are resource management skills that can be acquired effectively only through collective mission

planning exercises.

Allocation ofresponsibilities. Both Army doctrine and unit standing operating

procedures (SOPs) dictate the responsibilities of company personnel during mission planning.

However, because of constant unit turnover and the resultant variability in the experience level

ofunit personnel, commanders must be sensitive to the level of planning proficiency of each

staff member and must be able to make rapid adjustments to task organization to compensate for

short-term skill deficiencies. Hence, there is a requirement for training to ensure that

commanders possess the knowledge and skills needed to detect weak or failing performance of a

member ofthe planning staff and to take appropriate and timely corrective action to offset the

deficiency. It is doubtful that such knowledge and skills can be acquired in any way other than

through participation in collective mission planning exercises.

It is common for individual members ofthe company planning staffto specialize in one

component ofthe planning process (e.g., threat assessment, terrain analysis). Specialization

tends to motivate individuals to seek an ever-higher level ofskill in performing the specialized

tasks. When the capability to perform a task resides in one individual, however, it becomes

impossible to accomplish the task more quickly through a collective effort. Moreover, the

mission planning effectiveness ofthe entire company would be temporarily degraded in the

event ofillness or transfer of an individual who possesses most ofthe company's capability to

perform a complex planning task. The commander must acquire a clear understanding ofboth

the benefits and risks ofspecialization and must learn to allocate responsibilities in a manner that

achieves the optimal balance between specialization and cross training.
In summary, ARMS must be designed to support collective mission planning exercises

that enable commanders to (a) acquire in-depth knowledge about the specific capabilities and

limitations of each member ofthe planning staff and (b) develop the skills needed to assign

responsibilities in a manner that maximizes the collective productivity ofthe planning staff It is

worth noting that acquiring knowledge about the individual capabilities and limitations ofthe

planning staff also enables the commander to identify individual training needed by members of

the planning staff.

Recognizing and coping with resource management problems. It has been found that

certain types ofCockpit Resource Management (CRM) problems can contribute to catastrophic

aircraft accidents (see Wiener, Kanki, & Helmreich, 1993). One example of a CRM problem is

the reluctance of a subordinate to take issue with a decision made by or an action taken by a

person with a higher rank and/or more seniority. It is certain that similar problems contribute to

non-optimal plans, planning procedures, planning products, or some combination ofthese.

Participation in collective mission planning is probably the only way that the company

commander and other members ofthe company planning staff can learn to recognize and cope

with CRM-type problems that degrade mission planning efficiency and reduce the quality ofthe

resulting plans and products.

Learning to recognize and cope with resource management problems requires collective

mission planning exercises that foster improvements in inter-staff communications through

concentration on performance feedback that consists of constructive critique, delivered in a

positive manner, with clearly stated alternative courses of action.

Use ofDigital Equipment

The use of digital equipment for mission planning will increase in both frequency and

importance, as the plans for a digital battlefield become a reality. A great deal ofthe knowledge

and skills needed to use digital equipment effectively can be and should be acquired through

individual training, study, and practice. However, there is compelling evidence that some

requisite knowledge and skills are difficult to acquire in any way other than participation in

collective mission planning exercises. The examples discussed below should not be considered

comprehensive.

Capabilities and limitations of equipment. The Focused Dispatch AWE provided

compelling evidence that collective mission planning exercises (with realistic time constraints)

are needed to acquire essential knowledge about the capabilities and limitations ofthe Aviation

Mission Planning System (AMPS) and the All Source Analysis System (ASAS). As might be

expected, the commander and executive officer (XO), who had not been trained as an equipment

operator, acquired the most knowledge about the capabilities and limitations ofthe AMPS and

the ASAS as a result oftheir participation in the collective mission planning exercises.

However, a great deal of knowledge about equipment capabilities and limitations also was

acquired by the AMPS operators and ASAS operators, who had received extensive individual

training on their respective systems. In short, despite extensive individual training, the

equipment operators failed to learn or failed to recall some ofthe important capabilities and

limitations oftheir equipment.
These observations support the argument that collective mission planning exercises are

needed and should be designed to enable both the commander and the equipment operators to

acquire comprehensive knowledge about the capabilities and limitations ofthe digital equipment

that is used to accomplish mission planning tasks.

Equipment operation. The Focused Dispatch AWE also provided compelling evidence

that even highly trained AMPS and ASAS operators did not acquire all the requisite knowledge

and skills during individual training on the systems. Some knowledge and skill deficiencies

became apparent only when the equipment operators were required to support the planning of a

realistically complex mission under realistic time constraints. Most commonly, equipment

operators knew how to extract information from the system or to process information but were

unable to perform the tasks quickly enough. In a smaller number ofinstances, the operators

simply did not know how to perform information extraction or processing tasks that they knew to

be well within the capabilities ofthe systems. This finding suggests that over learning ofsome

equipment operations is necessary to ensure sufficiently robust performance under battle

conditions. Collective mission planning exercises should reveal the tasks for which over

learning is required and may provide a training environment that fosters the over learning that is

required.

It could be argued that such problems reflect shortcomings in the individual training

program. However, anyone who has learned to use a computer understands that (a) no amount of

studying instruction manuals and no amount ofsimple training exercises fully prepares one to

perform the full range of complex jobs for which the computer is capable, and (b) performing a

task only a few times does not ensure that the task procedures will be recalled after a few months

with no practice.

Terrain Analysis

Terrain analysis is a task that often can be accomplished better by a team than by an

individual. As is true for other mission planning tasks, time constraints may make it impossible

for even the most skilled individual to accomplish an effective terrain analysis in the time

available. In addition, the fundamental complexity ofthe terrain analysis for some missions may

make it difficult for an individual to accomplish an effective terrain analysis alone. Terrain

analysis involves such complex perceptual and cognitive processes that the same array of

information (e.g., commander's intent, intelligence data, topographic maps) is almost certain to

be perceived and evaluated differently by different staff members. Ifthese differences are shared

in a constructive manner, the terrain analysis is certain to be more complete and valid than

otherwise would be possible.

Even ifindividuals have received extensive individual training on terrain analysis,

collective mission planning exercises are required for the commander and the other members of

the company planning staffto appreciate fully the benefits of performing terrain analysis as a

collective effort and to learn how best to coordinate their efforts. Ifsome members ofthe

planning staff have not had extensive training on terrain analysis, collective mission planning

exercises provide an opportunity for commanders to observe skill deficiencies in their staff and

to eliminate those deficiencies through subsequent individual training or practice.
The knowledge acquired from collective mission planning exercises should lead the

company commander to develop a terrain analysis team consisting ofindividuals who are

capable and willing to articulate their perceptions and conclusions and, when appropriate, take

issue with another team member's perceptions and inferences. This benefit of collective mission

planning exercises overlaps the benefit, described above, oflearning to recognize and cope with

CRM-type problems.

Intelligence Information

A primary motive underlying the development ofthe "digital battlefield" is to provide

more comprehensive and more current intelligence information than has been available in the

past. Ifthe promised benefits ofthe digital battlefield are to be realized, members ofthe

company's planning staff must learn to acquire and to evaluate the full range of intelligence

information that is available and accessible. An important determinant ofsuccessful mission

planning is knowledge about (a) the intelligence information that is needed to plan an operation,

(b) the availability ofthe requisite information, (c) the best way to acquire the information, (d)

the amount oftime needed to acquire the information, and (e) the comprehensiveness and

reliability6ofthe information.

Participation in collective mission planning exercises is the best way and may be the only

effective way to acquire some ofthe knowledge needed to compile and evaluate intelligence

information in a timely manner. Because every mission is unique in some respects, it is not

possible to develop a detailed "template" that specifies the exact type and amount of intelligence

information that is needed to plan a given type ofmission. Rather, members ofthe planning staff

must learn to infer intelligence information requirements from the limited information available

at the outset of planning. It must be learned that inferring intelligence information requirements

is an iterative process whereby one intelligence item suggests the need for other items of

intelligence information. Company personnel also must learn the type and amount of

information that is necessary and sufficient, how best to use available resources to acquire

intelligence information, what to do when there is insufficient time to compile important

intelligence information, and how to assess the reliability ofintelligence information (taking into

account the source ofthe information and its currency).

Like terrain analysis, inferring intelligence information requirements and evaluating

intelligence information is a complex cognitive process that often can be accomplished better by

a team than by an individual. Collective mission planning exercises provide an opportunity for

the commander to determine the optimal team composition, allocate functions to team members,

and promote constructive interaction among team members. At the same time, collective

mission planning exercises enable members ofthe planning team to (a) learn to anticipate the

intelligence needs ofthe commander and other team members and (b) learn how to communicate

information and judgments efficiently and accurately.

6

The reliability of intelligence information is influenced by its accuracy and its currency. That is, even information

known to be accurate cannot be assumed reliable ifthe information is not sufficiently current.
Plan Formulation/Refinement

The formulation and refinement of a complex mission plan is complicated by the

requirement for the plan to be doctrinally acceptable and tactically sound while adhering to the

SOPs of higher and lower echelons of all task force participants. Although members of a

company staff may possess a great deal of academic knowledge about doctrine, tactics, and

SOPs, collective mission planning exercises are essential for acquiring the knowledge and skills

needed to formulate a tactically sound plan that is doctrinally acceptable and compatible with the

SOPs of other participants in the task force. Only through collective mission planning exercises

can the commander determine the organization and processes that enable members ofthe

planning staffto synthesize the planning components (e.g., results ofthreat analysis, results of

terrain analysis) into a unified mission plan.

Common Cognitive Model

There is anecdotal evidence that many ofthe problems that degrade mission effectiveness

stem from the fact that the members of aircrews and teams do not share a common cognitive

model ofthe mission, the situation, or both. The creation of a common cognitive model has been

defined as one ofthe most important objectives ofmission rehearsal. Therefore, it is reasonable

to assume that an important benefit of collective mission planning exercises is to develop ways

to promote a common cognitive model among the members of an aviation company and perhaps

other participants as well (e.g., battalion commander, commanders of other units). This process

is poorly understood, so little can be said at this point about how to determine when cognitive

models differ in important respects and how to eliminate such differences. Even so, it seems

certain that learning to develop common cognitive models will prove to be one ofthe most

important benefits of collective mission planning exercises.

Before proceeding, it should be noted that the concept of cognitive models is similar to

the concept ofsituational awareness, a concept that is discussed in more detail below. In the

present context, the emphasis is on the differences in the participants' cognitive models and the

problems that are caused by these differences. In discussing situational awareness, the emphasis

is on the comprehensiveness and validity of cognitive models and the problems that occur when

a participant's cognitive model is not sufficiently comprehensive or is invalid (indicating a lack

ofsituational awareness).

Collective Mission Execution Exercises

The main purpose ofthis subsection is to present preliminary thoughts about the

knowledge and skills that (a) are essential for mission execution and (b) can be acquired and

sustained only through collective mission execution exercises. The knowledge and skills

required to perform Category III tasks during mission execution vary to some degree as a

function ofsuch factors as mission type, aircraft type, and level of command. This discussion

focuses on knowledge and skill requirements that are common to members of an aviation

company. It is important to note, however, that some members of an aviation company require a

higher level ofsome types of knowledge and skills than other members. In general, the number

ofCategory III tasks that must be performed and the level of knowledge and skills required to

10
perform the tasks are closely related to the level of command. It follows that the potential

benefits of collective mission execution training exercises are related to command level.

Although the ARMS is to be designed to provide collective training at the company level,

there are reasons to question whether effective training can be accomplished without

participation by the battalion commander and members ofthe battalion staff A large amount of

resources have been and will continue to be expended to equip Aviation Tactical Operations

Centers (AVTOCs) and the Army Airborne Command and Control System (A2C2S) with digital

systems that enable the battalion commander to play a more active role in supporting aviation

companies during mission execution. Observations made during the previously cited studies

conducted in the AVTB provided clear evidence that a substantial amount of practice was

required for the battalion commander and the company commander (controlling the airborne

company) to learn how and when to use the information and the communication links available

to them. During the first few exercises, mission effectiveness was degraded substantially

(although not catastrophically) by (a) the battalion commander's and the company commander's

lack of knowledge about when to use the available digital equipment and communication links

that were available and (b) their lack ofskill in operating the digital equipment.

Many ofthe Category III tasks that must be performed by the company commander or

operations and training officer (S3) involve interaction with the battalion commander. It is

doubtful that the knowledge and skills that the company commander and S3 need to perform

these tasks can be acquired and sustained ifthe battalion commander does not participate in the

mission execution exercises in a realistic manner. It is equally doubtful that the battalion

commander can acquire the knowledge and skills required to perform Category III tasks without

participating in mission execution exercises with each company commander and company S3.

For these reasons, it is argued that the training effectiveness of ARMS would be far greater ifit

included a simulated AVTOC or A2C2S that would be occupied by the battalion commander

during mission execution exercises. A well equipped AVTOC or A2C2S simulation also would

provide the capability to train members ofthe battalion staffto perform important collective

tasks (both mission planning and mission execution).

Because ofthe many duties of battalion staff members, it is unreasonable to expect that

all members ofthe battalion staff could be available to participate in every collective mission

execution exercise of every company. When other duties prevent one or more members ofthe

battalion staff from participating in a collective training exercise, other members ofthe battalion

staff or members ofthe company staff could perform the job ofthe missing battalion staff

members. In fact, valuable cross training could be realized from requiring a member ofthe

battalion staffto perform another staff member's regular job during a collective mission

execution exercise.

There is no reason to doubt that a well-equipped command post would greatly increase

the training effectiveness ofARMS for both company-level and battalion-level staff. Hence, a

decision about whether or not to equip ARMS with a simulated command post must be based on

cost data and, ultimately, on judgments about whether the training benefits of a command post

outweigh its cost. Based on the limited information available, it is the authors' judgment that the

training benefits of a command post would easily outweigh its cost.

11
Skill Deficiencies

A very general but very important benefit of collective mission execution exercises is the

feedback information they provide about the skill deficiencies ofthe participants. This benefit

applies to both individual and collective tasks. Deficiencies in the skills required to perform

some individual or crew tasks may go undetected until it becomes necessary to perform the tasks

in the context of a mission with realistic threats and time constraints. Although the purpose of

ARMS is not to provide individual or crew training that can be accomplished elsewhere, the

feedback provided during a collective training exercise can be used to tailor subsequent training

(in the aircraft or in another training device) to eliminate the skill deficiencies revealed during

the exercise.

A unique benefit of ARMS mission execution exercises is the information they provide

on proficiency in the performance of collective tasks (Category II and Category III tasks). Such

information can be acquired in no other way short of a realistic field training exercise or actual

combat operations. It can be used to tailor subsequent collective training exercises to eliminate

knowledge and skills deficiencies. Feedback about skill deficiencies ofindividuals and crews

also can be used by commanders to organize their unit and allocate functions to minimize the

impact ofinexperienced or marginally skilled unit personnel.

Conducting collective mission execution exercises in the virtual environment provided by

ARMS yields many important and unique benefits. Unlike field exercises, collective training in

a virtual environment provides opportunities to repeat specific missions or mission segments as

needed to (a) eliminate specific performance deficiencies, (b) experiment with alternate tactics,

procedures, and task organizations, (c) accomplish cross training of personnel, and (d) compare

the performances of different companies and personnel.

The benefits of collective mission execution exercises discussed above have important

implications for ARMS functional and design requirements. Specifically, ARMS must be

designed in a manner that enables the performance ofunit personnel to be measured, recorded,

and assessed (both individually and collectively). The performance measures must provide the

diagnostic information needed to identify the specific knowledge and skill deficiencies that

contributed to inadequate performance. Moreover, an after action review (AAR) facility must

have the equipment needed to assess the unit's individual and collective performance and to

illustrate the incidence and consequences ofboth effective and ineffective performances.

Situational Awareness

Although the importance ofsituational awareness is widely recognized, research on how

best to define, measure, and train it has only recently begun. Three components ofsituational

awareness have been defined by Endsley (1995a, 1995b).7The following represents an attempt

to tailor (paraphrase) Endsley's definitions to be more specific to the situational awareness of

Army aviation unit personnel.

7

The literature contains numerous definitions ofsituational awareness. Endsley's definition has come to be the

preferred definition of many experts in the field.

12
• Perception ofrelevant battlefield elements. The perception of all available

information about battlefield elements, including both friendly and opposing

forces.

• Comprehension of meaning. Comprehension ofthe meaning of all information

about battlefield elements when evaluated in the context of existing

topography, mission, and command level.

• Projection of elements into the future. The projection of battlefield elements

into the near-term future and the interpretation ofthe forecast.

One ofthe greatest potential benefits of ARMS mission execution exercises is training

that enhances each participant's ability to establish and sustain an adequate level ofsituational

awareness. In order to realize fully this important benefit, it will be necessary to ensure that

ARMS is designed to provide the information needed for each trainee to establish and maintain a

realistic level ofsituational awareness. Specifically, ARMS must be designed to provide no less

information and, equally important, no more information than is likely to be available on the

battlefield. Similarly, ARMS should not provide information that is more or less comprehensive

and current than the information that is likely to be available on the battlefield. This important

point is discussed in more detail below.

In a virtual environment, it is a simple matter to provide comprehensive intelligence

information (e.g., OPFOR [Opposing Force] disposition and strength) with virtually no time

delay. Obviously, training with intelligence information that is unrealistically comprehensive

and current would not teach company personnel to function with the incomplete, aging

information that will be available to them on the battlefield. To the contrary, a host ofbad habits

could result from situational awareness training in which the information available to trainees is

unrealistically comprehensive, unrealistically current, unrealistically accessible, or unrealistically

valid.

Having argued the importance of providing realistic information, it must be

acknowledged that, early in training, the effectiveness of collective mission execution exercises

might be facilitated by providing more information or more current information than ordinarily

would be available on the battlefield. As participants' skills increase, the amount and quality of

the information could be decreased, in steps, until a realistic level is reached. Such action would

be justified only in instances in which the training value of an exercise is lost because one or

more participants lacked the skills needed to become situationally aware with realistic

information. In this sense, the amount and quality ofinformation could be manipulated as a

training tool.

An important objective ofsituational awareness training is to teach personnel to

recognize when they lack the information that is needed to be situationally aware. Although

little is known about how to accomplish such training, there is no doubt that humans can learn to

recognize (a) when they lack the information needed to perform a task effectively or to make a

13
prudent decision, and (b) the rapidity with which essential information ages and the frequency

with which it must be updated.

Another important objective ofsituational awareness training is to teach personnel where

and how to gain access to information that is needed to establish and sustain an adequate level of

situational awareness. As more and more digital systems become available on the battlefield, the

number ofsources ofinformation will increase along with an increase in the knowledge and

skills needed to gain access to the information quickly.

So, in very general terms, ARMS training must (a) teach personnel to assess continuously

their momentary level ofsituational awareness and (b) teach them where and how to gain access

to the information needed to become or remain situationally aware.

The AAR facility and procedures must be designed to identify failures in the three

elements ofsituational awareness: perceiving all important battlefield elements, comprehending

the meaning ofinformation perceived, and projecting elements into the future. Endsley (1995a,

1995b) has pointed out that it is not possible to perform the second or third element in the

sequence unless the preceding elements have been performed. It is possible that training benefit

on an element could be realized even though the preceding element(s) were not performed

adequately. However, it equally possible that inadequate performance of one element could

preclude useful training on all subsequent elements. For these reasons it is essential that trainees

be given detailed feedback on their performance of each element ofthe situational awareness

sequence and the reasons for inadequate performance.

Information Needs of Others

An important skill that can be acquired best through collective mission execution

exercises is the skill required to recognize the information needs of other members ofthe

friendly force. During battle, individuals occasionally are exposed to information that is

critically important for other personnel but not directly available to them. It is essential that all

members of an aviation company learn to evaluate information in terms ofboth their personal

needs and the needs of other company, battalion, and task force personnel. Individual aviators

must learn to recognize the needs ofthe company commander; the company commander must

learn to recognize the information needs ofthe battalion commander; and the battalion

commander must learn to recognize the information needs ofindividuals at higher command

levels. Conversely, individuals at all command levels must learn to recognize the needs of all

personnel under their command.

Equally important, individuals must learn how best to communicate important

information to others. The importance of collective training for acquiring the knowledge and

skills needed to communicate effectively is discussed below.

Communications

Effective communication has long been recognized as an important determinant ofthe

success ofmilitary operations. Many ofthe systems that have been or are now being developed

14
for the digital battlefield will increase the amount and currency ofinformation, but will not

necessarily make communication easier or simpler. To the contrary, the acquisition and

sustainment of communication skills is likely to be made far more difficult by (a) the number and

complexity of communication equipment, and (b) the requirement to communicate with a larger

number and variety oftask force elements. It seems certain that some ofthe essential

communication skills can be acquired only through collective training exercises.

The communication load is particularly heavy for aviation company commanders. For

example, company commanders must know when, how, and what to communicate to the

following nodes:

• other aircraft in the commander's company (e.g., convey commands, designate

targets, hand over targets),

• the battalion AVTOC and A2C2S (e.g., convey intelligence information, battle

damage reports, downed aircraft reports, respond to queries),

• fire support centers, including Army field artillery, Navy artillery, and allied

artillery (e.g., direct fire, obtain permission/instructions to pass through fire

zones),

• air defense (AD) units (e.g., alert AD to the presence ofthreat aircraft, obtain

permission/instruction to pass through air defense zones), and

• ground maneuver units (e.g., infantry, armor).

Effective communication involves much more than knowledge of how to operate the

various communication equipment of established communication procedures. The commander

also must possess the knowledge and skills needed to (a) select the type of communication

system that is most suitable for the circumstances (e.g., voice, digital), (b) establish proper

priorities when the communication load exceeds the time available, (c) obtain information that is

needed to establish and maintain an adequate level ofsituational awareness, (d) convey

information that others need to maintain their situational awareness, and (e) communicate

effectively under realistic workload conditions and time constraints. Of course, the act of

communicating requires a comprehensive knowledge ofthe lexicon, syntax, and abbreviations

that have been established (formally or informally) for each type of communication system.

Workload management, discussed in more detail below, is an essential part of effective

communication. That is, company commanders must learn to delegate communication tasks to

their subordinates (e.g., platoon leaders) when their workload is excessive. At the next higher

level, battalion commanders and a battalion S3s, whose workload is often clearly excessive, must

learn how and when to delegate responsibility for communication tasks to other members ofthe

battalion staff.

15
If collective mission execution exercises are to be effective in teaching communication

skills, the ARMS cockpits and command posts must contain simulations or emulations of all

communication equipment that are available in the parent aircraft and the command posts.

Furthermore, effective training will require the replication ofthe communication network nodes,

links, channel capacities, signal quality and traffic density. It seems highly likely that

instructional effectiveness would be increased by manipulating the attributes ofthe

communication system over realistic ranges.

Collective training with unrealistic communication equipment could lead to the

development ofineffective communication procedures and to over-reliance or under-reliance on

some communication systems. Moreover, the use of unrealistic communication equipment could

lead to unrealistically high or unrealistically low workload.

Workload Management

Acquiring the knowledge and skills needed to manage excessive workload is certain to be

an important benefit of collective mission execution exercises. Individual pilots must learn to

recognize when their workload is excessive and learn methods for coping effectively with

excessive workload. Similarly, company commanders and battalion commanders must learn to

recognize and cope with situations in which their workload or the workload of a subordinate is

excessive.

Although little is known about how to train individuals to recognize excessive workload,

some attempts have been made to teach individuals methods for coping with it. The methods

include (a) establishing the priority oftasks and performing them in the order oftheir priority

and (b) offloading oftasks to another person. The training of commanders should also include

methods for reallocating functions among unit personnel to avoid grossly unequal distribution of

work among the personnel under their command. A commander must learn the optimal

allocation offunctions for routine operation and must learn to make impromptu changes in the

allocation offunctions when the commander's workload or the workload of a subordinate

becomes excessive.

Commanders' options for managing workload by allocating functions are heavily

dependent on their subordinates' ability to perform more tasks than the ones associated with their

primary duty assignment. So, to the extent possible, commanders must promote the cross

training oftheir subordinates. It follows that important objectives of collective mission

execution exercises are to foster the cross training of personnel and to assess the effectiveness of

cross training in a variety ofmission contexts.

As has been stated earlier, workload management skills are equally important for mission

planning, especially when mission planning time is limited and the mission is complex.

Team Resource Management

Collective mission execution exercises in ARMS should enable members of an aviation

company to acquire resource management skills, which are somewhat different from the

16
Salas, 1993). Resource management skills are similar to those discussed in the subsection on

mission planning. CRM skills enable aircraft crews to function effectively and safely; team

resource management skills enable multiple aircraft teams and joint forces to function effectively

and safely.

Elements ofteam resource management skills are poorly understood at the present time.

However, it seems reasonable to assume that observation of only a few collective mission

execution exercises would lead to (a) the identification of critical team resource management

skills deficiencies and (b) the development oftraining methods for eliminating the deficiencies.

Tactical Decision Making

The importance oftactical decision making skills cannot be overemphasized. Although a

great deal ofresources are expended in developing Army aviators' tactical decision making

skills, the training provides few opportunities to exercise such skills in a realistic mission

context. There are many reasons to believe that collective training in a device such as ARMS is

the most effective way for personnel to acquire much ofthe knowledge and skills they need to

make prudent tactical decisions in a timely manner.

Assuming that ARMS is capable of creating a realistic virtual battlefield (topography,

friendly forces, and opposing forces), trainees can be exposed to a large number of different

tactical situations in a relatively short time. This makes it practical to observe trainees' tactical

decision making skills in a wide variety oftactical situations. The pace ofbattle can be varied to

control the time available for making tactical decisions; the number and disposition offorces

(friendly and opposing) can be varied to control the complexity ofthe tactical decisions that must

be made; and the availability ofintelligence information can be varied to control the difficulty of

making prudent tactical decisions.

There is no question that tactical decision making skills can be enhanced through

classroom training on tactics and SOPs and through complex field training exercises. In

addition, there is evidence that tactical decision making can be enhanced through classroom

training on practical thinking methods (Fallesen, et al., 1996). However, there is a limit to the

tactical decision making skills that can be acquired in the classroom. The cost offield training

exercises is so high that trainees can be exposed to only a limited number oftactical situations

during their career.

To provide effective tactical decision making training, ARMS must be capable of

confronting trainees with representative, event-driven tactical situations that require responses

under realistic time constraints and workload conditions. The training must facilitate effective

tactical decision making with a realistic amount ofintelligence information. Decisions must be

doctrinally acceptable and in compliance with unit SOPs.

17
Key Questions

The knowledge and skills needed to perform Category III tasks should be weighted most

heavily in defining ARMS training requirements. The overriding question is the extent to which

resources should be expended on ARMS design attributes to provide realism in Category I and

Category II tasks ifthe design attributes are not needed to perform Category III tasks in a manner

that is sufficiently realistic to support effective training. For illustration, consider two examples:

the aircraft equations ofmotion and the resolution ofthe out-the-window display.

The results ofthe studies conducted in the AVTB leave no doubt that effective training

on some collective tasks can be accomplished with highly unrealistic equations of motion. The

anecdotal evidence from the studies suggests that the key issue is whether or not the equations of

motion make the simulated aircraft so difficult to fly that the pilot's workload is far higher in the

simulator than in the parent aircraft. It seems reasonable to assume that problems would also

arise ifthe aircraft were so easy to fly that the pilot's workload in the simulator was far lower

than in the parent aircraft. Based on this evidence, it can be argued that it is the realism ofthe

workload level rather than the realism in specific handling qualities that dictates the requirements

for realism in the simulated aircraft's equations ofmotion. Of course, this becomes a moot issue

in the likely event that highly realistic equations ofmotion can be implemented at about the same

cost as less realistic equations of motion.

Display resolution is certain to be an important cost driver. There is no doubt that

extremely high resolution would be required to provide effective training on some individual

Category I tasks (e.g., target detection and identification, weapons firing from a realistic standoff distance). However, it is unlikely that the same high level ofresolution is needed to enable

the Category III tasks to be performed with sufficient realism to support effective collective

training. In fact, it has been argued that more effective collective training can be accomplished

with a relatively low resolution display than with a high resolution display. This argument is

based on the premise that the missions that are the most dependent on the effective performance

of collective tasks are those that must be accomplished under degraded visibility conditions. If

this premise is valid, it follows that (a) collective training under degraded visibility conditions

would be at least as effective as collective training under optimal visibility conditions, and (b)

degraded visibility conditions can be simulated adequately with a relatively low resolution (less

costly) out-the-cockpit display system. An opposing argument is that tactics are so dependent on

visibility conditions that collective training is required under both high visibility and degraded

visibility conditions.

The overriding question discussed above gives rise to a host ofmore specific questions

such as the ones listed below. All ofthese questions have a direct bearing on the definition of

ARMS training requirements, operational requirements, and design requirements. Answers to

these questions should increase the quality ofthe decisions that must be made about ARMS

functional and design requirements. It must be recognized, however, that answers to some

questions must await the development and evaluation of a prototype ARMS.

18
1. What are the Category HI tasks in each ofthe missions, mission segments,

and functions that are ofinterest? This question is clearly among the most

important and difficult questions that must be addressed. It is closely related

to the even more basic question listed next.

2. For each Category III task, what are the requisite knowledge and skills that

can be acquired and sustained only through collective training?

3. If a mission segment contains no Category III tasks, is it necessary or

beneficial to include that mission segment in an ARMS training exercise? To

answer this question, it is necessary to consider whether the exclusion ofthe

mission segment in question would adversely influence (a) the continuity of

the mission segment, (b) the situational awareness ofthe trainees, (c) the

workload of participants, and (d) trainee acceptance. Of course, costs also

must be considered. Even though a mission segment contains no Category

IE tasks, there is no reason to exclude it unless doing so would result in a

reduction in system acquisition costs or training costs.

4. Ifthe ARMS is designed to train company personnel to perform collective

tasks under degraded visibility conditions, can it be concluded that the

training will enable the personnel to perform the tasks with equal

effectiveness under better visibility conditions? The key issue is whether

tactics and/or collective tasks vary so much with visibility conditions that it is

essential to provide specific training under both good and poor visibility

conditions.

5. What level ofrealism in the performance ofCategory I tasks and Category II

tasks is required to support effective (collective) training on Category III

tasks? As stated above, realism in workload level (attentional demand) is

certain to be an important consideration. Similarly, it is important that the

performance ofCategory I tasks and Category II tasks does not result in (a)

negative transfer to the aircraft or (b) the establishment offormal or informal

operating procedures that are inappropriate for real world operations.

6. How much commonality in Category III tasks exists across the missions,

mission segments, and functions ofinterest? There is no question that some

types of collective tasks are an essential part of nearly every type of mission.

The issue is whether the operational conditions or mission context influences

task performance to such an extent that training on the task must be provided

in a wide variety oftopographic and mission contexts.

7. How could a collective training system be designed to maximize the

likelihood that it will accommodate the training of new tasks and procedures?

The adoption of new tactics and the introduction of new equipment in Army

aircraft and Army aviation command posts is certain to result in new tasks

and procedures for which collective training is required. Changes in tactics

19
and equipment at higher command levels also may result in the need for

collective training on new tasks at the company and battalion level.

8. What equipment and methods are needed to assess trainees' performance on

Category III tasks and to identify the key knowledge and skill deficiencies

that contribute to the inadequate performance ofthese tasks? The importance

ofthis question cannot be over emphasized. Some ofthe problems

associated with developing valid, diagnostic measures of collective task

performance are discussed below.

Although all ofthe above are key questions, the first two are most basic in the sense that

little progress can be made in defining ARMS training requirements without at least a general

understanding ofthe full complement ofCategory III tasks and the underlying knowledge and

skills that are required to perform them. Neither the research literature nor the training literature

contains clear-cut answers to these important questions. As a consequence, it will be necessary

to make best guesses about the answers to the questions and to use the best guesses as bases for

later evaluation and revision once a prototype ARMS becomes available.

The last question (Question 8), concerning performance measurement, also is a very

important and very basic question because effective training simply is not possible unless

trainees' performance can be assessed and their knowledge and skill deficiencies can be

identified. The measurement and diagnosis of performance on collective tasks is complicated by

a host offactors, but two are especially important. First, because the outcome of a collective task

is the end result of a sequence ofsubtasks performed by multiple individuals, it is difficult to

identify all the subtasks that contributed to inadequate performance ofthe collective task.

Inadequate performance of a subtask by one individual may make it impossible for other

individuals to perform subsequent subtasks adequately. Conversely, the effects ofinadequate

performance by one individual may be offset by the alert performance of another individual.

Second, it is difficult to measure performance on the many cognitive subtasks that

contribute to the performance of a collective task. This difficulty stems from the fact that (a) the

effectiveness with which cognitive subtasks are performed must be inferred from overt actions,

and (b) extended periods oftime may pass with no overt actions that provide an indication of

how well the cognitive tasks are being performed. Unless an event occurs that requires overt

action, there is no way to determine whether or not the cognitive tasks are performed adequately

without disturbing the flow of events. Further, even when an overt action indicates that a

cognitive subtask was not performed adequately, it is difficult to identify the knowledge or skill

deficiencies that led to the inadequate performance. Measuring and diagnosing performance is

particularly difficult for such cognitive subtasks as monitoring a situation display, evaluating a

stream ofintelligence data, establishing and maintaining situational awareness, and making

tactical decisions.

20
Recommended Actions

A comprehensive program of empirical research would be required to obtain complete

answers to the questions posed above and others that are relevant. However, the time and

resources needed to design and implement an empirical research program are not available.

Hence, the recommended actions include only analytical studies that can be accomplished in a

short period oftime with limited resources. The recommended actions listed and discussed

briefly below are not necessarily comprehensive.

Mission and Task Analysis

Task analyses for a sample ofmissions have already been accomplished in support ofthe

ARMS project. There is a need for a group ofsubject matter experts (SMEs) to review the task

analysis worksheets and to classify the tasks listed using the classification scheme described

earlier (Category I, Category II, and Category III tasks). It is recommended that the group of

subject matter experts (SMEs) include experienced Army aviators who have a thorough

knowledge of Army aviation combat operations and who have been briefed on both the

categorization scheme and the purpose ofthe classification effort. The group also should include

experienced behavioral scientists knowledgeable about Army aviation and Army aviation

systems.

The most important product ofthis effort is a listing ofCategory III tasks, viz., collective

tasks for which the requisite knowledge and skills can be acquired only through collective

training.

Review NTC Lessons Learned Database

There is anecdotal evidence that the training exercises conducted at the National Training

Center (NTC) yield much useful information about the types of collective tasks that are not

performed effectively by participating units. However, no known attempt has been made to

gather data at NTC for the expressed purpose of gaining insight about the collective tasks that

cannot be performed effectively. The knowledge and skills deficiencies that contribute to

inadequate performance of collective tasks have not been addressed. It is understood that NTC

maintains a database on lessons learned during the training exercises, but little is known about its

comprehensiveness or format.

It is recommended that effort be expended to learn more about the contents ofthe NTC

lessons learned database, the manner in which the database can be accessed and reviewed, and

the time and resources required to do so. With this information, it will be possible to make a

prudent judgment about the potential benefits and costs of an effort to systematically compile

and analyze data from the lessons learned database that would be useful for defining collective

training requirements.

21
Review Latest Literature on Situational Awareness

There is a need to compile and review the latest situational awareness literature that has

relevance for collective training. The literature review should locate definitions ofsituational

awareness that are most suitable for the collective training of Army aviators. Equally important

is locating literature on unobtrusive methods for measuring situational awareness and effective

methods for training aviators to achieve and maintain an adequate level ofsituational awareness.

Fortunately, much ofthis literature can be found in the published proceedings oftwo

international conferences on situational awareness (Gilson, Garland, & Koonce, 1994; Garland &

Endsley, 1995).

Identify/Describe Collective Task Knowledge/Skills

As was discussed earlier, collective training requirements cannot be defined in a

meaningful way by simply listing the composite set of collective training tasks that the

participants must be capable of performing. Rather, collective training requirements must be

defined in terms ofthe fundamental knowledge and skills that participants can acquire only

through collective training. Unfortunately, little is known about the fundamental knowledge and

skills that underlie collective task performance. Moreover, there is no commonly accepted

terminology or lexicon for describing the knowledge and skills.

It is recommended that a team ofbehavioral scientists study the set ofCategory in tasks

identified during the task classification effort described above, identify the underlying

knowledge and skills, and develop a lexicon for describing the knowledge and skills.

Review Performance Measurement Literature

There is a pressing need to compile and evaluate all available information about

methods and equipment for measuring the performance of collective tasks and the use of

performance measures to enhance collective training. A review ofthe performance measurement

literature must encompass at least the measures and measurement functions discussed by Bell

and his colleagues (Bell, et al., 1996) and listed below.

• Process Measures—measures ofthe step-by-step tasks and events that emerge

during a training exercise.

• Outcome Measures—measures ofthe overall results of a training exercise.

• Measures of Task Proficiency—measures that serve to identify both tasks that

were performed poorly and tasks that were performed well.

• Measures ofKnowledge/Skill Deficiencies-diagnostic measures that serve to

identify the specific knowledge and skill deficiencies that contribute to poor

performance.

22
• Feedback Measures—measures that serve as a source ofthe feedback that is

needed to maximize the training value of AARs.

• Learning Trend Measures—measures that can be used to track learning trends

across training sessions.

The literature review also must encompass the methods, procedures, and equipment that

have been found useful for assessing the performance of collective tasks (e.g., Meliza, Bassemer

and Tan, 1994). Of interest are methods, procedures, and equipment that have been found

effective for observing performance, evaluating performance, recording performance measures,

tabulating performance measurement data, and interpreting performance data.

23
REFERENCES

BeU, H. H., Dwyer, D. J., Love, J. E, Meliza, L. L., Mirabella, A., & Moses, F. L. (1996).

Recommendations for planning and conducting multi-service tactical training with

distributed interactive simulation technology (ARI Draft Technical Report). Alexandria,

VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

Cross, K. D., & Bierbaum, C. R. (1995). Training on the use of digital C4I devices: Lessons

learned from the Focused Dispatch AWE (ASI1058-08-705-95). Fort Walton Beach,

FL: Anacapa Sciences, Inc.

Endsley, M. R. (1995a). Toward a theory of situation awareness in dynamic systems. Human

Factors. 34, 32-64.

Endsley, M. R. (1995b). Measurement of situation awareness in dynamic systems. Human

Factors. 34, 65-84.

Elliott, G. S., Sanders, W. R., & Quinkert, K. A. (1996). Training in a digitized battalion task

force: Lessons learned and implications for future training (ARI Research Report 1695):

U.S. Army Research Institute for the Behavioral and Social Sciences, Armored Forces

Research Unit.

Fallesen, J. J., Michel, R. R., Lussier, J. W„ & Pounds, J. (1996). Practical thinking: Innovation

in battle command instruction (ARI Technical Report 1037). Alexandria, VA: U.S.

Army Research Institute for the Behavioral and Social Sciences.

Garland, D. J., & Endsley, M. R. (Eds.). (1995). Experimental analysis and measurement of

situation awareness. Daytona Beach, FL: Embry-Riddle Aeronautical University Press.

Gilson, R. D., Garland, D. J., & Koonce, J. M. (Eds.). (1994). Situational awareness in complex

systems. Daytona Beach, FL: Embry-Riddle Aeronautical University Press.

Koger, M. E., Long, D. L., Britt, D. G, Sanders, J. J., Broadwater, T. W., & Brewer, J. D.

(1996). Simulation-based mounted brigade training program: History and lessons learned

(ARI Research Report 1689). Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences.

Leedom, D.K. (1990). Aircrew coordination training and evaluation for Army rotary wing

aircrews: Summary ofresearch for fiscal year 1990. Fort Rucker, AL: U.S. Army

Research Institute for the Behavioral and Social Sciences, Aviation Research and

Development Activity.

Meliza, L. L., Bassemer, D.W., & Tan, S. C. (1994). Unit performance assessment system

development (ARI Technical Report 1008). Alexandria, VA: Army Research Institute

for the Behavioral and Social Sciences.

25
Prince, C, & Salas, E. (1993). Training and research for teamwork in the military aircraft. In E.

L. Wiener, B. G. Kanki, & R. L. Helmreich (Eds.), Cockpit resource management, (pp.

337-366). New York: Academic Press.

U.S. Army National Guard Bureau. (1995a). Armv National Guard aviation reconfigurable

mission simulator (ARMS): Draft operational requirements documents. U.S. Army

National Guard Bureau, Acquisition Contracting Division.

U.S. Army National Guard Bureau. (1995b). Armv National Guard aviation simulation and

training strategy. U.S. Army National Guard Bureau, Acquisition Contracting Division.

U.S. Army National Guard Bureau. (1995c). Armv National Guard aviation warfighting

simulation system work plan. U.S. Army National Guard Bureau, Acquisition

Contracting Division.

U.S. Army National Guard Bureau. (1995d). Mission need statement for aviation reconfigurable

manned simulator (ARMS). U.S. Army National Guard Bureau, Acquisition Contracting

Division.

U.S. Army National Guard Bureau. (1996). Armv National Guard aviation warfighting system

(AVWSS). performance specifications: aviation reconfigurable manned simulator

(ARMS) (draft): U.S. Army National Guard Bureau, Acquisition Contracting Division.

Wiener, E. L., Kanki, B. G., & Helmreich, R. L. (Eds.). (1993). Cockpit resource management.

New York: Academic Press, Inc.

Winsch, B. J., Garth, T. H., Ainslie, F. M., & Castleberry, J. D. (1996). The development of

structured simulation-based training for digital forces: Initial battalion staff-level efforts

(ARI Research Product 96-08). Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences.

26
4

APPENDIX A

LIST OF ABBREVIATIONS AND ACRONYMS

A2C2S Army Airborne Command and Control System

AAR After Action Review

AC Active Components

AD Air Defense

ARI Army Research Institute

AMPS Aviation Mission Planning System

ARMS Aviation Reconfigurable Manned Simulator

ARTEP Army Training and Evaluation Program

ASAS All Source Analysis System

ASE Aircraft Survivability Equipment

AVTB Aviation Test Bed

AVCATT Aviation Combined Arms Tactical Trainer

AVTOCs Aviation Tactical Operations Centers

AVWSS Aviation Warfighting System

AWE Advanced Warfighting Experiment

CATS Combined Arms Training Strategy

CRM Cockpit Resource Management

DIS Distributed Interactive Simulation

DTX Digital Training Exercise

FD Focused Dispatch

NOE Nap ofthe Earth

NTC National Training Center

OPFOR Opposing Force

RC Reserve Components

S3 Operations and Training Officer

SME Subject Matter Expert

SOP Standing Operating Procedure

TADSS Training Aids, Devices, Simulators, and Simulations

TOC Tactical Operations Center

USAAVNC U.S. Army Aviation Center

USANGB U. S. Army National Guard Bureau

xo Executive Officer

A-l