Research Product 98-37

A Procedure for Development of Structured

Vignette Training Exercises for Smail Groups

19980729 003

July 1998

Armored Forces Research Unit

U.S. Army Research Institute for the Behavioral and Social Sciences

Approved for public release; distribution is unlimited.

BBC QUALHT MfirtOT» 1
U.S. Army Research Institute

for the Behavioral and Social Sciences

A Directorate of the U.S. Total Army Personnel Command

EDGAR M. JOHNSON

Director

Research accomplished under contract

for the Department of the Army

Human Resources Research Organization

Technical reviewby

LTC Emmet Holley, TSM for Force XXI

Carl Lickteig, ARI

NOTICES

FINAL DISPOSITION: This Research Product may be destroyed when it is no longer

needed. Please do not return it to the U.S. Army Research Institute for the Behavioral

and Social Sciences.

NOTE: This Research Product is not to be construed as an official Department of the

Army position, unless so designated by other authorized documents.
REPORT DOCUMENTATION PAGE

1. REPORT DATE (dd-mm-yy)

July 1998

2. REPORT TYPE

Final

4. TITLE AND SUBTITLE

A Procedure for Development of Structured Vignette

Training Exercises for Small Groups

6. AUTHOR(S)

Charlotte H. Campbell. Laura A. Ford. Rov C. Camobell (HumRRO).

and Kathleen A. Quinkert (ART)

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Human Resources Research Organization (HumRRO) U.S. Army Research Institute for

66 Canal Center Plaza, Suite 400 the Behavioral and Social Sciences

Alexandria, VA 22314 ATTN: TAPC-ARI-K

2423 Morande Street

FortKnox,KY 40121

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue

Alexandria, VA 22333-5600

3. DATES COVERED (from... to)

July 1996 - January 1998

5a CONTRACT OR GRANT NUMBER

DASW01-94-D-0011

5b. PROGRAM ELEMENT NUMBER

0602785-A

5c. PROJECT NUMBER

A791

5d. TASK NUMBER

2228

5e. WORK UNIT NUMBER

R05

8. PERFORMING ORGANIZATION REPORT NUMBER

10. MONITOR ACRONYM

ARI

11. MONITOR REPORT NUMBER

Research Product 98-37

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13 SUPPLEMENTARY NOTES Kathleen A. Quinkert, Contracting Officer's Representative. This report provides guidance

supplemental to Campbell, C. H. & Deter, D. E. (1997). Guide to Development ofStructured Simulation-Based Training

(ARI Research Product 97-14). Alexandria, VA: US Army Research Institute for the Behavioral and Social Sciences.

14. ABSTRACT (Maximum 200 words):

This research product presents a procedure that can be followed to develop small group focused training exercises. It is a

specific application of the methodology contained in the Guide to Development ofStructured Simulation-Based Training,

published by ARI in 1997. Both that guide and this product are based on work performed under the auspices of the Army

Research Institute (ARI) and the Force XXI Training Program. The development effort, entitled Combined Arms Operations

at Brigade Level, Realistically Achieved Through Simulation Ocnown as COBRAS), resulted in construction of training

support packages (TSPs) for large scale exercises and for small group vignettes. This product contains examples and

explanations that specifically address the construction and evaluation ofsmall group structured exercises. It addresses the use

ofsimulation (live, virtual, and constructive) and discusses the advantages and disadvantages of the uses of simulation for

small group exercises.

15. SUBJECT TERMS

COBRAS Training Program

Small-Group Training

Structured Training

Brigade Staff Training

Training Support Package

Training Development Methodology

Formative Evaluation

Vignettes

SECUfiJTYCtASS^CATlONOF :

16. REPORT

Unclassified

17. ABSTRACT

Unclassified

18. THIS PAGE

Unclassified

19. LIMITATION OF

ABSTRACT

Unlimited

20. NUMBER

OF PAGES

70

21. RESPONSIBLE PERSON

(Name and Telephone Number)

Kathleen A. Quinkert

DSN 464-6928
Research Product 98-37

A Procedure for Development of Structured

Vignette Training Exercises for Small Groups

Charlotte H. Campbell, Laura A. Ford, and Roy C. Campbell

Human Resources Research Organization

Kathleen A. Quinkert

U.S. Army Research Institute

Armored Forces Research Unit

Barbara A. Black, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

July 1998

Army Project Number Education and Training Technology

20262785A791

Approved for public release; distribution is unlimited.

in
FOREWORD

Within today's Army, two conflicting forces are at work: decreasing resources for training

and increasing demands for highly trained and proficient personnel. Force reductions and other

cost efficiencies require the Army to reduce its expenditures for high-fidelity institutional and field

training. At the same time, introduction ofmore complex systems and equipment, changes to

doctrine and organization, and a changing geopolitical landscape require that training be more

committed to quality and efficiency than ever before.

To meet these challenges, Congress provided Fiscal Year 1994 research and development

funding for the establishment ofthe Force XXI Training Program (formerly known as the Virtual

Brigade Training Program) at Fort Knox, Kentucky. The intent ofthis program is to explore and

utilize simulation technologies and instructional principles to create structured training programs

that fully leverage available resources in providing efficient, effective training to brigade staffs.

The focus is on preparing the Army ofthe 21* century and ensuring that today's Army is

sufficiently ready to provide the foundation for continuing change and modernization.

The U.S. Army Research Institute for the Behavioral and Social Sciences (ARI), Armored

Forces Research Unit (AFRU) at Fort Knox, the Force XXI Training Program, and the U.S.

Army Armor Center (USAARMC) joined forces to sponsor training research and development

(R&D) for one element ofthe Congressionally-mandated effort: simulation-based training for the

conventional mounted brigade. The R&D work was performed under a project known as the

CombinedArms Operations at Brigade Level, Realistically Achieved Through Simulation

(COBRAS).

This research product presents a procedure for developing structured training exercises for

small groups. The guide incorporates experience and lessons learned during the COBRAS

vignette exercise development work and includes more detailed discussion ofthe particular

characteristics ofstructured vignette training. This guidance is designed for use by training

designers and developers, as well as by training program reviewers, evaluators, and managers.

ZITAM. SIMUTIS

Technical Director
A PROCEDURE FOR DEVELOPMENT OF STRUCTURED VIGNETTE TRAINING

EXERCISES FOR SMALL GROUPS

CONTENTS

Page

PREFACE ix

INTRODUCTION 1

Purpose 1

Definition 1

Contents and Organization ofThis Guide 1

Audience for the Guide 2

BACKGROUND 2

Structured Training Characteristics 3

Expanded Definition of Vignettes 5

Use of Simulation in Vignettes 6

Training Support Package Products 7

Note on the Previous Methodology Reports 8

OVERVIEW OF THE METHODOLOGY FOR DEVELOPMENT OF STRUCTURED

SIMULATION-BASED TRAINING 9

Developmental Activities in the Methodology 9

PROCEDURE FOR DEVELOPMENT OF VIGNETTES 13

PHASE 1. PREPARE INITIAL DESIGN 14

Activity 1.1 Document initial scope decisions 15

Activity 1.2 Select vignette event and participants (tentative) 17

Activity 1.3 Outline vignette presentation model 18

Summary: Phase 1 18

PHASE 2. DESIGN VIGNETTE SCENARIO 21

Activity 2.1 Analyze event context and participation requirements 21

Activity 2.2 Determine vignette scope and specific participants 22

Activity 2.3 Determine vignette inputs and method for delivery 23

Activity 2.4 Describe specifications oftactical products 24

Summary: Phase 2 25

PHASE 3. PREPARE DETAILED TRAINING OBJECTIVES 25

Activity 3.1 Identify performance requirements sources, domain, and standards 26

vn
CONTENTS (continued) .

Page

Activity 3.2 Draft vignette objective and performance requirement statements 27

Activity 3.3. Draft AAR aids 28

Summary: Phase 3

28

PHASE 4. DEVELOP TRAINING SUPPORT PACKAGE 31

Activity 4.1 Prepare training support package components 32

Activity 4.2 Assemble the training support package 41

Activity 4.3 Evaluate the vignette and the training support package 43

Summary: Phase 4 44

LESSONS LEARNED 45

REFERENCES 47

APPENDIX A. ACRONYMS A"1

B. PROCEDURE FOR DEVELOPMENT OF STRUCTURED

SIMULATION-BASED TRAINING B-l

LIST OF FIGURES

Figure 1. The framework ofstructured training 4

2. Components of a training support package for a brigade/battalion battle simulator

(BBS)-based structured training exercise 7

3. The four phases in the methodology for development ofstructured vignettes 10

4. Overview of a procedure for development ofstructured small-group exercises 13

5. A sample concept outline for a COBRAS vignette 19

6. Presentation model for the COBRAS vignettes 20

7. Sample vignette objective and performance requirements statements 27

8. Sample AAR questions and response considerations for a COBRAS vignette 29

9. Sample participant job aid from a COBRAS vignette 34

10. Sample orientation briefing from a COBRAS vignette 35

11. Sample situation briefing from a COBRAS vignette 36

12. Samples ofscenario overview, vignette scope, and vignette limitations and

restrictions from a COBRAS vignette 38

13. Sample listing of TSP materials for COBRAS vignettes 39

vni
PREFACE

This guide is intended to serve as an aid for training developers, training reviewers, training

evaluators, and training managers. It addresses one segment of a broad range oftraining

development projects that all fall under the general heading of "structured training."

The work that led to the codification ofthis methodology was performed under the

sponsorship ofthe U. S. Army Research Institute for the Behavioral and Social Sciences (ART),

Armored Forces Research Unit (AFRU), Fort Knox, on a project entitled CombinedArms

Operations at Brigade Level, Realistically Achieved through Simulation (COBRAS). The

performing contractor consortium included the Human Resources Research Organization

(HumRRO, prime contractor), Hughes Training, Inc. (now Raytheon), BDM International (now

TRW), and PRC Incorporated (now Litton-PRC).

The procedure described in this guide addresses specific training and research challenges.

While the process will generalize to other conditions, it will require the creative developer to

modify the specific products, reviews, and development processes to match the specific

circumstances found in other efforts. This guide will help the developers understand the issues

and challenges they will face and must resolve in their training programs. It will also be of

assistance to reviewers, evaluators, and managers, who will be better equipped to perform their

specific roles by understanding the rationale underlying small-group exercise construction.

IX
A PROCEDURE FOR DEVELOPMENT OF STRUCTURED VIGNETTE

TRAINING EXERCISES FOR SMALL GROUPS

INTRODUCTION

Purpose

This procedure for development ofstructured small-group exercises describes an approach

based on the Systems Approach to Training (SAT) (Department ofthe Army [DA], 1988). The

procedure is a specific application of a more general methodology for development ofstructured

simulation-based training (Campbell & Deter, 1997). Specifically, it addresses:

• how to focus training on key critical objectives and performance considerations,

• how to constrain a small group exercise so that a select group of participants can be

targeted,

• how to "slice" a single event out ofthe larger scenario, and

• how to keep user preparation time and materials to a minimum.

With this guide in hand, training program evaluators and prospective users ofthe process can

understand the basic tenets ofthe procedure and adapt it for their own needs.

Definition

In the development work on which this guide is based, a vignette is defined as follows:

A vignette is a brief, structured, task-based training exercisefor the members ofa small

team.

Contents and Organization of This Guide

The development procedure is presented in four sections:

• The following section, Background, reviews the origins ofthis vignette development

procedure and some ofthe important concepts associated with structured training.

• The second section, Overview, describes the vignette development approach, delineating

its relationship to the SAT (DA, 1988).

• The third section, Procedurefor Development of Vignettes, is a step-by-step presentation

ofthe process. It has the following features:

o Describes the activities to be followed in developing the vignette components, such as

scenarios and trainer materials.

o Details review and formative evaluation requirements.

o Describes the purpose and characteristics of each component ofthe vignette training

support package (TSP).

• The fourth section, Lessons Learned, summarizes key considerations in vignette

development.
Audience for the Guide

In following the process described in the guide, various objectives for various users may be

met:

• Training developers will be confronted with most ofthe critical considerations that must

be faced in designing and developing structured small group training programs.

• Reviewers oftraining program development projects or ofTSPs will find that the

procedure outlines the issues that should be addressed in development and

accommodated in the TSP.

• Evaluators oftraining programs and TSPs will be able to identify the probable causes of

weaknesses in programs by tracing through the critical development steps.

• Managers who oversee the development and implementation ofstructured training

programs will be better equipped to modify the developmental process and elements of

the TSP, through a full understanding ofthe procedure.

The approach outlined in this guide is directed primarily to training developers.

BACKGROUND

During 1993-1995, the U.S. Army Research Institute for the Behavioral and Social Sciences

(ARI) sponsored a research and development effort to construct structured simulation-based

training exercises for multiple echelons ofthe armored force. The finished program is known as

the Virtual Training Program (VTP)1. In addition to the TSP materials, project personnel also

produced a report entitled Methodologyfor Development ofStructured Simulation-Based

Training (Campbell, Campbell, Sanders, Flynn, & Myers, 1995). That methodology is a step-bystep procedure for designing and developing training materials and scenarios2

in simulation

environments3.

1 The program was originally known as the Reserve Component Virtual Training Program (RCVTP). As active

units and schoolhouse training managers became interested in using the program, the "Reserve Component" (RC)

designator was dropped. The work was performed as an ARI project entitled Simulation-Based Training

Multiechelon Training ProgramforArmor Units (S1MUTA), reported in Hoffman, Graves, Koger, Flynn, and

Sever (1995). Additional VTP work was performed in the project Expansion ofthe Virtual Training Program

(SIMUTA-B), reported in Graves and Myers (1997).

2 The methodology itself is not specific to any particular technology, type of unit, unit level, or mission. It is,

however, particularly applicable to the development of collective military training exercises that focus on

command and staffskills in planning and execution.

3

In general, simulations are classified as constructive, virtual or live. "Constructive simulations" are those where

the technology represents actions and consequences, but does not replicate the full physical environment.

"Virtual simulations" are those in which users actually interact with a replication ofthe physical environment, in

much the same way that they would with the real environment. "Live simulation" refers to those situations where

the real environment is reproduced in a less than full mode. Whether or not constructive or virtual simulation

technologies are used, all performance practice training involves some degree of live simulation, in that it

involves real people in less-than-real conditions. In this report, when the term "simulation" is used, it will refer

to constructive and/or virtual (i.e., technology-based) simulation.

2
Subsequent to its use in developing the VTP, the methodology (Campbell et al., 1995) was

used in the construction of a wide variety ofstructured, simulation-based training programs for

different echelons, missions, and simulations. Adaptations were made as necessary to customize

the methodology for each usage. The basic methodology, however, has proven to be robust with

respect to differences in environments and training audience.

One ofthe most recently completed ofthese programs is the Force XXI Training Program:

CombinedArms Operations at Brigade Level, Realistically Achieved Through Simulation,

usually referred to as COBRAS. This ARI project developed training exercises for selected

members of a conventionally equipped, heavy, mounted brigade (bde). The training development

focused on three missions (movement to contact, area defense, and deliberate attack) and on all

phases ofthose missions (plan, prepare, and execute, including consolidation and reorganization).4

One component ofthe COBRAS project is the Brigade StaffVignettes. These vignettes are

a series of 24 independent training exercises designed for brigade staff use. The vignettes provide

concentrated practice on specific skills and behaviors for a small group ofthe brigade staff

training audience, usually with low resource requirements (relative to full field exercises).

Feedback is focused on the interactions within the small group, allowing the exercise to function

as a "bridge" training opportunity between individual and collective training. The experience and

lessons learned in performing this research and development effort are the basis for the procedure

described in this guide.

The vignette work has roots that go deeper than the VTP methodology development and

COBRAS expansion ofthe methodology. Other ARI work (Lickteig, 1991; Lickteig & Emery,

1994) explored the use ofvignettes for training and assessing tactical command and control skills.

These were much shorter drills than the COBRAS vignettes (generally 10 minutes in length) and

focused on leader behaviors in sorting out and reacting to complex message sets. However, the

procedure they describe for development ofthose vignettes is similar to the procedure described

in this guide. Both procedure descriptions have a common overall goal: to stabilize extraneous

variables impacting performance and standardize the important cues, in order to provide

opportunities for performance ofselected tasks and objectives.

Structured Training Characteristics

A comprehensive definition ofstructured training is presented in Figure 1. The key features

ofstructured training are:

• a focus on performance of critical tasks, selected on the basis of a needs analysis,

• standardized exercise control to ensure practice ofthe selected tasks,

• standardized feedback to correct and reinforce performance on the selected tasks, and

• exercise support within available resources by means of comprehensive training materials.

4 The COBRAS exercises and TSPs and the development process are more fully described in the project report:

CombinedArms Operations at Brigade Level, RealisticallyAchieved Through Simulation I (COBRAS I) ProjectDevelopment and Lessons Learned (Graves, Campbell, Deter, & Quinkert, 1997).
As shown in Figure 1, the needs analysis leads into preparation ofthe supporting materials

that comprise the TSP. The TSP provides three kinds oftools:

• the scenario specifications and tactical materials that provide the context for the exercise,

• the support guides that provide the cues to control the trainingprocess (i.e., the

behaviors and activities ofthe trainers in controlling the flow ofthe exercise), and

• the training objectives, observation guides, and associated after action review (AAR)

materials that support the learningprocess (i.e., the activities ofthe training participants

and the means for providing feedback to them).

Vignettes that are produced according to the development procedure described here will

incorporate these features. The procedure also describes considerations in using virtual,

constructive, and/or live simulation capabilities. When used appropriately, simulations can

provide tactically realistic scenarios, causing the unit to be immersed in an environment that elicits

performance ofthe critical tasks.

Needs

Analysis

*

Critical

Tasks

and

Resource

Requirements

TRAINING SUPPORT

PACKAGE (TSP) |

Scenario Specifications

& Tactical Materials

Guides for Roleplayers,

Interactors,

Administrators,

Opposing Forces,

Simulation Control, etc.

Training Objectives,

Observation Guides,

andAAR Materials

TRAINING PROCESS:

Cues from simulation,

scripted messages, and

roleplayers

H

LEARNING PROCESS:

Monitored performance

and feedback on critical

tasks

After Action Reviews

V (AARs)

Figure 1. The framework ofstructured training.

The training focus in structured training is maintained by means of careful attention to

standardized implementation. That standardization is achieved in adherence to the following

guidelines:
Conditions that provide the setting for the training should be precisely defined and crafted

to be realistic for task performance.

Selected tasks that form the training objectives should be doctrinally correct. Whether or

not they are drawn from published documentation, there should be acceptance ofthe

technical accuracy ofthe tasks and how they must be performed.

Training should take advantage of and work within the capabilities ofthe selected

simulator/simulation.

Training materials should allow the unit to focus on participating in the training.

Participants should not spend time on designing and developing the training, nor on

observing and evaluating their own performance.

Materials should permit standardized (i.e., replicable) implementation ofthe program, so

that the selected tasks are cued consistently.

Training should make use of performance observers, and materials should direct their

attention to the key training objectives.

Training should be designed to fit within the typical unit's time and resource constraints,

so that local modifications are minimal.

Training materials should be flexible, so that different situations can be accommodated,

but should also include warnings and explanations about what should not be modified.

Expanded Definition of Vignettes

Within the overall definition ofstructured training exercises, a more restricted definition

applies to vignettes. In contrast to most structured exercises, vignettes are generally more limited

in audience, in scope, and in resources required. As stated earlier, a vignette is defined as follows:

A vignette is a brief, structured, task-based training exercisefor the members ofa small

team.

A vignette has the following characteristics:

• Focused on the performance of a small group ofselected individuals working together

toward a common goal or outcome (i.e., a team)

• Focused on practicing the performance of a single well-defined task that is part of a

larger situation

• Designed for execution under limited resource conditions ~

• Exportable (with or without dedicated observer/controller [O/C] team)

• Minimal user preparation time (2-8 hours) and execution time (4-8 hours)

• Little or no equipment requirement, except simulation.

COBRAS vignettes were developed for a military context, specifically, the staff of a

combined arms brigade. The procedure may be translated for other uses, either military or nonmilitary. However, because that translation and subsequent application have not yet been
performed, the procedure and examples in this guide will be drawn from the COBRAS work

accomplished to date.

Use of Simulation in Vignettes

Because ofthe emphasis on limited resource expenditures in vignettes, the use ofsimulation

is often contraindicated. Simulation has the potential for providing significant training benefit.

There are at least six ways in which simulation could enhance training, including:

1. Efficiently replicate battlefield activity, negating the need for large "live" support

requirements such as terrain, equipment, and soldiers.

2. Automatically generate a continuous stream of cues for training audience, as opposed to

trying to manually present cues.

3. Provide reasonable outcomes to battlefield actions after the appropriate period oftime

has elapsed.

4. Provide standardized training conditions.

5. Provide instructional features that enhance efficiency and effectiveness, including

performance measurement.

6. Provide replay capabilities showing battlefield effects to support objective feedback.

However, ifthe simulation does not significantly achieve one or more ofthese advantages,

then it is not an enhancement to training. For vignettes, COBRAS developers found that

simulations were often not ofsufficient benefit to justify the cost. Specifically:

• Live (field) training for a group and event is already achievable for a low cost when the

training participants are located primarily in command posts (CPs). The expense comes

in placing weapon systems and units on the ground (something that simulation does well).

For events that do not require such resources, simulation would likely drive training costs

up.

• Ifthe vignette concerns a planning activity, the essential cues can be provided from

printed material or radio communications, rather than requiring simulation.

• The short time duration ofvignettes is usually insufficient to allow the simulation to

portray outcomes ofdecisions and actions.

• For vignettes that focus on group interactions and synchronization, replay capabilities

that show battlefield effects are too remotely linked to the group work (although they

may be appropriate for evaluating tactical decision-making).

Thus the cost-benefit considerations lead to the conclusion that simulation is a significant

contributor to training value only if:

• The vignette requires representation ofweapon systems or troops to provide essential

tactical realism, dynamically cue the training audience, and react to the training audience;

and

• The vignette can be extended long enough to allow the situation to develop so that

battlefield effects can be seen.
The alternative to using technology-based simulations is to use "live" simulation. For most

ofthe COBRAS vignettes, this was relatively easily accomplished. The training participants,

normally located in a CP, perform the vignette in any room large enough to work in.

Communications are represented by means ofwritten messages or by having the training monitor

simply provide any necessary information. By using this type ofsimulated environment, resource

costs are kept low, and the vignettes are feasible as unit-administered exercises5.

Training Support Package Products

The products of a structured training development project comprise the TSP. For the ARIsponsored projects and as defined in Training and Doctrine Command (TRADOC) Regulation

350-70 (DA, 1995), the TSP structure is shown in Figure 2. The components are:

Tactical materials: ' —

• Operations orders (OPORDs) and other mission-specific materials

• Prepared messages and scripted materials

• Descriptions of personnel and equipment status

Unit preparation materials:

• Guidance on exercise selection

• Guidance on unit preparation

• Description of time, personnel, and facility requirements

.mmm^, ^^^. Annex C, OPORD >0<-nnU

i Guidance for otherparticipants:

• Observer materials on task performance and feedback sessions

• Instructions for operation ofthe simulation components

Guidance for roleplayers

Administrative guidance for managers:

• Instructions for setting up the training

ffvLvfr- r

Instructions for controlling scenario events

Simulation tapes and documentation,

• Initialize the system

• Provide starting conditions

• Show graphic graphic control measures measures MTC PIS V.£M.

®^S$$gg&B*.

Figure 2. Components of a training support package for a brigade/battalion battle simulator

(BBS)-based structured training exercise.

5 Both the original methodology guide (Campbell et al., 1995) and the expanded methodology (Campbell & Deter,

1997) contain detailed guidance on considerations for virtual and constructive simulation-based training. The

guidance is also appropriate for simulation-based vignettes. Interested users should refer to those other reports for

additional information concerning development ofTSPs for simulation-based exercises.
• Tactical materials. Such materials may include operations orders (OPORDs) and other

mission-specific materials to be used in unit preparation and rehearsals, prepared

messages and scripted materials that are used during the exercise to cue tasks, and

descriptions of personnel and equipment status to set the scenario conditions.

• Preparation materialsfor the trainingparticipants. These assist unit leaders in selecting

the appropriate training for their needs, provide guidance on unit preparation, and

describe the time, personnel, and facility requirements.

• Guidancefor other participants. These guides represent the "train the trainer" materials.

They include observer materials that direct attention to specific task performance,

describe acceptable performance, and outline the feedback sessions. For simulationbased vignettes, there will also be instructions for operation ofthe simulation components

and guidance for roleplayers who interact with the unit during the exercise.

• Administrative guidancefor managers ofthe training. These include instructions and

suggestions for setting up the training and controlling scenario events for the individual

designated to implement the exercise (usually referred to as the "Training Coordinator").

• Simulationfiles and documentation. Simulation files, on disk or tape, are used with the

selected constructive or virtual simulation to initialize the system, provide starting

conditions (e.g., locations, task organization), and show graphic control measures. The

documentation is provided as back-up in case the initialization tapes are unusable.

This structure for the components of a TSP allows developers considerable latitude in

organizing and presenting their materials. Such flexibility is essential, because every structured

training program will require an individually prepared TSP, specific to the requirements ofthat

program.

With vignettes in particular, such flexibility is key to attaining low-resource training

opportunities. One TSP characteristic that allows vignettes to focus on limited audience and

events is that the TSP will contain all ofthe essential situational information up to the starting

point ofthe exercise. All ofthe necessary cues and conditions will be included, and unnecessary

information is carefully screened out ofthe TSP materials. While this presentation may

sometimes be less than fully realistic, it is the necessary compromise that permits the vignette to

occur with whatever realism is necessary and sufficient.

Note on the Previous Methodology Reports

The most recent version ofthe methodology for development ofstructured simulation-based

training (Campbell & Deter, 1997) contains expanded explanations ofTSPs and formative

evaluation activities. The Appendix to that guide is a checklist that can be used to review

developing or completed TSPs. The report on the revised methodology (Campbell, Deter, &

Quinkert, 1997) provides additional details on the history, purposes, and uses ofthe methodology.
OVERVIEW OF THE METHODOLOGY FOR DEVELOPMENT OF STRUCTURED

SIMULATION-BASED TRAINING

The methodology for developing structured training exercises originated in the SAT model

(DA 1988). The SAT process comprises five phases:

• Analysis—Determine the training requirement, training audience, and appropriate training

media.

• Design—Prepare the specifications ofthe training components.

• Development—Construct and refine the various training package components.

• Implementation—Deliver the training in an operational setting.

• Evaluation—Assess success in achieving the program's training goals.

The development procedure described here parallels the SAT process in many respects.

However, the SAT activities phases have been narrowed in this vignette development procedure

to focus on the development ofstructured training for small groups and focused situations. This

procedure addresses analysis only briefly, and concentrates primarily on design and development.

For vignette development, the activities include:

• Analysis—initial decisions and selection oftraining objectives.

• Design—specification ofthe exercise scope and structure.

• Development—construction of TSP components, tryouts, and formative evaluation.

Developmental Activities in the Methodology

There are four phases to the basic development methodology, as shown in Figure 3

6

:

• PHASE 1. PREPARE INITIAL DESIGN7—Determine the design specifications,

including the user, situational context, appropriate training environment, specific

audience and event, and vignette presentation model.

. PHASE 2. DESIGN VIGNETTE SCENARIO—Determine the limits ofthe vignette in

terms ofmission, enemy, troops, terrain, time available, and communications (METTTC); generate the tactical framework for the vignette; specify the scope ofthe vignette

event; define and begin development ofsimulation or scripted materials.

6 The wording ofthe phases and activities in the vignette methodology presented here varies slightly from the

original (Campbell et al., 1995) and expanded (Campbell, Deter, & Quinkert, 1997) versions, in order to make it

more specifically suitable for vignette development.

7 While a rigorous and thorough needs analysis is the foundation for design and development, it is frequently the

case that the training requirement is not defined as a part ofthe same effort that performs design and

development. This guide demands that the developer know what the training requirements are, whether or not

they were identified within the design and development work. For more guidance on performing training needs

analysis see, for example, Muchinsky (1987), Goldstein (1980), and TRADOC Regulation 350-70 (DA, 1995).
PHASE 3. PREPARE DETAILED TRAINING OBJECTIVES—Focus the training on

critical performance requirements and standards for the selected audience and event.

PHASE 4. DEVELOP TRAINING SUPPORT PACKAGE—Construct and try out all

ofthe components ofthe training program, including materials for the trainers and for the

participants and, as appropriate, the simulator/simulation-based components.

PHASE 1:

Prepare Initial Design

Decisions on target audience, training

context, simulation technology

M

PHASE 2:

Design Vignette Scenario

Description of the tactical event(s) that

provide the context for the vignette

Jl

PHASE 3:

Prepare Detailed Training

Objectives

Prepare the statements of behavioral

objectives and develop feedback aids

M

PHASE 4:

Develop Training Support

Package

Construct and evaluate the TSP

components

FORMATIVE EVALUATION: Content reviews, map/simulation

exercises, pilot tests, trial implementations

Figure 3. The four phases in the methodology for development ofstructured vignettes.

These four phases are not done in a strictly linear fashion. There will be considerable overlap

among the phases. For example, understanding ofthe initial decisions will change as more is

learned about the training environment and the performance requirements; information that feeds

into development ofthe TSP will be captured during documentation ofinitial decisions; and so

on.

The SAT implementation and evaluation phases are not replicated in the development

methodology as separate and distinct phases. Both operational implementation and the

summative evaluation take place after the activities covered by the methodology, when the

training package is received by the user. Therefore, they are not covered in this guide.

However, the procedure does incorporate a series ofrequired and criticalformative

evaluation steps throughout the development process. Some ofthese are formal activities (such

10
as roleplay enactments ofthe vignette conducted by the design team, tryouts with representative

individuals, technology-driven tryouts, or content reviews with experts and stakeholders or

proponents8). Formative evaluation also includes a continuous attention to the need for revisions

and improvements. The purpose offormative evaluation, both the formal activities and the

continuous revisions, is to insure the quality ofthe product by attending to quality throughout

development. In Figure 3, formative evaluation is depicted as underlying all ofthe design and

development work. This continuous examine-evaluate-refine process, which derives from the

formative evaluation attention, is one reason for the non-linear aspect of design and development.

It is based in part on the procedure presented in Herman, Morris, and Fitz-Gibbon (1987).

This guide presents the development activities specifically for producing small group, focused

event training situations. It should be noted that the design and development effort for one

vignette is as intensive and difficult as is the effort for a much larger exercise. Developers must be

as careful of consistency and doctrinal correctness, functionality of TSP materials, and training

utility and benefits for an exercise for a small group as they would be for large-group integrated

exercises.

Detailed guidance regarding the application ofthe methodology is contained in the next

section ofthis report, Procedurefor Development of Vignettes. Each phase ofthe procedure is

introduced by a discussion ofthe purpose ofthat phase, along with a list ofthe activities in the

phase. Each activity is then presented with a lengthy explication ofthe considerations for that

activity and examples drawn from the COBRAS work.

The activities should, in general, be performed in the order described. However, the process

is not meant to be rigid. The continuous attention to formative evaluation during an activity will

often cause the developer to go back and revise the products of an earlier activity. In fact, the

entire process should be regarded as flexible, and developers should be prepared to move back

and forth between activities during the development, revising decisions and products as necessary.

Whenever decisions orproducts are revised, the developer must trace back through earlier

development and correct allrelatedproducts, whether they are interim products or parts ofthe

final TSP. We cannot emphasize too strongly the importance ofkeeping all products current and

in agreement with each other. We should also point out that, in our experience, this is one ofthe

most difficult challenges in development. Contrary to expectation, the amount oftactical and

other TSP materials required to support a vignette may well be more than what is required for a

full large-group mission-based exercise.

Although the procedure section presents a succinct procedure for developing vignette

exercises, it is not in itselfsufficient for development. Subject matter expertise for associated

military aspects (e.g., the mission type, unit type, operations, and enemy tactics), training and

instructional design expertise, and expertise on the selected simulation technology (as

appropriate) will be required throughout the development and review processes.

8 By "proponent" we mean whoever decided that the training was required. This person or agency is usually the

primary client for the development effort.

11
The Appendix to this report contains an outline ofthe methodology which will serve as a job

aid or checklist for the development process. It includes a listing ofthe phases, activities, and

formative evaluation considerations in the methodology. As appropriate, some ofthe most

important design and development considerations are also listed. For each ofthe activities, the

outline indicates the activity's product or outcome. Not all ofthe indicated products are used in

the final TSP; some are interim products that assist developers in moving through the

development process.

12
PROCEDURE FOR DEVELOPMENT OF VIGNETTES

This section details the activities and considerations in the development ofstructured smallgroup exercises, or vignettes. Figure 4 outlines the process, showing the four phases, the

activities, and the role ofthe formative evaluation. This section also contains examples ofthe

development process, drawn from work on the COBRAS vignettes. The procedure is addressed

most directly to developers, who are referred to as "you" in this section.

PHASE 1:

Prepare Initial Design

PHASE 2:

Design Vignette

Scenario

PHASE 3:

Prepare Detailed

Training Objectives

PHASE 4:

Develop Training

Support Package

DE

DE

TE

Activity 1.1 Document initial scope decisions.

Activity 1.2 Select vignette event and participants.

Activity 1.3 Outline vignette presentation model.

Evaluation: Internal reviews, proponent reviews.

Activity 2.1

Activity 2.2

Activity 2.3

Activity 2.4

Evaluation:

Analyze event context and participation requirements.

Determine vignette scope and specific participants.

Determine vignette inputs and method for delivery.

Describe specifications of tactical products.

Proponent reviews; subject matter expert (SME)

reviews and walkthroughs.

Activity 3.1 Identify performance requirements sources, domain,

and standards.

Activity 3.2 Draft vignette objective and performance

requirement statements.

Activity 3.3 Draft AAR aids.

Evaluation: Internal reviews, proponent reviews.

Activity 4.1 Prepare training support package components.

Activity 4.2 Assemble the training support package.

Activity 4.3 Evaluate the vignette and the training support

package.

Evaluation: Proponent review, tryouts (pilot tests and trials),

expert reviews.

Figure 4. Overview of a procedure for development ofstructured small-group exercises.

13
PHASE 1. PREPARE INITIAL DESIGN

Activity 1.1 Document initial scope decisions.

Activity 1.2 Select vignette event and participants (tentative).

Activity 1.3 Outline vignette presentation model.

Formative Evaluation: Internal reviews, proponent reviews.

In the first phase of development, you will make or identify certain decisions about the

vignette that is to be developed, and then document those decisions. The input for this phase may

be external to your vignette development efforts, in that the requirement to develop vignettes for

a particular type ofunit, mission, and training environment may be specified for you by the

proponent. In other cases, the proponent will rely on you, the training developer, to identify

alternatives and make recommendations. In either event, the front end analysis should tell you in

general terms what the training need is (the event, the participants, and the performance

requirements).

Whether you are performing the initial needs analysis or simply recording the results of an

already-completed analysis, it is crucial that those initial decisions be preserved. They form the

foundation for the work, and will help to guide subsequent decisions. In vignette development,

Phase 1 has three activities: to document the initial decisions, make tentative decisions about the

vignette event and participants, and outline the vignette presentation model.

Formative Evaluation

At the early part ofthis phase, concentrate on internal reviews to assess the wisdom ofthe

decisions. This means having a group of other subject matter experts (SMEs) and instructional

developers review your decisions to verify that you have captured the event correctly, that you

have designated the right participants, and that the vignette is worth doing (has training value).

When all three activities in this phase are completed, you should plan on a briefing to the training

proponent. In that briefing, your goal will be to get approval ofthe decisions made, or input on

any required changes. The briefing should present those decisions and the reasons, as well as any

outstanding issues to be decided. This will help to ensure that your design and development

efforts are in accordance with the intentions ofthe proponent.

You should ensure that both the proponent and the design and development team have

thought about the implications ofthe decisions and are clear on what the decisions are. The initial

design phase can be an exciting time, with seemingly boundless opportunities for innovative

approaches and ambitious (even grandiose) plans. The challenge will not be in finding valuable

training needs that can be addressed. It will be in excluding activities so that the training can

focus on specific performance objectives. Try to anticipate the efforts that will be required to

carry out those plans, and be willing to modify those plans ifthey turn out to be unrealistic in

execution.

14
Activity 1.1 Document initial scope decisions

Product/Outcome: Documentation of initial decisions on vignette scope.

The decisions that you make, examine, and record will serve as the basis for all ofthe other

activities in developing the vignette. These decisions should arise out of a needs assessment,

performed by you or the proponent before the remaining design work begins.

For vignette development, there are at least three areas of decisions:

1. Training Audience: The "training audience" is the personnel for whom the training is to

be designed. Although the vignette itself will focus on a small group, the purpose ofthis

activity is to identify the larger group. In some cases, this will be the entire population at

the selected level (e.g., armor platoon, scout team). This may seem too obvious to be

noted, but it should be documented, to ensure that all parties understand the training

focus. In Activity 1.2, you will make a tentative selection ofthe specific training

audience for a specific vignette (referred to as the "vignette participants") from this larger

group.

2. Scenario Context: The scenario context is the tactical situation that will provide the

"storyline" in which the vignette is set. The scenario context for military training

programs in tactical settings will likely include the mission type, enemy type, terrain, and

unit type.

During this activity, you will make some initial broad decisions about the vignette

scenario context. In Activity 1.2, you will carve out a specific event or set of activities,

from the training context which will serve as the basis for the vignette itself. You will

then continue to examine the context as the vignette is refined, in order to be certain that

all necessary specifications have been addressed, decided, approved, and documented.

Some ofthe context considerations include:

• Mission Type. The mission provides the context for the scenario. The mission type

will affect much ofwhat follows, and should be specified early in the process.

Examples ofmission types include attack, defend, and break out from encirclement.

• Enemy Type. The enemy must organize and fight according to defined rules and

tactics, such as are found in the TRADOC models.

• Terrain. The terrain setting that the exercise uses should be indicated in terms ofthe

general area, such as the National Training Center (NTC), or Fort Knox, or Korea.

Later you can get more specific about the precise locations where events will occur, if

necessary.

• Unit Type. It has already been stated that the training audience should be specified.

Make sure that you consider any additional defining factors. For example, ifthe

training is designed for "brigade staffs," you will want to specify the type ofbrigade

(e.g., conventional or digitized, armor or mech or light, 3-battalion or enhanced 4-

battalion).

15
• Appropriate Training Environment. Appropriate training environments may

include classrooms, rehearsal sites (indoors or outdoors), CPs or tactical operations

centers (TOC), or a simulation or simulator environment. Because ofthe high

resource costs for the use ofsimulation, vignettes frequently use "live" simulation

rather than constructive or virtual simulation.9For example, ifthe training

participants are normally located in a CP for the event, they can perform the vignette

in any room large enough to work in, representing the CP. Communications may be

represented by means ofwritten messages or by having the training monitor simply

provide any necessary information. By using this type ofsimulated environment,

resource costs are kept low, and the vignettes become more accessible as unitadministered exercises.

Note: Because oftheir limited scope, many vignettes will not require that a particular

mission, enemy, terrain, or unit type be completely specified as a part ofthe essential

vignette conditions. In particular, vignettes that address certain aspects of planning

processes or non-battlefield activities (such as those that occur in assembly or rear

areas) may not require such a fully developed METT-TC. But some contextual

description will always be required in the vignette introductory material, just to help

the participants get their minds into the situation.

Other Considerations: Depending on the particular development project, there may be

other considerations. These may include vignette time limits, linkages to other vignettes,

availability oftrainer resources, and so on.

• Vignette time limits. Every training implementation will have to be conducted

within certain constraints; time constraints may be the most immediate and apparent.

You need to establish a realistic target for vignette duration, so that the intended

users can use the TSP. This estimate of available training time allows you to

determine the scope ofthe event that can be addressed in the vignette. In COBRAS

vignette work, the limit for live simulation vignettes has been 4 hours. That 4 hours

includes an initial briefing, performance ofthe vignette event, and an AAR.

Technology-based simulation vignettes, using constructive or virtual simulation will

usually run closer to a full day.

• Linkages to other vignettes. There may be a requirement to link several vignettes

by having them share the same tactical METT-TC or making them represent different

aspects ofthe same mission (e.g., fire support, logistics). In other cases, several

vignettes may be under development simultaneously, with the intent that they will, as

a set, provide some complete coverage of a larger training audience. Yet another

situation may be where each vignette covers a limited segment of a given scenario

storyline, and each vignette leads to the next.

9 Additional guidance on development ofsimulation based exercises is presented in Campbell et al. (1995);

Campbell & Deter (1997); and Koger et al. (in preparation).

16
Once any such linkages are identified, the development efforts should be planned and

constructed in such a way as to ensure that all ofthe related efforts can satisfy their

training goals while satisfying the requirement of commonality. In the case where

vignettes lead from one to the next, be very careful about using products of one

vignette as inputs for another vignette. The Training Coordinator should be the one

with the responsibility to ensure that the products generated by the training

participants are good enough to be used as inputs for another vignette. Therefore

you, as the developer, have the responsibility to describe the product standards clearly

enough that the Training Coordinator can judge the quality ofthe products without

any question.

Trainer and Other Resources. Know what training resources you are expected to

plan for. For example:

You may be able to plan on the use of a dedicated training cadre or O/C team, or you

may need to design for a Training Coordinator (who manages and facilitates the

training) from within the unit's resources.

The training program may be destined for a single implementation site, or you may be

developing a completely exportable package.

Activity 1.2 Select vignette event and participants (tentative)

Product/Outcome: Initial description of the vignette event and participants.

This activity sounds fairly simple, but together with the other Phase 1 activities, it lays the

groundwork for development. Here you should state, at least tentatively, the event and the

participants that will be the focus ofthe vignette. The two are completely interdependent, and

must be drawn from the decisions documented in Activity 1.1.

An event is a group activity or task -it is the subject ofthe vignette. Usually, it is easy to

make an event big in scope. The difficult task is to make it small enough for an easily

administered vignette exercise. Three considerations in limiting the event scope are performance

requirements, time, and audience.

.• Performance requirements: Avoid long, global tasks that are performed continuously

in the background (e.g., Maintain situational awareness). Make sure that the

performance requirements are specific and can be performed under some specified

conditions so that it is an event rather than a state ofmind.

• Time: The event should be a limited set of activities that can be performed in a limited

period oftime by a limited number of people under some specified set of conditions.

Remember that there is a time limit for the vignette; for COBRAS work, this was usually

4 hours. Allowing 15 minutes for the start-up activities, 15 minutes for AAR

preparation, and at least an hour for the AAR itself, the vignette execution could not

exceed 2

l

A hours. You may specify an event that would take longer, if you are willing to

"compress" time by allowing the participants to omit certain activities.

17
• Participants: You must also limit the number of participants, that is, the specific

training audience members for this particular vignette. The participants are drawn from

the larger training audience group defined in Activity 1.1. Because vignettes are intended

to be focused, the Training Coordinator will have a difficult time focusing the attention of

a large group on a limited event. Some vignettes may seem trivial in the larger scope of a

mission or a full staff or unit. But you should consider how the vignette activities will

appear for those few people designated as the participants. Make sure that the activities

and the event are significant and nontrivial for them.

In the COBRAS work, the initial work on a vignette was captured by means of concept

outlines. An example is shown in Figure 5.

Activity 1.3 Outline vignette presentation model

Product/Outcome: Description of the implementation design for the vignette.

By "presentation model," we mean a description of how the vignette will be presented, what

the components ofthe TSP will likely be, who will be involved in implementation, and so on.

This outline is essentially a summary ofthe initial decisions described in Activity 1.1, with the

addition of a tentative statement ofthe event and participants from Activity 1.2. An example

from the COBRAS work is show in Figure 6. As you go on, you may need to revisit the

decisions that have been made. Inconsistencies between and among vignette components may

emerge and require resolution.

Summary: Phase 1

In the development methodology, these first three activities and their products serve as input

to all other activities. The decisions on training audience and training objectives that came out of

the needs analysis are central to virtually every activity in the remainder ofthe development

process. They form the basic structure for the exercises, serving as the foundation for your

development objectives. For that reason, it is essential that you make sure that the decisions from

all three activities are clearly spelled out and available to and understood by the design and

development team.

Don't underestimate the value ofreviews by proponents, other members ofthe development

team, and other external training developers. Every bit of expert input can help you prepare a

more solid foundation for your subsequent efforts.

18
...., .MI i j. ,., ':. , ,'.'' .„-'"...,.,,,,,,.V"""''"'''''''''"''''

1. Event

2. Participants

3. References

4. Simulation or

Facility

Requirement

5. Preceding

Situation

6. Start Point

7. Vignette

Activities

8. End Point

9. Limitations

10. Materials

11. Objective

12. Performance

Requirements

Plan brigade combat service support rehearsal.

S1, S4, Forward Support Battalion (FSB) commander. Consider S2?

S3?

. FM 71-3, chapter 3, section V, "Brigade Rehearsals."

. FM 71-123, chapter 2, section II, "Rehearsals."

. FM 71-123, chapter 8, "Combat Service Support."

• FM 12-6, Personnel Doctrine.

• FM 63-20, Forward Support Battalion.

• TC 25-10, Appendix G, "Conduct Rehearsals."

Live simulation only, need a planning room. Planning for Combat

Service Support (CSS) rehearsal, not actually conducting.

The brigade is preparing to conduct a deliberate attack against a

Motorized Rifle Regiment (MRR) defending out of contact. Brigade

occupies an assembly area. Planning is completed, order published.

Plan for conduct of combined arms rehearsal (terrain model) near the

main CP also completed.

Commander's guidance for CSS rehearsal.

Plan the CSS rehearsal.

CSS rehearsal plan. Planning worksheet to be provided?

Participants should not change the brigade plan or concept of support.

Will not need to develop CSS execution matrix or FSB synch matrix.

Will not be able to interact with other brigade, battalion (bn)/task force

(TF), or subordinate units.

Parts of division OPORD (para 4, Appendix 1 & 3 to Annex Q). Basic

brigade OPORD, Annex A, synch matrix, Annex Q with Appendixes 1

and 4. Commander's situation report (SITREP). Brigade decision

support template, service support overlay.

Job aid: Rehearsal planning worksheet (optional?)

Prepare CSS rehearsal plan, specifying key activities and participants.

• Select CSS activities for the rehearsal

• Specify participants and responsibilities

• Determine type and technique

• Select time and site

• Develop rehearsal agenda

Figure 5. A sample concept outline for a COBRAS vignette.

19
WMffi&M&M

• . PRESENTATION MODEL

COMPONENT

Unit leader decides to conduct

vignette training

Training Coordinator named and

provided with the vignette

training management guide

Vignette time and facility

scheduled, participants notified

Training Coordinator prepared

materials

Training Coordinator conducts

initial orientation briefing and

distributes readahead materials

to participants

INFORMATION NEEDED

List of available vignettes

Description of Training

Coordinator duties

Vignette description and

participant list

Preparation instructions

Initial orientation briefing

Readahead materials

Participants prepare for vignette Preparation guidance;

conduct readahead materials

Simulation-based: simulation

files loaded, roleplayers and

interactors trained

Training Coordinator conducts

situation brief and distributes

start-up materials

Vignette Conduct: Training

Coordinator observes and

guides

Training Coordinator conducts

after action review

Simulation files and

instructions for use

Training package for

roleplayers and interactors

Situation brief, start-up

materials

Training objectives and

observation guidance

Training objectives and

observation guidance, AAR

guidance, sample products

LOCATION IN TSP

Vignette Management

Guide

Vignette Management

Guide

Training Coordinator Guide

for the selected vignette

Training Coordinator Guide

for the selected vignette

Training Coordinator Guide

for the selected vignette

Participant Guide with

readahead materials for the

selected vignette

Participant Guide with

readahead materials for the

selected vignette

Simulation files and

Support Coordinator Guide

Support Coordinator Guide,

Roleplayer/lnteractor Guide

Training Coordinator Guide

with start-up materials

package for the selected

vignette

Training Coordinator Guide

for the selected vignette

Training Coordinator Guide

with sample products for

the selected vignette

Figure 6. Presentation model for the COBRAS vignettes.

20
PHASE 2. DESIGN VIGNETTE SCENARIO

Activity 2.1 Analyze event context and participation requirements.

Activity 2.2 Determine vignette scope and specific participants.

Activity 2.3 Determine vignette inputs and method for delivery.

Activity 2.4 Describe specifications of tactical products.

Formative Evaluation: Proponent reviews; SME reviews and walkthroughs.

During this phase, you will plan and outline the tactical scenario that will provide the specific

conditions for the vignette and refine decisions about the participants in the vignette. This is a

lengthy phase, comprising four activities. In addition to producing the structure ofthe vignette

scenario, you will also begin to specify the statements ofthe vignette training objective and

performance requirements, work you will continue in Phase 3.

Formative Evaluation

For all ofthe activities in this phase, the formative evaluation activities are primarily internal,

with continual information updates to the proponents. Internal checks and walkthroughs,

conducted among vignette developers and subject matter experts, will help to ensure that the

evolving vignette concept is ofthe appropriate scope and that your decisions about participants,

inputs, and products, are correct, internally consistent, and acceptable.

Activity 2.1 Analyze event context and participation requirements

Product/Outcome:

• Outline of activities before, during, and after the vignette event.

• Description of roles of all persons involved before, during, and after the vignette

event.

The process of analyzing the tentatively-selected event is referred to as "event analysis."

During an event analysis, you will walk through the extended event (i.e., the event itself and the

activities immediately preceding and following the event), noting participants, conditions and

cues, actions, and products. The purpose ofthis event analysis is to ensure that you have

captured all ofthe necessary information about real-world occurrence ofthe event, concerning:

• WHO are the key participants,

• WHERE would they be during the event,

21
• WHAT do they use as input for activities preceding the target event and for the event

itself,

• HOW do they use the inputs,

• WHAT do they produce, and

• WHY are they doing this, what led to this event.

To "walk through" is to use subject matter experts to perform the process and write down all

ofthe details for those key information needs listed above. By walking through the event and the

preceding and following events, and documenting the information on participants, conditions,

inputs, actions, and products, you will have a complete list of everything that could go into the

vignette to fully account for the event.

Of course, ifyou were to include all ofthat in the vignette, it would no longer be a focused,

small group exercise. You will have to draw the line somewhere. Activity 2.2 discusses how to

"draw the line."

Activity 2.2 Determine vignette scope and specific participants

Product/Outcome:

• Description of cues and products (inputs and outputs).

• Designation of target participants and their actions

• Specification of event beginning and ending points.

In this activity, you must draw the line between what is a part ofthe vignette, and what is

not. Decide what actions, participants, and products ofthe extended event will be in the vignette,

and what should stay out, to be represented as scripted or "given" or notional information or

activity. Look carefully at the participants and the actions to make sure that they are

synchronized: The person(s) involved in the selected actions should be either included as vignette

participants, or explicitly excluded and their roles be scripted. Don't exclude anyone who is a key

information-provider in the event.

By the same token, don't include anyone who does not have an important function to fill

within the scope ofthe event. Certain individuals may usually be involved in particular actions,

but that does not necessarily mean that they should be participants in the vignette. You have

already decided to focus on some particular interaction, using the event as the venue for practice

ofthe interaction. Rather than including all ofthe possible participants, your goal should be to

focus on a selected set ofparticipants. Be especially wary ifthe selected set seems to be growing:

A group of 3-7 vignette participants is about as much as can be accommodated in most short

exercises.

This requires a great amount of consideration and care. For example, you may be trying to

design a vignette for training interactions among four key persons, but the event doesn't seem to

require that all four ofthem be participants. You have two choices: Keep the event as it is and

22
revise the vignette participant list, or retain the vignette participant list and modify the event so as

to make them all be full participants.

During the event analysis (Activity 2.1) you kept a record ofwhat everyone was doing before

and during the event. Now consider that record of activity, and note what the selected vignette

participants will do during the vignette. This may not be identical to a list of everything that

happened during the time period. The vignette scope is deliberately focused on the event, and

although other things might happen during that time period, they might not be relevant or

necessary for the vignette.

Finally, compose a draft description of how the vignette begins, what happens during the

vignette, and how it ends. For example,

• Begin: The initial mission analysis is complete, the commander has given his intent and

guidance, and the staffis ready to begin a course of action development process.

• During: They will develop three courses of action, based on the statement ofthe

commander's intent.

• End: The process will be completed when the staffis ready to briefthe commander on

the three courses of action.

Just as limiting the number ofvignette participants is a way ofmaintaining the focus, so is

limiting the time. Ifyou find that the designated event for the vignette is likely to require more

than 2 or 3 hours, you may be losing the focus. Recall that vignettes are intended to be short and

easily implemented. With unit demands what they are, a small group half-day exercise is

manageable.

Activity 2.3 Determine vignette inputs and method for delivery

Product/Outcome:

• Description of vignette tactical inputs and inputs from nonparticipants.

• Description of how the inputs will be provided.

Having drawn the line to select the event actions and vignette participants that will be

included in the vignette, you can determine how to ensure that those participants will be prompted

to perform. Look at what causes the activities. For example, it may be commander's guidance, a

particular next step in a process, or some enemy activity. Decide how to represent that cause in

the vignette: a written or verbal delivery ofthe specific commander's guidance, a summary ofthe

steps and products leading to this step, an intelligence report detailing the enemy actions, some

simulation-based Opposing Force (OPFOR) actions, and so on.

Look also at what other information the vignette participants need. This information might

be presented by means ofreadiness reports, other intelligence summaries (INTSUMs), orders or

23
their annexes, messages by radio or tactical fax (TACFAX) that come in during the vignette to

augment the initial information, or other simulation-based conditions and events.

Be careful not to confuse "need" with "would have." Just because a brigade S3 would

normally have the division order before wargaming takes place does not mean that he will need

the complete division order to do the wargaming. It may be more efficient to provide the

OPORD summary and a course of action description, ifthe vignette is focused only on the

wargaming itself. Evaluate each suggested piece ofinformation carefully to make sure there is a

purpose for including it.

You will need to find a balance between realism and unnecessary detail. Normally the

brigade staffmay get information for developing a course of action branch from a variety of

sources. But you can provide that up-front information in a summarized form, without requiring

that they read through all possible sources in order to find out what has already happened. The

point is to include the necessary information, to present it in a way that is digestible, and to not

require performance that is outside the scope ofthe vignette.

You must also examine roles ofthe individuals that you decided were outside the line (in

Activity 2.2), and determine how to account for their inputs, roles, and actions. Either the

situation and underlying story have to make their involvement unnecessary, or you will need to

provide their input in their absence. Providing their input requires that you devise a way to make

their input standardized, rather than dependent on what the vignette participants ask for. Ifyou

can neither standardize the input, nor construct a storyline that does not require their presence,

then you must either include them as participants or re-draw the line so that the event does not

require their input.

Ifyou have decided that this will be a simulation-based vignette (in Activity 1.1), then the

simulation itselfwill provide some ofthe cues. But you will be responsible for ensuring that cues

occur when and where they make the most sense. You do this by controlling the simulationgenerated OPFOR, in terms ofwhere and when they move, how they're organized and equipped,

and so on. You will also specify, and probably script, the input from higher, adjacent, and

subordinate units that will be provided by radio or digital commo.

Activity 2.4 Describe specifications of tactical products

Product/Outcome: Description of the products.

Not every vignette will necessarily result in a product, but we have found that most vignettes

do have products. In particular, planning and preparation activities nearly always have products,

while execution events often do not.

At this point, you should be preparing a description ofwhat the product or products will be:

for example, a decision support template, an annex, or a rehearsal plan. In many cases, the

product will be exactly what the participants would develop in a real-world version ofthe

vignette. Ifthere are such real world products, such as orders or matrixes, you should try to

specify them as vignette products.

24
Other times, the product may be somewhat contrived. You may, for example, require that

they write down their complete list ofimplied, specified, and essential tasks; normally, they might

only record the essential tasks. You may even want to provide a worksheet, on which

participants fill in the blanks. This can be somewhat unrealistic, and make the vignette appear

more like a classroom exercise. But ifyou do it well, you may be inventing a very valuable job

aid that fills a void, and that participants will continue to use outside the vignette.

Summary: Phase 2

The activities in Phase 2 will have provided the details ofwhat the vignette will contain.

However, these details will very likely be modified during Phase 3, as you capture specifics about

the training objectives and standards. The SME reviews and walkthroughs will help to ensure

that the scenario is tactically correct and consistent. Ask yourselves:

• Is the tactical situation plausible?

• Is there just enough and not too much detail?

• Are all ofthe participants involved and interacting?

• Will they be able to understand what they're supposed to do?

• Is the situation challenging without being overwhelming?

PHASE 3. PREPARE DETAILED TRAINING OBJECTIVES

Activity 3.1 Identify performance requirements sources, domain, and standards.

Activity 3.2 Draft vignette objective and performance requirement statements.

Activity 3.3 Draft AAR aids.

Formative Evaluation: Internal reviews, proponent reviews.

During this phase, you will formulate the specific performance requirements that will be the

focus ofthe vignette. The three activities will involve finding sources ofinformation on

performance requirements for each vignette participant for the selected event (Activity 3.1),

refining the list to reflect the performance requirements for the vignette participants (Activity 3.2),

and then preparing a tentative list of AAR questions and the considerations for answers (Activity

3.3).

The list of performance requirements will serve two purposes:

1. It is the foundation ofthe observation and feedback mechanism ofthe exercise.

2. It defines the performance objectives for the intended users.

25
Formative Evaluation

Throughout this phase, the formative evaluation is still primarily internal, with frequent

proponent briefings to obtain concurrence on decisions and approaches, or to obtain guidance for

refining and modifying the work.

Activity 3.1 Identify performance requirements sources, domain, and standards

Product/Outcome: Performance requirements and standards.

The first activity in Phase 3 involves identifying sources of performance requirements

information for the selected event and participants. For many Army training projects, the most

likely task sources are the Army Training and Evaluation Program (ARTEP) Mission Training

Plans (MTPs) and field manuals (FMs). Other sources include training materials prepared by the

proponent agencies and task lists developed by the proponent schools. Consider and examine all

sources that are known or suggested to you. You might not use them all—they may be redundant

or in an early development stage or out of date, for example. You should keep a record of all

sources considered and the reason you did not use particular sources that you considered, in case

you need to revise your approach later.

Make sure that you have the most recent approved versions ofthe sources. If you know that

another version is coming out soon, you should review the draft ifyou can, see what differences

there will be, and then make or obtain a decision as to whether to use the current or the emerging

version. That decision will depend on when the change or update will be released and how vast

the differences are. Using a different version during development than the training participants

will use can cause confusion when the program is implemented. Your sponsor can help resolve

questions on the most acceptable references for task performance.

For a source to be useful, it must contain tasks or descriptions of performance requirements

for the selected event and vignette participants (specified during Activity 2.2). The tasks or

descriptions must clearly describe how to perform, under what conditions, and to what standard.

Sometimes, information from several sources can be combined to yield complete task analytic

information. Other times, you may need to add specificity by means ofsubject matter expert input

and proponent reviews.

The product for Activity 3.1 is a fairly comprehensive list oftasks and performance

requirements that describes the expected behavior ofthe vignette participants. In Activity 3.2, the

list will be refined in order to determine which tasks can be cued and observed during the

vignette.

26
Activity 3.2 Draft vignette objective and performance requirement statements

Product/Outcome: Vignette objective and list of performance requirements.

The vignette objective will appear in both participant and Training Coordinator guides, and

helps both the vignette participants and the Training Coordinator to know what the vignette

training should accomplish. It should tell them what the vignette training event is and what we

want to see the vignette participants do. Although it is clearly a training exercise, the vignette

objective should be worded to focus on the event, rather than the training (not "Learn how to plan

a rehearsal," but rather "Plan a rehearsal"). The objective should describe the focus ofthe

vignette, should come from the event description, and should be brief and direct. Figure 7 shows

a sample of a vignette objective and the associated requirements statements for a COBRAS

vignette.

The purpose ofthe performance requirements statements is to tell the user and the Training

Coordinator specifically what the participants will do or generate during the vignette. These

statements are an expansion ofthe vignette objective, and will help the user select the appropriate

vignette by telling what goes on in the vignette. The performance requirements statements

describe the major chunks ofbehavior or products that are involved in accomplishing or meeting

the objective. They will be based on the performance requirements that you derived in Activity

3.1, condensed to form a list of 4 to 10 statements ofthe meaningful chunks ofbehavior. This

need not be a strictly sequential listing ofthe steps in the event. The statements should tell what

has to be accomplished, rather than how to get there.

Vignette Title

Objective

Performance

Requirements

Plan brigade CSS rehearsal

Prepare a CSS rehearsal plan that specifies key CSS activities and key

participants.

1. Select CSS activities to be rehearsed.

2. Determine the participants and their responsibilities.

3. Determine the rehearsal type and technique.

4. Select the rehearsal time and site.

5. Develop the rehearsal agenda.

Figure 7. Sample vignette objective and performance requirements statements.

Ifyou are working on a simulation-based vignette, you will also have to make judgments

about which performance requirements can be included in the training, based on simulation

capabilities. You may find performances that you want to train, but that cannot (or should not) be

trained using the selected simulator or simulation. For example, a leader's reconnaissance might

be an important part ofmission planning, but it can't be realistically represented in a constructive

simulation. Some performance can be partially represented, and you will have to judge whether

27
those performance parts are critical and are well enough represented to justify their inclusion in

the training.

As you work through this activity for simulation-based vignettes, you will have to be, or have

access to, someone who is very familiar with the capabilities ofthe simulator/simulation. It will

not be enough to look at the capabilities descriptions provided by the vendor, nor to take the

word ofthe development engineers and technicians. There is no adequate substitute for first-hand

knowledge of how the simulation works. Unless you are an expert on the simulation, you will

need to work closely with a technician who can demonstrate the functions that you need to

replicate.

Activity 3.3. Draft AAR aids

Product/Outcome: List of AAR questions tied to performance requirements, with

considerations for responses.

The purpose ofthe AAR is to help the Training Coordinator and the vignette participants

"discover^' how to accomplish the tasks, what to consider, what to produce - in other words, how

to better perform during the event. Because the vignette is intended to be implemented with a

minimum of additional preparation on the part ofthe participants and the unit, vignettes usually

contain some kind of AAR aids. Any type ofAAR assistance that you provide has to be strictly

tied to the objective and performance requirements that you composed in Activity 3.2. While the

performance requirements focused on what had to be accomplished, the AAR will focus on

discussion of how to accomplish those things.

One type of AAR aid, used in the COBRAS vignette TSPs, is a set ofleading questions to

guide the discussion among participants after the vignette is completed (an example is shown in

Figure 8). Other approaches include providing an AAR outline or a list of key points for

discussion. In order to continue to keep preparation requirements to a minimum, you should also

provide some key considerations for answers to the AAR questions or discussion points. By

providing this information, you will also force yourselfto be sure that the questions can be

answered or the discussion points can be comprehended.

Summary: Phase 3

All ofthe training content decisions should have now been made. Phase 2 covered the

scenario context, and Phase 3 provided the detailed material to support guided practice and

feedback on the training objectives. You should continue to obtain SME and training developer

reviews ofyour ideas and decisions. You and your reviewers should be asking:

• Will the scenario support and cue these training objectives?

• Can an observer (who may not be an SME) use these materials to provide feedback?

• Is the information in the AAR aids correct?

• Is this exercise going to bring out important techniques and procedures ofperformance?

28
Objective: Prepare a CSS rehearsal plan that specifies key CSS activities and

key participants

Performance Requirements, Discussion Questions, and Response Considerations

1. Select CSS activities to be rehearsed.

1.1 What sources were used to identify CSS activities that might be rehearsed?

At a minimum, sources should include Commander's Guidance, Concept of Support

(paragraph 4 of the Bde OPORD), Synchronization Matrix (CSS line), and Service

Support Annex to Bde OPORD.

1.2 How were CSS activities selected to be rehearsed?

Sources should be used as follows:

• Commander's Guidance describes his interest areas; it may be restated in terms of

specific CSS activities.

• Concept of Support (paragraph 4a of the Bde OPORD) lists CSS activities that take

place before, during, and after the operation.

• Synchronization Matrix (CSS line) lists CSS activities that support the battlefield

framework. (Use it to confirm CSS activities drawn from Concept of Support and to

focus them for the rehearsal.)

• Service Support Annex to Bde OPORD provides addition details in support of CSS

activities.

CSS activities to be rehearsed should be bde-bn or across-bn (not within bn).

Consideration is made of time constraints allowing enough time to attend the

rehearsal, to decide on changes to the plan, to disseminate changes, and to return

to unit.

Determine the participants and their responsibilities.

2.1 Who were designated as rehearsal leader and recorder?

Rehearsal leader keeps rehearsal on track (on time and focused on selected CSS

activities and events). Usually S4, because he runs bde logistics for the

commander. Also, could be S1 as OIC of REAR, or FSB Cdr, as the senior

logistician.

Recorder keeps track of changes for the bde - usually someone from leader's section.

2.2 Who were designated for rehearsal setup and orientation?

Site preparation - usually the responsibility of the leader's section

Terrain and enemy orientation - S2 section

Maneuver orientation - S3 section or XO

2.3 Who were designated to demonstrate or describe CSS activities?

Those who will provide information, make decisions, and carry out decisions in

support of the selected CSS activities.

2.4 Who else should attend and why?

Those who are affected by or need to know about the decisions in order to make

their own decisions.

Continued on next page

Figure 8. Sample AAR questions and response considerations for a COBRAS vignette.

29
-ST.*

Objective: Prepare a CSS rehearsal plan that specifies key CSS activities and

key participants.

Performance Requirements, Discussion Questions, and Response Considerations

3. Determine the rehearsal type and technique.

3.1 Is the type appropriate to the situation (where they are and what they are doing)?

"Type" may be terrain model, map exercise, or key leaders.

Terrain model is best since the maneuver rehearsal is to be on a terrain model.

3.2 Is the technique appropriate to the selected events?

Three appropriate techniques are:

• Focus on each selected CSS activity throughout the full mission or operation

• Focus on all of the selected CSS activities during each phase of the operation

• Focus on each selected CSS activity during the most critical period of time for that

activity

Complete rehearsal of all of the brigade's CSS activities is not appropriate since the

XO has specified that only critical activities be rehearsed.

4. Select rehearsal time and site.

4.1 Are the time and site appropriate to the situation?

Consider

• Time available for planning, preparing for, and executing the rehearsal

• Time available to implement and disseminate changes

• Site and time of the combined arms rehearsal

• Conflicts for participants' time or the site ^

5. Develop the rehearsal agenda.

5.1 What agenda items were specified?

At a minimum, include:

• Rehearsal standard

• Overview of the rehearsal events

• Orientations (enemy, terrain, maneuver)

• CSS activities to be rehearsed

• Critique

5.2 What is the rehearsal standard?

(1) Inter-battalion activities for the specified events are synchronized, and (2)

rehearsal participants understand their roles and responsibilities in the event

activities.

Figure 8. Sample AAR questions and response considerations for a COBRAS vignette

(continued).

30
PHASE 4. DEVELOP TRAINING SUPPORT PACKAGE

Activity 4.1 Prepare training support package components.

Activity 4.2 Assemble the training support package.

Activity 4.3 Evaluate the vignette and the training support package.

Formative evaluation: Proponent review, tryouts (pilot tests and trials), expert

reviews.

This is the final phase in the development methodology, wherein you will actually develop all

ofthe components associated with the structured vignette. To a great extent, this process began

when you first outlined the scenario (Phase 2) and listed the performance requirements and AAR

aids (Phase 3). This section describes how to translate the earlier documentation into the final

products.

In Activity 4.1, the specific components ofthe TSP will be prepared. These components

include preparation materials, reference lists, tactical cues, and so on. In Activity 4.2, the task is

to design, build, and evaluate a TSP that will be suitable for implementation, so that the materials

developed in Activity 4.1 can be used in as many ofthe most likely situations as possible. In order

to do this, you need to know what the implementation situation(s) will be and what the materials

need to help the implementers do. By "situation," we mean the constraints and resources

available for implementation at the site or sites where the program will be used.

Formative Evaluation

The formative evaluation should be very externally oriented in this phase. Various level of

tryouts with experts and with representative users will be required in order to test the doctrinal

and instructional quality ofthe vignette. Because these tend to be fairly rigorous and intensive,

they are discussed more completely within the three activity descriptions that follow.

31
Activity 4.1 Prepare training support package components

Product/Outcome: TSP components.

Formative Evaluation: Proponent review; review by representative users;

developer walkthrough (pilot test).

During the event analysis (Activity 2.1), you walked through the extended event (i.e., the

event itself and the activities immediately preceding and following the event), in order to capture

all ofthe necessary information concerning:

• WHO are the key participants,

• WHERE would they be during the event,

• WHAT do they use as input for activities preceding the target event and for the event

itself,

• HOW do they use the inputs,

• WHAT do they produce, and

• WHY are they doing this, what led to this event.

In Activity 2.2, you "drew a line" between those event actions and participants that would be

part ofthe vignette and those that would not. In Activity 2.3, you decided how to ensure that the

vignette participants would be prompted to perform. In this activity, you will actually prepare the

specific TSP materials for the vignette.

The five kinds ofmaterials usually included in a TSP are described below specifically for

vignettes. They include:

• tactical and other scenario materials,

• unit preparation materials for vignette participants,

• materials to aid or train roleplayers or interactors (as needed),

• simulation materials (as needed), and

• exercise management materials.

Tactical and Other Scenario Materials

These are the items that set the conditions for the vignette, provide cues at the start of and

during the vignette, and serve as models in examining outcomes ofthe vignette. During Phase 2,

you may have decided that the participants would need a description ofthe mission analysis

outcomes in order to perform their course of action (COA) development. So you must find a way

to present those outcomes. Unless there is a set format or method for presenting the information,

you will need to design a way. Remember to keep it simple and focused on the vignette objective.

32
Ifyou are simply providing background information, do it in the simplest way possible. But ifyou

need to drive the participants to read and interpret tactical products, then provide the products.

These descriptions or products have to include specific details about the particular situation,

in enough detail to cue participants and enable them to actually perform the activities. Consider

use ofthe following types ofitems:

Tactical overlays or overlays showing current positions

Orders, selected annexes, selected portions of annexes, or summaries of annex

information

Transcripts or descriptions of commander's guidance or commander's intent

INTSUMs or descriptions of enemy activity

Supply reports and status update reports or descriptions

Decision support templates, synchronization matrixes

Description ofrehearsal plan

It is often better to contrive a simple way ofimparting information, rather than making

participants wade through more realistic tactical products, ifyour goal is to get them smart on

their situation before plunging into the vignette. Normally (in the real world), they would already

be familiar with the conditions, because they would have been present as the situation developed.

Therefore, there is no point in requiring that they become familiar with the situation the hard way.

Consider providing sample products in the vignette materials. The purpose ofthe sample

products is to give the Training Coordinator and participants something to compare the

participants' work to. Keep in mind that the sample products must be exemplary. In most cases,

the specific content ofthe product will not be the same as what the unit produces. But it should

be possible for them to infer something about the correct process by examining the product, and

by comparing the product to an exemplary product.

In some vignettes, but certainly not all, you may want to prepare and include job aids for

participants. The purpose ofjob aids is to give the participants a sample approach to a process

when the process is not very clearly defined in their manuals; or to give them the tools they

normally have. Job aids are useful when:

• there is only a vaguely defined process or no documented process for how to do the

vignette activity; in this case prepare a process diagram, outline, or flowchart;

• there is no set way to record information or decisions; in this case prepare a worksheet to

be completed; or

• there is a definite set way to record information or decisions; in this case give them the

actual forms to fill out.

As an example ofthe second case, in the COBRAS vignette on planning a CSS rehearsal, the

processes discussed in doctrinal manuals were nonspecific in terms of a desired outcome.

Therefore, the vignette developers prepared a form for CSS rehearsal planning, that the

participants filled out during the course ofthe vignette. A sample ofthe blank form is shown in

Figure 9. The vignette TSP also included a completed form as a model for use in the AAR.

33
COMBAT SERVICE SUPPORT REHEARSAL PLANNING WORKSHEET

Rehearsal Organization

OPORD

MISSION

REHEARSAL

STANDARD

PARTICIPANTS Leader

Recorder

Participants

Rehearsal

Attendees

CONDITIONS Type

Technique

Site

TIMELINE Bde Orders Prep

(Current Time)

Combined Arms Rehearsal

CSS Rehearsal Planning

CSS Rehearsal Warning Order

CSS Rehearsal

Disseminate Changes

Acknowledgm

Line of Depart

ent of Changes

ure

AGENDA WHAT WHO ; TIME

_____

Figure 9. Sample participant job aid from a COBRAS vignette.

Tactical information may also be contained in an initial orientation briefing and a situation

brief. The initial orientation briefing for a given vignette should usually be provided to

participants by the Training Coordinator several days before that vignette is conducted. Its

34
purpose is to tell the participants to start preparing, to hand out the participant guides, and to

point out the preparation steps and key features ofthe preparation materials. An example of an

orientation briefing outline is shown in Figure 11.

Orientation Briefing Outline - Modify as needed for the vignette and situation

Review the vignette scope, objective, and tasks

Review the limitations for the vignette activities

Review preparation activities that are required of the participants

Announce the date, time, and training site for the conduct of the vignette

Distribute the Participant Guides and preparation materials, to include acetate

copies of the Decision Support Template (DST) and CSS overlay. Provide the

map sheets to the participants at this time as well.

Figure 10. Sample orientation briefing from a COBRAS vignette.

The situation briefis generally presented to participants at the beginning ofthe vignette. It

includes the last pieces ofinformation that participants need in order to begin the vignette (e.g.,

commander's intent, INTSUM, order). The aim is to prompt the startup, rather than telling

participants exactly what to do; you want them to recognize the requirement for particular

performance, rather than being told the starting steps. The information is in a situation brief

rather than in readahead materials because you want to ensure that they work together, with their

"coach" on the requirement. If everything is in the readahead, they may work independently and

the Training Coordinator may not be able to observe their activities and interactions. A sample is

shown in Figure 11.

Preparation Materialsfor Training Participants

The preparation materials to be developed include a detailed list ofreferences for the

performance requirements. This will probably include FM and ARTEP-MTP references. The

purpose ofthe reference list is twofold:

1. to provide the doctrinal basis for the vignette objectives and

2. to provide further information for participants as they prepare.

Be selective, and be specific. You don't need to include every possible document that refers to

the activity, only those that provide important and helpful performance information. And cite the

section or chapter within the document that refers to the activity. With MTPs, list the tasks that

are included in the vignette.

35
Situation Brief Script (Read aloud to participants.)

In preparation for this vignette, you should have familiarized yourself with the tactical

materials and posted the overlay to the map. The brigade operations order has been

distributed to subordinate units, and planning for the combined arms rehearsal has been

completed We are now ready to start planning the CSS rehearsal. Because I have

allocated only two hours for the conduct of the CSS rehearsal, we only have time to

rehearse critical CSS activities. The brigade commander has provided some guidance

for planning the CSS rehearsal, as follows:

"Plan the rehearsal so that it is an opportunity to identify and practice what the brigade's

service support staff will do during the deliberate attack. I am specifically interested in

ensuring that the casualty evacuation piece is nailed down. Penetrating GRANITE and

COPPER may generate some serious losses for 3-5 and 1-80, and I want to make sure

we have ambulance exchange points (AXPs) and the other casualty evacuation pieces

where we need them - and when we need them.

Second I think division may launch us further west after we seize KIOWA, so plan to

rehearse the CSS actions we'll take to rearm, refuel, and reorganize the task forces after

KIOWA.

Make sure the S4 identifies any service support refinements that are necessary.

To assist you in developing your rehearsal plan, here is a CSS Rehearsal Planning

Worksheet. (Hand out the CSS Rehearsal Planning Worksheet.) I will want you to be

able to describe the details of your plan with me when you're done.

Do you have any questions before we begin?"

Figure 11. Sample situation briefing from a COBRAS vignette.

For some vignettes, you may also want to suggest specific preparation activities. For

example, a rehearsal or a wargaming session would be helpful in preparing for an executionoriented'vignette, in order for the participants to become familiar with the situation and the plan.

Even when there is little in the way oftactical preparation materials, the participants should get

together ahead oftime to discuss what they know and to make sure they have a common

understanding ofthe situation. An emerging approach incorporates multimedia-based

demonstrations of performance (Hoffman, 1997). Ifyou are going to recommend preparation

activities, the description should go in both the Training Coordinator materials and in the

participant materials.

The preparation materials should include information to set the context for the vignette. One

such type of context information is a scenario overview paragraph. The purpose is to help users

understand quickly what the general situation is, and how the unit got itselfto this point in the

mission This is where you hope to get the users out oftheir "real world" and into the vignette

world and mindset. Prepare it at a level somewhat wider than just the training audience level, and

refer to the mission situation rather than the training situation (e.g., "The 3 Bde, 55 ID, is

preparing to defend NLT 170600 May 11"). This paragraph should tell the users (at a minimum):

36
• who or what they are in the vignette, as opposed to who they are in real life,

• where the action takes place,

• what is going on and what hasjust happened, and

• who/what the enemy is.

Another item that may be used to define the context is the definition ofthe vignette scope.

The purpose ofthe scope is to help users understand how the vignette event fits into the

underlying scenario, so that they can identify where in the process they will start and how they

will know when they're done. It should have three parts:

• Preceding events: Describe what processes or activities have taken place before the

vignette event. This may include a compressed or general list ofwhat has already been

developed.

. Activities during the vignette: Describe how the vignette begins (e.g., with receiving the

commander's guidance), and describe what should happen during the vignette. This is

not the same as the performance requirements list; it is much briefer and more general.

• Ending point: Describe how the vignette will end and/or what the outcome or products

will be.

Yet another context-setting component is a description of vignette limitations or restrictions.

These tell the users what they cannot or will not be permitted to do during the vignette. The

purpose is to forewarn them about what they can and can't change and what they have to work

with. Limitations and restrictions include such things as who they can't talk to such as to their

division counterparts who are not represented during the vignette. It also includes what they

can't change, such as the COA already selected and developed for them to wargame. You should

mention only the things that you think they might want to do, and that they either shouldn't or

won't be able to do.

Samples of all three ofthese items - scenario overview, vignette scope, and vignette

limitations and restrictions - are shown in Figure 12.

Materials to Aid or Train Roleplayers and Interactors

The extent ofthis requirement will depend on the scope ofthe vignette. At the very least,

"other participants" includes the Training Coordinator. Much ofthe Training Coordinator's

guidance will be straightforward and relatively generic: notify the specific participants, distribute

materials for preparation, arrange for the facility and other resources, and so on. However, if

there is anything particularly important about the vignette setup, something that you fear will be

misunderstood or overlooked, be sure to prepare guidance that emphasizes the requirement.

37
Vignette Scenario Overview: In this vignette, the brigade, 3 Bde, 55 ID (M), is preparing to

conduct a deliberate attack (DATK) against an enemy motorized rifle regiment (MRR)

defending out of contact. The brigade currently occupies an assembly area (AA). The

vignette focuses on the staff activities required to plan a CSS rehearsal.

Vignette Scope: Prior to the start of this vignette, the notional staff has completed the

planning process and has published orders, including annexes. It has also completed

the plan for conduct of a combined arms rehearsal near the main command post

(MAIN) using a terrain model. Using information contained in the division OPORD and

the brigade OPORD, the staff must now plan a CSS rehearsal.

The commander's planning guidance for the CSS rehearsal will be given to the

participants. They will then plan the rehearsal.

The vignette ends when the participants complete the rehearsal plan.

Restrictions and Limitations: Limitations on the participants' activities include:

• They are not expected to conduct the planning activities which preceded their actual

entry into the process.

• They are not expected to develop support products such as a CSS execution matrix

or the FSB synchronization matrix.

• They will not be able to interact with other brigade and battalion/TF staff to seek

clarification of the annexes and support products.

• They will not be able to coordinate with subordinate units during their planning.

• They are not expected to conduct the rehearsal.

• They are not permitted to alter the brigade concept of support (COS) as they

develop their rehearsal plan. Their plan must be developed based on the given COS

and synchronization matrix.

Figure 12. Samples ofscenario overview, vignette scope, and vignette limitations and restrictions

from a COBRAS vignette.

Materials for the Training Coordinator should also include either an outline or the specific

text for the initial orientation briefing and the situation brief. The tactical and technical contents

ofthose briefings were described earlier.

Frequently, the most difficult part ofthe Training Coordinator'sjob is to be the vignette

director, performance coach, and AAR facilitator. To give the Training Coordinator the extra

advantage, the TSP provides performance requirement descriptions and AAR aids. Remind the

Training Coordinator to study the vignette requirements and be familiar with the vignette conduct

description, performance requirements, AAR questions, and response considerations.

For simulation-based vignettes, there will also be roleplayers, simulation interactors, and/or

simulation site managers. You should not need to prepare instructions on how to control the

simulation, as guidance already exists for most simulations, and simulation site managers generally

have a training package to use. But you will need to provide the specific instructions on how to

control the simulation for the vignette.

38
Simulation Materials

For vignettes that use virtual and constructive simulations, you will need to prepare the

simulation initialization data, on the medium used by that simulation (usually tape or disk). This

might include readiness level ofthe unit assets, locations, task organization, and even movement

and firing capabilities. You should also provide a hard-copy listing ofthe initialization data, to be

used in case the electronic medium fails or ifthe simulation is upgraded and the files are no longer

completely compatible. Ifthere is anything about the files that needs explanation or adjustment,

instructions for the simulation site manager may also be needed.

Exercise Management Materials

Finally, the vignette TSP will require management guidance. This is directed to the Training

Coordinator, and gives suggestions about how to set up the vignette, how much time to allow,

how to keep the participants on track, how to answer specific questions, and so on. Only include

what is necessary. For example, ifthere is only a minuscule chance that the participants could get

offtrack, then don't waste space telling how to keep them on track.

Summary ofMaterial Preparation

When you have completed this activity, you should have prepared the specific materials for

the vignette. These materials incorporate realistic information concerning the situation, the

enemy, and the unit itself(the METT-T). Figure 13 describes the TSP components for the

COBRAS vignettes.

TSP Components and Structure for the COBRAS Brigade Staff Vignettes

Guide to Use and Implementation of COBRAS Vignettes: Overview of the vignette intents;

how to select, prepare, and conduct

Training Coordinator Materials: Overview of the vignette scope, participants, and tasks;

information on how to get ready; list of training objectives; how to initiate and control the

vignette; AAR questions

Training Participant Materials: Overview of the vignette scope and tasks; information on

how to get ready; list of training objectives and references

Preparation Materials: Selected tactical materials to provide the setting and situation for the

vignette event

Execution Materials: Selected tactical materials to cue and shape the vignette problem

Job Aid Materials: Provided for selected vignettes to help participants perform the tasks

Sample Products: For use in illustrating general form and content of brigade staff products

Support Coordinator Materials: For use in simulation-supported vignettes; guidance for

roleplayers and interactors; simulation tapes and documentation

Figure 13. Sample listing ofTSP materials for COBRAS vignettes.

39
The final piece to construct is the vignette title. This should be a short handle (3-5 words)

that describes what the vignette activity is. The title should give users a handy reference to this

vignette that distinguishes it from other vignettes, can be remembered, and is appropriate to the

vignette. It's the final piece because it has to capture (as much as possible) the foregoing

decisions about the vignette event and coverage.

It is critical at this point to do an SME scrub ofthe vignette materials. This may take the

form of a tryout (or pilot test) ofthe components, even though the rest ofthe TSP has not yet

been assembled; you and your fellow developers will provide the information and guidance

normally contained in the TSP. You will also roleplay the Training Coordinator, and SMEs who

did not work on the vignette will represent the participants. Your focus should be:

• Could the participants understand what you (the TSP) told them to do? This applies to

preparation activities as well as vignette conduct. If not, find out what the

misunderstanding was, and correct the source ofthe error. It may be in the vignette

limitations description, administrative briefinstructions, situation brief, or the actual

tactical materials.

• Could they do what they were supposed to do with the materials given them? Were the

materials complete, correct, and useful? Be absolutely scrupulous in ensuring that there

are no discrepancies or inconsistencies between components.

• Did they produce the product that you anticipated? If not, either adjust your expectation

or correct the instructions to bring them into line.

• Could they understand and respond to the AAR questions? Were the response

considerations appropriate, given their responses? If not, you should look closely at the

objective, the performance requirements, the AAR questions, and the response

considerations to discover the source ofthe disconnect. Ask the participants for input on

why the questions did not produce the sorts of discussion that you expected them to.

• Was the training participant designation appropriate? If not, consider whether to add or

drop participants, or whether to add or drop the cues for those people.

• Was this a vignette with some training value? Ifthe SMEs think not, they may be able to

suggest changes that would enhance the training value. Otherwise, you and your sponsor

will need to take a close look at the vignette value.

Ifyou are constructing a vignette that uses constructive or virtual simulation, you should go

to the simulation to try out the vignette components. In addition to the focus points listed above,

you will also need to:

• verify that everything that is required can be represented on the technology,

• verify that the exercise works on the terrain, and

• ensure that all specifications, friendly and enemy, are set appropriately.

In order to do this formative evaluation, you will need to build the preliminary files for the

simulation. Although it will depend on the technology you use, the simulation files will probably

include starting conditions, routes, graphics, decision points, and system capabilities and readiness

40
levels. You need to do these preliminary tryouts using the simulation before the trials with

representative participants (Activity 4.3).

Activity 4.2 Assemble the training support package

Product/Outcome: The draft TSP.

Formative Evaluation: Expert reviews.

This activity brings together all ofthe design and development work and products that has

gone on to date. This is the culmination ofthe development, leading up to the trial

implementation and evaluation, revisions, and final product. During the previous activity, you

prepared the vignette-specific components ofthe TSP. Where earlier, you might have indicated a

need for Annex G or an ESTTSUM, now you have constructed the particular Annex G or

INTSUM that the participants will use. It is not generic, but has plausible data and information

that will cue the participants to perform the activities underlying the vignette objective.

These materials now need to be packaged. Packaging occurs on at least two levels. Some of

it is tailored to the type oftraining you are constructing (e.g., vignettes), and some is a

requirement ofthe training proponent. The latter category, which will not be addressed in this

guide, includes classification markings, control numbers, proponency identification and

imprimatur, and so on. For those items, follow the guidance of TRADOC Regulation 350-70

(DA 1995) and the training proponent for your vignettes.

Some ofthe most important considerations that affect your efforts for this activity are the

time-honored instructional design considerations: to make the materials not only technically

correct and complete, but also situation- and user-friendly. Broadly defined, the TSP is

everything that is needed to support every aspect ofimplementation ofthe vignette. The

remainder ofthis section contains some suggestions drawn from COBRAS experience about how

to prepare a TSP. But you should be ready to improvise as necessary so that the TSP you

develop is appropriate for the training you develop.

Ifyou are an instructional developer, then you understand the principles of materials

development that can make the program accessible; if not, the points below will highlight some

important things that you can do to make sure that the users can, in fact, use the materials.

• The organization ofthe materials has to be almost precise and clear. The user must be

able to find the critical information quickly. It should be clear to the Training

Coordinator:

• where things are,

• who gets them, and

• where they get them.

• While it may also be important to include background, explanation, justification, and so

on, the critical how-to information must be clearly visible. Think about the difference

between "just in case" presentation that includes all ofthe whys and wherefores, and "just

41
in time" presentation that is instructional and directive, with a separate section ofreasons

why.

• Don't be bound by too many traditional ideas of how training materials should look.

You're writing for users, and the desktop publishing people have done a lot to help

specify what it takes to make materials useful. You should be aiming for materials that

are easy to read and understand, using graphics where they aid in understanding. But, of

course, be aware ofand adhere to any layout andformat restrictions that are already in

place.

• Some ofthe user guides put out by computer manufacturers and software writers serve

as useful models. Spend some time looking at them for ideas, or look at the VTP or

COBRAS TSP materials'0. You'll probably notice lots ofwhite space, use of graphics

and look-up tables, icons and symbols that catch the user's attention, lines or borders that

help to group relevant material, headers and footers that help the user know where he/she

is, and so on.

Once the TSP is constructed, and before you move on to the trial (Activity 4.3), conduct a

series of expert reviews ofthe technical content and the instructional features ofthe vignette.

This will help to insure face validity as well as helping you to refine all ofthe components ofthe

training program and package. There are several levels of expert review recommended:

• Individuals who are experts on the simulation and its components should make sure that

your simulation controller instructions are clear and correct.

• Individuals who are experts on the doctrinal issues for the vignette event should review

the materials to be used during preparation, vignette conduct, and the AAR for doctrinal

accuracy.

• Someone who is familiar with the needs and resource constraints ofthe target units

should check the advance materials and program management materials to see if

instructions for duplication and distributing are clear.

• A review by instructional design experts can help you with the presentation of all ofthe

information.

10 Copies of VTP materials may be viewed by contacting the Warthog O/C Team, 16th Cavalry, Fort Knox,

Kentucky. COBRAS materials may be viewed by contacting the Force XXI Training Program, Fort Knox,

Kentucky.

42
Activity 4.3 Evaluate the vignette and the training support package

Product/Outcome: The TSP, revised as necessary after evaluation.

Formative Evaluation: Trials with representative participants.

This final, major formative evaluation activity is to conduct a formal trial ofthe vignette.

Trials involve representative personnel acting as participants and provide evidence ofthe usability

ofthe materials. Data collection will be standardized and intensive, involving individual and

group interviews and detailed observations of a wide variety ofindicators. In order to ensure

completeness and rigor, you should have a formal plan for the trials. Some ofthe important

considerations include:

• Involve actual unit personnel and other target personnel in as many ofthe participant

roles as possible, as well as in the role ofTraining Coordinator.

• Use TSP materials that are as close to "perfect" as you can get; don't plan on using

available developers to interpret the materials to the users (as you did in the Activity 4.1

roleplay). You want the users to diligently attempt to use the materials as they are

intended to be used. But do be ready to interrupt the vignette if necessary to determine

what went wrong or to work around a fatal error.

• Before the trial, develop questions about any vignette points you are unsure of. For

example, "Was the Chemical Officer (CHEMO) needed? Did we need the intelligence

annex? Was the job aid on rehearsals useful?"

• Collect information throughout the whole process ofvignette preparation by the Training

Coordinator and participants, as well as during the conduct ofthe vignette. Use both

observations and informal discussions. After the training, you should interview the

participants to get their reactions to the training content and materials.

• Use a variety ofmethods to collect the information: observations with prepared question

lists, group discussions, structured interviews, questionnaires and surveys, video- and

audio-tapes.

• The aspects ofthe program that are best examined during a trial include verification of

assumptions and expectations, assessment or refinement ofthe methods for conducting

the exercise, evaluation ofthe clarity and utility ofthe materials, and judgment ofthe

completeness ofthe materials and absence of extraneous material.

• Document your evaluation trial plan: participants, process, data collection methods, and

any caveats or alibis about how the trial is conducted and how that differs from the

intended implementation.

43
Activity 4.3 Evaluate the vignette and the training support package

Product/Outcome: The TSP, revised as necessary after evaluation.

Formative Evaluation: Trials with representative participants.

This final, major formative evaluation activity is to conduct a formal trial ofthe vignette.

Trials involve representative personnel acting as participants and provide evidence ofthe usability

ofthe materials. Data collection will be standardized and intensive, involving individual and

group interviews and detailed observations of a wide variety ofindicators. In order to ensure

completeness and rigor, you should have a formal plan for the trials. Some ofthe important

considerations include:

• Involve actual unit personnel and other target personnel in as many ofthe participant

roles as possible, as well as in the role of Training Coordinator.

• Use TSP materials that are as close to "perfect" as you can get; don't plan on using

available developers to interpret the materials to the users (as you did in the Activity 4.1

roleplay). You want the users to diligently attempt to use the materials as they are

intended to be used. But do be ready to interrupt the vignette if necessary to determine

what went wrong or to work around a fatal error.

• Before the trial, develop questions about any vignette points you are unsure of. For

example, "Was the CHEMO needed? Did we need the intelligence annex? Was the job

aid on rehearsals useful?"

• Collect information throughout the whole process of vignette preparation by the Training

Coordinator and participants, as well as during the conduct ofthe vignette. Use both

observations and informal discussions. After the training, you should interview the

participants to get their reactions to the training content and materials.

• Use a variety ofmethods to collect the information: observations with prepared question

lists, group discussions, structured interviews, questionnaires and surveys, video- and

audio-tapes.

• The aspects ofthe program that are best examined during a trial include verification of

assumptions and expectations, assessment or refinement ofthe methods for conducting

the exercise, evaluation ofthe clarity and utility ofthe materials, and judgment ofthe

completeness ofthe materials and absence of extraneous material.

• Document your evaluation trial plan: participants, process, data collection methods, and

any caveats or alibis about how the trial is conducted and how that differs from the

intended implementation.

43
Summary: Phase 4

While formative evaluation is a continuous and ongoing process in all phases of development, it

becomes highly visible during Phase 4. There is no good alternative to a trial with representative

users in evaluating structured training. You owe it to the other developers and the participants, as

well as to yourself, to insist on the resources for a trial, and then to plan and prepare carefully

enough to justify the expense.

44
LESSONS LEARNED

This process for developing vignettes evolved over a period oftwo years on the ARI

COBRAS projects. During that period, there were many occasions for revising the procedure,

and many instances that reinforced the viability ofthe approach. The following points emerged as

consistently important considerations during vignette design and development.

w Design and development must begin with a general selection ofthe event and target

audience, but this decision should be kept open until after the event analysis is

completed.

tm- The analysis ofthe extended event must be complete, and the outcomes must be allowed

to influence the selection ofthe precise event and training audience.

w It is easy to make vignettes extensive. The difficult but necessary effort is to keep them

focused on a small group and a selected, limited event. Without this focus, the vignette

becomes a "little war" and the purpose of having easily implemented focused exercises is

defeated.

tm- More than one thorough review of all ofthe tactical materials must be conducted in

order to ensure that they are 1) correct and consistent; 2) necessary and sufficient for

performance; and 3) understandable by the participants. These reviews cannot consist

only ofreading the materials, but must also include tryouts with SMEs and

representative users.

W A thorough review ofthe TSP by instructional designers must be conducted to ensure

that the package is usable and comprehensible.

w A rigorous trial ofthe full TSP, under full implementation conditions, must be

conducted to ensure that the entire vignette can be implemented by the intended user,

with the intended outcomes.

The authors and the COBRAS developers are convinced that the approach and the lessons

stated above will assist other training developers in constructing instructionally sound and

valuable structured exercises for small groups.

45
REFERENCES

Campbell, C. H., Campbell, R. C, Sanders, J. J., Flynn, M. R., & Myers, W. E. (1995).

Methodology for the development ofstructured simulation-based training (Research

Product 95-08). Alexandria, VA: U. S. Army Research Institute for the Behavioral and

Social Sciences.

Campbell, C. H., Deter, D. E., & Quinkert, K. A. (1997) Report on the guide to development of

structured simulation-based training (Research Report 1710). Alexandria, VA: U.S.

Army Research Institute for the Behavioral and Social Sciences.

Campbell, C. H. & Deter, D. E. (1997). Guide to development of structured simulation-based

training (Research Product 97-14). Alexandria, VA: U.S. Army Research Institute for

the Behavioral and Social Sciences.

Department of the Army (1988). Systems approach to training (TRADOC Regulation 350-7).

Washington, DC: Author, Training and Doctrine Command.

Department of the Army (1995). Training and doctrine development management, products, and

process (TRADOC Regulation 350-70, Draft). Fort Monroe, VA: Author, Training and

Doctrine Command.

Goldstein, I. L. (1980). Training and organizational psychology. Professional Psychology, 11,

421-427.

Graves, C. R., Campbell, C. H., Deter, D. E., & Quinkert, K. A. (1997). Combined arms

operations at brigade level, realistically achieved through simulation I (COBRAS I)

protect: Development and lessons learned (Research Report 1719). Alexandria, VA:

U.S. Army Research Institute for the Behavioral and Social Sciences.

Graves, C. R. & Myers, W. E. (1997). An expansion of the virtual training program: History and

lessons learned (Research Report 1703). Alexandria, VA: U.S. Army Research Institute

for the Behavioral and Social Sciences.

Herman, J. L., Morris, L. L., & Fitz-Gibbon, C. T. (1987). F.valnator's Handbook. Newbury

Park, CA: Sage Publications.

Hoffman, R. G. (1997). A description of multimedia presentation ofCOBRAS vignette training

support package information (Research Product 97-16). Alexandria, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences.

Hoffman, R. G., Graves, C. R., Koger, M. E., Flynn, M. R., & Sever, R. S. (1995). Developing

the reserve component virtual training program: History and lessons learned (Research

Report 1675). Alexandria, VA: U.S. Army Research Institute for the Behavioral and

Social Sciences.

Koger, M. E., Quensel, S. L., Sawyer, A. R., Sanders, J. J., Crumley, K. A. & Brewer, J. D. (in

preparation). Staff Group Trainer: Development of a computer driven, structured, staff

training environment (Research Report 1718). Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences.

47
Lickteig, C. W. (1991). Prototype methods for training and assessing future tactical command

and control skills (Research Product 92-01). Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences.

Lickteig, C. W. & Emery, C. D. (1994). Information management performance of future platoon

leaders: An initial investigation (Technical Report 1000). Alexandria, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences.

Muchinsky, P. M. (1987). Psychology anolied to work (2nd Ed). Chicago: Dorsey Press.

48
Appendix A

Acronyms

AA Assembly Area

AAR After action review

AFRU Armored Forces Research Unit

ARI Army Research Institute

ARTEP Army Training and Evaluation Plan

AXP Ambulance Exchange Point

BBS Brigade/Battalion Battle Simulator

BDE Brigade

BN Battalion

CDR Commander

CHEMO Chemical Officer

COA Course of action

COBRAS Combined Arms Operations at Brigade Level, Realistically Achieved through

Simulation

COS Concept of Support

CP Command post

CSS Combat Service Support

DA Department ofthe Army

DATK Deliberate Attack

DST Decision Support Template

FM Field Manual

FSB Forward Support Battalion

INTSUM Intelligence summary

METT-T Mission, enemy, troops, terrain, time

METT-TC Mission, enemy, troops, terrain, time, communications

MRR Motorized Rifle Regiment

MTP Mission Training Plan

NTC National Training Center

O/C Observer/Controller

OIC Officer in Charge

OPFOR Opposing force

OPORD Operations order

RC Reserve Component

A-l
Appendix A

Acronyms

AA Assembly Area

AAR After action review

AFRU Armored Forces Research Unit

ARI Army Research Institute

ARTEP Army Training and Evaluation Plan

AXP

BBS Brigade/Battalion Battle Simulator

BDE Brigade

BN Battalion

CDR Commander

CHEMO Chemical Officer

COA Course of action

COBRAS Combined Arms Operations at Brigade Level, Realistically Achieved through

Simulation

COS Concept of Support

CP Command post

CSS Combat Service Support

DA Department ofthe Army

DATK Deliberate Attack

DST Decision Support Template

FM Field Manual

FSB Forward Support Battalion

INTSUM Intelligence summary

METT-T Mission, enemy, troops, terrain, time

METT-TC Mission, enemy, troops, terrain, time, communications

MRR Motorized Rifle Regiment

MTP Mission Training Plan

NTC National Training Center

O/C Observer/Controller

OIC Officer in Charge

OPFOR Opposing force

OPORD Operations order

RC Reserve Component

A-l
R&D Research & Development

RCVTP Reserve Component Virtual Training Program

51 Personnel Officer

52 Intelligence Officer

53 Operations Officer

54 Logistics Officer

SAT Systems Approach to Training

SIMUTA Simulation-Based Multiechelon Training Program for Armor Units

SIMUTA-B Simulation-based Multiechelon Training Program for Armor Units - Battalion

Exercise Expansion

SITREP Situation Report

SME Subject matter expert

TACFAX Tactical fax

TF Task Force

TOC Tactical Operation Center

TRADOC U.S. Army Training and Doctrine Command

TSP Training support package

USAARMC United States Army Armor Center

VTP Virtual Training Program

WARNO Warning Order

XO Executive Officer

A-2
Appendix B

Procedure for Development of Structured Simulation-Based Training

OUTLINE OF THE METHODOLOGY

B-l
Phase 1. Prepare Initial Decisions

Activity 1.1 Document Initial Scope Decisions

Product or Documentation ofinitial decisions on vignette scope.

outcome

Formative Internal review, proponent review.

evaluation

Decision areas 1. Target training audience

(see page 15) 2. Scenario context

• Mission type

• Enemy type

• Terrain/locale

• Unit type

• Appropriate training environment (e.g., simulation type)

3. Other, e.g.:

• Vignette time limits

• Linkages to other vignettes

• Trainer and other resources

B-2
Activity 1.2 Select Vignette Event and Participants

Product or Initial description ofthe vignette event and participants.

outcome

Formative Internal review, proponent review.

evaluation

Limiting the 1. Performance requirements

SCOpe 2. Time

(see page 17) ^ _,. „

3. Number ofparticipants

Activity 1.3 Outline Vignette Presentation model

Product or Description ofthe implementation design for the vignette.

outcome

Formative Internal review, proponent review.

evaluation

Considerations 1. TSP components

(see page 18) 2. Who is involved in implementation

3. When/where it should be implemented

B-3
Phase 2. Design Vignette Scenario

Activity 2.1 Analyze Event Context and Participation

Requirements

Product or

outcome

• Outline of activities before, during, and after the vignette event.

• Description ofroles of all persons involved before, during, and after the

vignette event.

Formative

evaluation

Internal review, proponent review, SME review and walkthrough

Event analysis • WHO are the key players,

information

(seepage 21)

• WHERE would they be during the event,

• WHAT do they use as input for activities preceding the target event and

for the event itself,

• HOW do they use the inputs,

• WHAT do they produce, and

• WHY are they doing this, what led to this event.

B-4
Activity 2.2 Determine Vignette Scope and Specific

Participants

Product or

outcome

Formative

evaluation

Description of cues and products (inputs and outputs).

Designation oftarget participants and their actions

Specification of event beginning and ending points.

Internal review, proponent review; SME review and walkthrough

Drawing the line • Decide what actions, participants, and products to include

(seepage 22) • Decide what and who to "script out"

• Limiting goals: 3-7 participants, 2-3 hours

B-5
Activity 2.3 Determine Vignette Inputs and Method for

Delivery

Product or • Description ofvignette tactical inputs and inputs from nonparticipants.

• Description of how the inputs will be provided.

Formative Internal review, proponent review; SME walkthrough

evaluation

Considerations • Provide information that will prompt participants to perform

(seepage 23) • Provide a minimum - only what is needed

• Format to balance realism in inputs against unnecessary detail

• Account for (provide) input from nonparticipants

Activity 2.4 Describe Specifications of Tactical Products

Product or Description ofthe products

outcome

Formative Internal review, proponent review, SME walkthrough

evaluation

Considerations • Not required for all vignettes; more common for planning or preparation

(see page 24) events tnan f°rexecution events

• May be real-world product or an invented outcome

B-6
Phase 3. Prepare Detailed Training Objectives

Activity 3.1 Identify Performance Requirements Sources,

Domain, and Standards

Product or

outcome

Performance requirements and standards

Formative

evaluation

Internal review, proponent review

Considerations • Obtain most recent approved versions ofthe sources

(seepage 26) • Check for performance descriptions, including how to perform, under what

conditions, standards

B-7
Activity 3.2 Draft Vignette Objective and Performance

Requirement Statements

Product or

outcome

Vignette objective and list of performance requirements

Formative

evaluation

Considerations

(seepage 27)

Internal review, proponent review

The vignette objective tells what the vignette training event is and generally

what we want to see the participants do

The performance requirements statements tell specifically what the

participants will do or generate during the vignette

• Describe the major chunks ofbehavior or products that are involved

in accomplishing or meeting the objective

• Condensed from list in Activity 3.1 to form a list of4-10 statements

• Tell what has to be accomplished, rather than how to get there

• For simulation-based vignette, ensure that the activity can be cued,

performed, and observed in the simulation environment

B-8
Activity 3.3 Draft AAR Aids

Product or List of AAR questions tied to performance requirements, with considerations

outcome for responses

Formative

evaluation

Internal review, proponent review

Considerations • Performance requirements focused on what had to be accomplished; the

AAR will focus on discussion of how to accomplish those things. (see page 28)

One approach:

• A set ofleading questions to guide the discussion among participants

after the vignette is completed

• Key considerations for answers to the AAR questions or discussion

points

B-9
Phase 4. Develop Training Support Package

Activity 4.1 Prepare Training Support Package Components

Product or

outcome

TSP components, including (at a minimum):

J tactical and other scenario materials,

/ unit preparation materials (training participants),

/ materials to aid or train other participants,

y simulation materials (as needed), and

S exercise management materials.

Formative

evaluation

Proponent review; review by representative users; developer walkthrough

Tactical and

other scenario

materials

(see page 32)

Items that set the conditions for the vignette, provide cues at the start of and

during the vignette, and serve asjob aids in examining outcomes ofthe

vignette

Keep it simple and focused on the vignette objective

Some information may be presented in initial orientation briefing and/or

situation brief

May include sample products for their comparison

May include job aids to help them structure how they perform, when the

process is not yet very structured; or to give them the tools they normally

have

• unit preparation materials (training participants)

• materials to aid or train other participants

• simulation materials (as needed)

• exercise management materials

Continued on next page

B-10
Activity 4.1 Prepare Training Support Package Components,

Continued

Unit preparation • Purpose is to provide evidence of doctrinal basis for the vignette objectives

materials and to provide further information for participants as they prepare

(seepage 35) . Includes detailed list ofreferences for the performance requirements

• May include specific preparation recommendations

• Includes context-setting information:

• Who (and what) they are in the vignette (as opposed to who they are

in real life)

• Where the action takes place

• What is going on and what has just happened

• Who/what the enemy is

• Includes statement ofvignette scope:

• Preceding events: Describe what has taken place before the vignette

event. This may include a compressed or general list ofwhat has

already developed.

• Activities during the vignette: Describe how the vignette begins and

what will (or should) happen during the vignette. This is not the

same as the performance requirements list; it is much briefer and

more general.

• Ending point: Describe how the vignette will end and/or what the

outcome (products) will be.

• Includes vignette limitations and restrictions

Continued on nextpage

B-ll
Activity 4.1 Prepare Training Support Package Components,

Continued

Materials to aid

or train other

participants

(seepage 37)

Includes guidance for Training Coordinator:

• Administrative requirements (notify the specific participants,

distribute materials for preparation, arrange for the facility and other

resources)

• Outline or the specific text for the initial orientation briefing and the

situation brief

• Performance requirement descriptions and AAR aids and how to use

them

For simulation-based vignettes, vignette-specific instructions for roleplayers,

simulation interactors, and/or simulation site managers

Simulation

materials

(seepage 39)

For vignettes that use virtual and constructive simulations:

Includes simulation initialization data, on the medium used by that simulation

Includes hard-copy listing ofthe initialization data, to be used in case the

electronic medium fails or ifthe simulation is upgraded and no longer

completely compatible

Exercise • Directed to the Training Coordinator:

management

materials

(see page 39)

• How to set up the vignette

• How much time to allow

• How to keep the participants on track

• How to answer specific questions

Continued on next page

B-12
Activity 4.1 Prepare Training Support Package Components,

Continued

Pilot test • Tryout of completed components, not the entire TSP

(seepage 39) • Uses vignette developers to provide guidance usually contained in the TSP;

focus is on the vignette content

• Use SMEs as training audience

• Questions to be answered:

• Could the participants understand what they were to do?

• Could they do what they were supposed to do with the materials

given them? Were the materials complete, correct, and useful?

• Did they produce the product that you anticipated?

• Could they understand and respond to the AAR questions? Were the

considerations appropriate, given their responses?

• Was the training participant designation appropriate?

• Was this a vignette with some value?

• For simulation-based vignettes:

• Verify that everything can be represented on the technology

• Verify that the exercise works on the terrain

• Ensure that all capabilities and specifications, friendly and enemy, are

set appropriately

B-13
Activity 4.2 Assemble the Training Support Package

Product or

outcome

The draft TSP

Formative

evaluation

Expert reviews

Design

considerations

(see page 41)

Pay attention to proponent requirements (e.g., classification markings,

control numbers, proponency identification and imprimatur)

The organization ofthe materials has to be precise and clear

The user must be able to find the critical information quickly

Aim for materials that are easy to read and understand, using graphics where

they aid in understanding

Expert review

considerations

(see page 41)

Individuals who are experts on the simulation and its components should

make sure that simulation controller instructions are clear and correct.

Individuals who are experts on the doctrinal issues for the vignette event

should review the materials to be used during preparation, vignette conduct,

and the AAR for doctrinal accuracy.

Someone who is familiar with the needs and resource constraints ofthe

target units should check the advance materials and program management

materials.

A review by instructional design experts can help you with the presentation

of all ofthe information.

B-14
Activity 4.3 Evaluate the Vignette and the Training Support

Package

Product or

outcome

The TSP, revised as necessary after evaluation

Formative

evaluation

Trials with representative participants

Considerations

for the trial

(see page 43)

• Involve actual units and other target personnel in as many ofthe roles as

possible.

• Use TSP materials that are as close to "perfect" as you can get; don't plan

on using available developers to interpret the materials to the users. But do

be ready to interrupt the vignette if necessary to determine what went wrong

or to work around a fatal error.

• During vignette preparation and the conduct, collect data (observations,

opinions and suggestions, discussions ofways offixing things that aren't yet

right). After the vignette, you will interview the participants to get their

reactions to the training content and materials.

• Use a variety ofmethods to collect the information (e.g., observations,

group discussions, structured interviews, questionnaires and surveys, videoand audio-tapes).

• The aspects ofthe program that are best examined during a trial include

• Verification of assumptions and expectations

• Assessment or refinement ofthe methods for conducting the exercise

• Evaluation ofthe clarity and utility ofthe materials

• Judgment ofthe completeness ofthe materials and absence of

extraneous material.

• Document your formative evaluation trial plan: participants, process, data

collection methods, and any caveats or alibis about how the trial is

conducted and how that differs from the intended implementation.

B-15