IPC FILE COPY

ARI Research Note 87-16

METHODOLOGY DEVELOPMENT FOR DERIVING LESSONS LEARNED

FROM THE NATIONAL TRAINING CENTER:

PROGRESS AND FUTURE DIRECTIONS

0

o

0 I

William J. Doherty

(Editor)

The 8DM Corporation

for

Contracting Officer's Representative

Nancy K. Atwood

ARI Field Unit at Presidio of Monterey, California

James H. Banks, Chief

TRAINING RESEARCH LABORATORY

Jack H. Hiller, Director

DTJC _______ELECTE,

MAY 1 3 1987 U ___

E

U. S. Army

Research Institute for the Behavioral and Social Sciences

February 1987

Aoo'oved for Pub,C release. jstrbution un r'ted.

87\ I
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES -

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WM. DARRYL HENDERSON

EDGAR M. JOHNSON COL, IN

Technical Director Cornnunding

Research accomplished under contract-' .

for the Department of the Army

The BDM Corporation

Technical review hy

John J. Kessler

SThis toPort. as submitted by the contractor. has been cleared lot Weease to Defense Technical information Cenit

(OTICI to comply with regulatory requirements. It has been given no primary distr ibution other than to DTlC

and will be available only through OTIC or other reference services such as the National Technical Information

Satri~c* (NTIS). The vits cpfni- ns. and/or findings contained in this report are thoje of the author(sl and

should not be construeu as an offiesia CIepaitmervi of the Army position, policy. or decision. unless so design~ated

by other official documtetaton.

6i, lJi k
UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (Whe.n Data Entered) __________________

REPORT DOCUMENTATION PAGE READ INSTRUCTIONS

BEFORE COMPLETING FORM

I. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

ARI Research Note 87-16

4. TITLE (and Subtitle) S. TYPE OF REPORT & PERIOD COVERED

Methodology Development for Deriving Lessons Final Report

Learned from the National Training Center: March 85-February 86

Progress and Directions 6. PERFORMING ORG. REPORT NUMBER

BDM/ARI-TR-0018-86

7. AUTHOR(.) 8. CONTRACT OR GRANT NUMBER(e)

William J. Doherty (Editor) MDA 903-35-C-0472

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT PROJECT, TASK

AREA & WORK UNIT NUMBERS

The BDM Corporation 2Q263743A794

2600 Garden Road, North Building 511CI

Monterey, CA 93940

1 I. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

ARI Field Unit at Presidio of Monterey, February 1987 California, P.O. Box 5787 13. NUMBER OF PAGES

Presidio of Monterey, CA 93944 73

14. MONITORING AGENCY NAME A ADDRESS(I! different from Controlling Office) 1S. SECURITY CLASS. (of this report)

U.S. Army Research Institute for the Behavioral Unclassified

and Social Sciences, 5001 Eisenhower Avenue, I

Alexandria, VA 22333-5600 Is. DECLASSIFICATION/DOWNGRADING J SCHEDULE

16. DISTRIBUTION STATEMENT (of Wi Report)

Approved for public release; distribution unlimited.

.

17. DISTRIBUTION STATEMENT (of the abstract entered In Block 20, If different from Report)

I' 18. SUPPLEMENTARY NOTES

Nancy K. Atwood, contracting officer's representative

19. KEY WORDS (Continue on reverse side If necessary and Identify by block number)

Army Research Institute (ARI)

Home Station

National Training Center (NTC)

Training

20. ABSTRACT (rCaifIme asm re,*r"s alt it ner area md Ideulty by block number)

As part of a 3 year contract effort, in support of the Army Research Institute's (ARI's) research program at the National Training Center (NTC), a plan

has been prepared that summarizes the first year's progress, and describes the

activities for the second year.

During the first year, 22 identifiable products or reports were prepared.

These fell into four major areas of endeavor: measurement of unit performance,

development of lessons learned, investigation of data quality, (OVER)

DO I JOARM3) 1473 EDITION OF N NOV 6s IS OBSOLETE UNCLASSIFIFD

iSECURITY CLASSIFICATION OF THIS PAGE (WIen Dot Entered)

M2.
SECURITY CLASSIFICATION OF THIS PAGE(OWh D.C. E-tu..d)

ARI Research Note 87-16

20.Abstract (continued)

"-and creation of guides to the NTC instrumentation system.

Year two efforts will continue and expand the progress achieved, in large

part through the use of an integrating model of unit NTC performance developed

during the first year. This should enhance ARI's support for the development

and dissemination of lessons learned from the NTC.

I Acct~ iln For

JJ Li:. ,. .-

%, %

*''" . et .'2 t 1,,n/_ -''/, .. -. ,, " ~i L,) .tvt. Codes ._ _-"

A~..o .

1I1

UNICLASSIFIED)

-p 1 SECURITY CLASSIFICATION OF THIS pAc(Ih.. 0.e. Enl-rd)

-- f

-jig .

]
METHODOLOGY DEVELOPMENT FOR DERIVING LESSONS LEARNED FROM THE 'p

NATIONAL TRAINING CENTER: PROGRESS AND FUTURE DIRECTIONS

IN TR OD U C T IO N ..................... ............................. I

SECTION ONE: YEAR ONE OF THE CONTRACT .......................... 3

INTROD UC T ION ............................................. 4

TASK 1: Develop and Apply Procedures for

Identification and Correction of Erroneous

Data from the NTC .................................. .

TASK 2: Develop and Apply Concepts and Methods for

Measurement and Interpretation of Unit

Performance ..................................... 12

TASK 3: Deelop Guides to the Use of NTC Information .... 17

TASK 4: Determine Requirements for More Effective

Integration of NTC and Home Station Training .... 23

SECTION TWO: YEAR TWO OF THE CONTRACT ......................... 26

INTRODUCT ION ............................................. 27

TASK 6: Analyze Unit Performance ........................ 30

TASK 7: Develop Products to Support Training and ,

Feedback at the NTC...............................35

TASK 8: Develop Products to Support Home Station

Training and Training Assessment ................. 41

TASK 9: Develop Products to Support Feedback to

Army Users and Army-wide Utilization of the

Products of the Research ........................ 49

TASK 10: Prepare Proposed Third Year Research Plan ..... 54

TASK 2A: Develop Concepts and Methods for the Measurement

and Interpretation of Unit Performance .......... 55 . -

TASK AND TIME MATRIX ............................................. 67

* . . - . - . - - € 0. ., . . . , - , , ,, , , . - W .4 :
LIST OF FIGURES

Figure Page'

-1 Task 2 Seauence of Subtasks ........................ . 12

1-2. Task 3 Sequence and Relationship of Subtasks ......... 17

1-3. Subtask 3.1 -- Develop NTC Lessons Learned Strawman.. 20

1-4. Task 4 Sequence and Relationships of Subtasks ........ 24

II-i. Unit Combat Effectiveness Model (UCEM) ................ 29 r

11-2. Methodology for Producing Products Which Meet

U s e r N e e d s ........................................... 5 1

11-3. Developing Feedback Based on Mission Effectiveness

Criteria ............................................. 56

11-4. A Model for Measuring Mission Outcomes ............... 57 ."K

11-5. Example of Mission Performance Standards ............. 58

11-6. Example of Performance Measurement ................... 59

11-7. Example of Criterion Performance Effectiveness

I n d ic e s .............................................. 6 0 N 1.

11-8. Measuring Orqanizational Processess at NTC ........... 61

11-9. Example of Structure for Measuring Performance of

Mission Critica l Tasks ............................... 62

i%

.. !

i v V.

I""'.
METHODOLOGY DEVELOPMENT FOR DERIVING LESSONS LEARNED FROM THE

NATIONAL TRAINING CENTER: PROGRESS AND FUTURE DIRECTIONS

EXECUTIVE SUMMARY --

Requirement:

The U.S. Army Research Institute (ARI) provides research

support to the Combined Arms Training Activity's (CATA) mission

at the National Training Center (NTC). Ft. Irwin California. As part of this support. ARI has undertaken a multi-year research,.

effort to investigate new methodologies for the measurement of

unit performance and for the derivation and dissemination of

Lessons Learned at the NTC. In this regard, ARI has contracted

with The BOM Corporation to perform a three year investigation

entitled: "Research Support for a Unit Home Station Training and

Feedback System". One of the requirements of this contract effort

is to provide an annual synopsis of progress achieved and to

present a research plan for the subsequent year.

Procedure:

Each of the four first year tasks were reviewed to document

progress against its intended outcomes. This review focused on the

intended task outcomes, factors which influenced task performance,

documentation of completed activities and resultant products.

Based upon the progress achieved in the first year and the

intended outcomes for the second year tasks, a detailed plan for

each task was prepared. A description of the purpose of the task,

an understanding of the task. the methodology for task

accomplishment and a summary of anticipated products were

prepared. Milestones charts and personnel loading information were

also derived.

Findings: :

Twenty-two research products or reports were produced as a

result of the activities performed as part of the Year One

contract effort. These represented considerable progress in the

areas of unit performance measurement, development of Lessons

Learned, documentation of NTC data quality, and guides to the NTC

instrumentation system.

The Year Two research efforts will continue and exoand the

progress achieved in the first year. In large part this will be

Ve. facilitated through the use of an integrating model of unit NTC

-A performance developed in the first year. It is anticipated that

the results of the Year Two efforts will further enhance ARI's

support to CATA's mission to develop and disseminate Lessons from

the NTC.

• =.V

k..
Ii

Utilization of Findings:

This document serves a dual purpose. First, it summarizes and

documents the progress and products of the first year's contract

effort. Second, it proposes a plan of activities that continues

the accrual of expertise and information begun in Year One.

IJ4*

I. p

viw

-V-7 -
INTRODUCTION

n February of 1985. 8DM Corporation was contracted by the

Army Research Institute (ARI) to conduct a three year research

effort in support of ARI's mission as primary research agency for

the Army at the National Training Center (NTC). Performance of

this contract required that BDM successfully complete five tasks

in each of the three years of the contract. The structure of these

tasks was such that the output from each year's effort feed

directly into the conduct of the next year's tasks. The mechanism

identified in the contract for the incorporation of the previous

year's efforts and results into the plans for the succeeding year

was a research plan for the future activities. The present

document represents the research plan for the second year of the

contract effort and is the contract deliverable for rask Five of

the first contract year.

Purpose of Research Plan

The research plan has been prepared in response to the

requirements of the RFP, the BOM proposal, and the events of the

first year of contract performance. As such, it represents both a

summary of past events and intended outcomes as well as a description of future objectives, anticipated activities, and products.

Specifically. this document addresses three primary topics:

1. T he accomplishments and deliverables of the First Year of

the Contract

2. The impact of these accomplishments on the second year

program.

3. The revised plans for the second year.

Organization of the Plan

To facilitate the presentation and discussion of these topics,

the plan has been organized into two sections with supporting

appendices. Each section focuses on the activities of a specific

contract year. Thus the first section contains a aescription of

the Year One activities while the second section focuses on Year

Two of the Contract. The structure of each of the two sections is

similar in that the sDecific contract tasks are addressed

separately. however, the purpose and nature of the discussion

differs considerably from the first section to the second section.

In the first section, Year One of the Contract, the

discussion for each task provides:

o A statement of the intended outcomes for the task as oer

the orginal SOW and BDM proposal.

1.,.

. . .;::';: v .-.', -'-.'; -> . - > ,- .-. ..---- --,.----.. - ..- ..- .. . ,:- ;-, .. . .. l... . . . . . . -: - .-- :, •
o A delination of events or circumstances which influenced the

actual performance of the task;

o A description of the progress made during the first year on

the task;

o A listing of all significant products and reports produced in

the performance of the task.

Preceding the Year One task descriptions is a brief

introduction presenting the thrust and purpose of Year One. This

sets the stage and provides a context for the individual task

activities so that an assessment of progress can be meaningfully ,

be derived. It is anticipated that the above structure provides a

comprehensive review and assessment of the first year contract

* efforts and its results.

The second section of the document, Year Two of the Contract,

focuses on the performance of Year Two activities. By design and

* requirement, this section incorporates the results of the first

year as an input into the framework contained in the orginial SOW

and proposal. Thus, the activities plannned and projected for the

second year have been shaped and conditioned by the first year

achievements and events. The effect of this approach is

immediately apparent in the introduction to the second section

where a model for measuring unit combat performance, an outcome of

Task 2 in Year One, is presented and used as the unifying

conceptualization for all Year Two activities. The specific tasks

to be conducted in Year Two can be seen both in terms of their

relationship to an overall model of battalion NTC performance and

their relationship to each other.

Each Year Two task is presented separately in the second

section of the plan. For each task, the discussion includes:

o A statement of the nature of the task

o A statement of the understanding of the task

o A proposed set of activities to address the task

o A delineation of anticipated products

In support of the discussion of Year Two tasks and

activities, a staffing and resource chart is included. The chart

provides the level of effort and mix of personnel for each task.

-'pm

Finally, a separately bound set of Appendices has been

prepared as a supplement to the plan. These Appendices include

copies of all Year One reports and products. Due to their

considerable volume, the Appendices are not included here.

Rather. references to specific Appendices have been provided for

each product or report. It is felt that this organization will -

not deter the ability of the reader to understand the described

research effort.
'-o

-W %4

Year One

of the

Contract . o ,p '

3 I
INTRODUCTION

As described above the initial section of the research plan is

intended to provide a description of the progress achieved in the

first year of the contract To facilitate an understanding of this

progress and the activities undertaken, it is important to

understand that there were three primary thrusts to the Year One

effort

* 1. To investigate and establish the quality of the data from

the NTC,

* 2. To design and develop a model of battalion combat performance at the NTC and the determinants of that performance;

3 To identify user needs and then develop products and

reports using NTC data and also to establish a methodology

for future application of NTC data.

The Year One tasks were shaped and performed to maximize the

4benefit to be accrued to the Army in the process of accomplishing

these objectives. Evidence of this benefit is found in the partial

list of Year One products presented below:

Task 1: Develop and Apply Procedures for Identification and

Correction of Erroneous Data from the NTC

Program GOETAP Documentation

Capability to Analyze NTC Data

NTC Data Library

Comparison of National Training Center Digital Data Sources

Task 2: Develop and Apply Concepts and Methods for Measurement

and Interpretation of Unit Performance

An NTC Live Fire Performance Analysis

A Briefing on a Concept and Methodology For Measuring Unit

Combat Effectiveness at the NTC

A Research Plan for Developing Concepts and Methods for

Measurement and Interpretation of Unit Performance

A Briefing On a Preliminary Analysis of NTC Force-on-Force

Performance

A Preliminary Analysis of NIC Force-on-Force Performance

4

" -'- --'- -- - ~ . .. . ... .. . .- L- .
Task 2 (Cont'd):

A Briefing on the Performance of Battalion Task Forces on the

Live Fire range at the National Training Center (NTC)

A Research Plan for Measuring Unit Performance Effectiveness

National Training Center Research Issues

Comparison of the BFVS and M113 Equipped Battalion Task Forces

on Live Fire Performance at the NTC

Task 3: Develop Guides to the Use of NTC Information

A Detailed Description of the NTC System Initialization

Procedure

Lessons From the NTC

Interview Guide: Unit Home Station Training and Feedback

System

A Briefing on the Use of Instrumentation to Improve Combat

Readiness

Research Plan for the Evaluation of Acoustic Quality and

Research Potential of Communication Recordings from the

National Training Center (NTC)

The DeAnza Primer: A Basic Introduction to the DeAnza Graphics

Display

Documentation for "What Now. Captain?": A Training Concept

for Exporting Lessons Learned from the National Training Center

A Method of Analysis for the Investigation of the Bradley

Fighting Vehicle at the National Training Center

Task 4: Determine Requirements for More Effective Integration

of NTC and Home Station Training

Research Plan for the Evaluation of the Requirements for the

Effective Integration of the National Training Center

(NTC) and Home Station Training

".4"
The remainder of this section is organized arojnd the four

tasks of Year One. A brief description of the tasks is provided

below:

Task 1 -- Develop and Apply Procedures for Identification and

Correction of Erroneous Data from NTC: This task

has as its central concern the investigation of the

quality of the digital data from the NTC. As part

of this effort. several comparisons were undertaken

to determine the relative accuracy of varying data

sources.

Task 2 -- Develop and Apply Concepts and Methods for

Measurement and Interpretation of Unit Performance:

Defining and Measuring Battalion Combat

Effectiveness was the central issue for this task.

The work undertaken in Year One resulted in the

development of a new systems model of combat

effectiveness.

Task 3 -- Develop Guides to the Use of NTC Information: Work

on this task followed two different research

strands. One strand led to the creation and

production of a prototype "Lessons from NTC" The

other strand was concerned with the preparation of

guides for use at the NTC by personnel working with

the NTC instrumentation system.

Task 4 -- Determine Requirements for More Effective

Integration of NTC and Home Station Training:

Capitalizing on the model developec in Task 2 and

the result from "Lessons from NTC". the effort

undertaken in this task provided a research plan

for implementation in Year Two. The focus of this

plan was Home Station Training factors related to

NTC performance.

The discussion of each task is intended to provide a thorough

assessment of the conditions surrounding the performance of the

task, its progress relative to these conditions, and its products

In providing this assessment, the nature of each task should be

clear as well as its contribution to the achieveme't of the

overall objectives for the first year.

* 6 4
,- - .- - - .- - - w'u ws-- - - , 

:.".

1.1 Task 1 -- Develop and Apply Procedures for Identification and

Correction of Erroneous Data from the NTC

1.1.1 Intended Year One Outcomes

Four subtasks were identified in the Proposal for this task: Z,.

(1) Prepare a Research Plan for Examination and

Selection of NTC Data,

(2) Develop Methods to Review and Annotate Data,

(3) Design and Implement Macro-Level Raw Data

"Cleaning" Methodology, and

-" (4) Produce Report Describing Methodology for Performing

the Task and Providing Recommended Procedures for -

Routine Use.

*1.1.2 Factors Influencing Task Performance

"- Three major factors caused a major redirection in the Task 1

effort from that originally proposed and presented above:

5.' t(1) The emphasis by ARI changed to short, quick-reaction

analyses. As a result. Task 1 resources were placed in support of

repairing the NTC Database Research System and imolementing

procedures to allow it to be used more easily.

(2) The lack of documentation for the NTC archival data

made any substantive analysis impossible without a major analysis

effort. In ef'ect it would have been necessary to apply "reverse

* engineering" techniques to the software in order to decipher the

formats of the input files.

(3) The discovery of an alternate digital data source

(the Range Data Measurement Subsystem log) was considered

important enough that a detailed analysis of that source was added

to the Task I schedule.

1.1.3 Descriotion of Year One Progress

As a result of the factors listed above, the subtasks

actually performed during Year 1 included:

(1) Achieve operational caoability for the NTC Database

Research System (NTCORS).

(2) Identify and investigate additional sources of NTC

data. including Take Home Packages and the ROMS log tapes, and

7'

SNN
(3) Initiate the process of redesigning the NTC database

to more capably support ARI-POM research objectives.

1.1.3.1 NTC Oatabase Research System {NTCORSI

The NTC Oatabase Research System (NTCDRS) is a contractordeveloped (SAIC) system which was designed to convert the log

tapes received from the NTC into a form that would facilitate

analysis and research. It was implemented as a series of computer

programs which culminated in a series of relational database

tables that could be accessed in combination to provide simple .

reports, including rudimentary statistics.

As implemented, the system was deficient in several areas:

(1) It was limited to the creation of one database, which

included the data from one mission in one rotation. This

limitation resulted from hard-coding the database name (ARIDATA)

in the program.

(2) The system was prone to fail catastrophically at a number

of points.

(3) Operation of the system was invoked by use of a number of

command files which were badly documented, awkward to use, and

error-producing.

As of 1 September, 1985, the NTCDRS was operational. This

was achieved by overcoming the deficiencies listed above by:

(1) Making the database name an input option. This allows

the generation of a database for each mission segment in each

rotation. A naming convention was selected which provides for a

unique (and standard) name for each mission segment processed. J,

(2) Each problem which caused a catastrophic failure was

analyzed and overcome. A variety of methods were used, including

generation of short FORTRAN programs, the use of system editors,

and judicious deletion of garbled data.

(3) New command files were generated which allow for much

more straightforward processing. In addition, fixes for some of

the more persistent problems were incorporated into the command

files to eliminate the requirement for operator intervention.

In addition to the steps listed above, approximately thirty

utility programs and command files were developed to allow for

specialized processing needs, such as reloading specific tables,

documenting database contents, and dumping intermeciate files.

Since production was initiated, more than 500 mission

segments have been processed by NTCORS and are available for use. .

8

8 |9.

- e * *" - -'- - , t ' " ' " " " ' " "
1.1.3.2 Additional NTC Data Sources

The identification and investigation of additional data

sources from the NTC was intended to ensure that future ARI-POM

research efforts would make use of the full range of data

available from the NTC. The ultimate outcome of this effort would

be the incorporation of the additional data into a redesigned NTC

data base. Part of the accomplishment of this job was the

identification of all NTC data sources available, an inventory of

all NTC material available at ARI-POM, and establishment of an NTC

Library at ARI-POM. .

The next step included a more in-depth analysis of the data

contained in sources not previously examined, and a determination

of what, if any, data from the "new" sources was relevant to the fig-,

upcoming research program that was not already present. .4

The initial effort in this regard resulted in the preparation

of two reports concerning NTC battalion performance, based on data

from the take home packages (THPs), one of the primary sources of

data routinely produced at the NTC. Based on the utility of THP

data. they should be incorporated into the redesigned data base.

Selection of THP data to be included, designing their format, and

developing a means of incorporating them into the redesigned NTC

data base are all steps properly included in the data base design

and development task.

Another NTC data source was also examined in detail: the log

tapes from the Position Tracking Computational Component (PTCC) of

the Range Data Measurement Subsystem (RDMS), referred to as RDMS

log tapes. These data, and their comparison with data from the

NTCDRS, were the subject of a BOM report which is summarized as

follows. First, analysis indicates that raw data from the ROMS r

log tape are transmitted to the Core Instrumentation Subsystem

(CIS) and logged with a great deal of fidelity. Second, there are

critical data on the RDMS log which are not available from the CIS

log. Third, software has been developed which allows processing

of the RDMS log, and will facilitate incorporation of selected

data into the redesigned data base.

While some preliminary analysis has occurred, detailed

specification of additional data to be included in the follow-on. "

data base is properly a function of the data base design effort.

1.1.2.3 NTC Data Base Redesign

The initial design of an NTC database, as implemented by the

NTCDRS, prepared by SAIC, has certain shortcomings:

(1) The data contained within the NTCDRS databases are

adequate to support only a small percentage of the research

efforts possible with access to all NTC data.

V"%

9

,K,
(2) The data within the NTCORS tables are arranged so that ",.

they are awkward to use. The development of useful database

queries is difficult and time-consuming.

(3) Design of the present database structure is deficient, in

that there are multiple occurrences of identical data, awkward

data relations or keys, and common inclusion of data that can be

derived easily from other tables.

The remedy to the problems discussed above is to redesign the

data base so that it properly supports the NTC research objectives

of ARI-POM. This is a fairly long term process which must consist

of the following steps:

(1) Requirements Analysis. This step specifies the data

base, in terms of which data are required, including explicit

definition, accuracy, units, and derivation, if necessary.

(2) Design. This phase allocates the requirements into data

base terms, including choosing the database structure down to the

table structure, data relations (i.e. the specific data tables),

and keys. Also included in the design effort are complete

specifications of all data sources, down to explicit format

information, and design of the processes which will be necessary

to incorporate the data into the new data base structures.

(3) Development. During this phase, the processes necessary

to build the new data base are developed. The processes might

include computer programs, command files, manual entry, data

verification, derivation of some data elements, etc.

(4) Implementation. This phase provides the transition into

a production status, and ties all of the separate entities P

developed in the previous step into a working system. In the

ARI-POM environment, the implementation also includes the retrofit

of data presently in the NTCDRS format into the new data base

format.

The previous discussion serves as background to the totality 1

of the database definition process. While the process. as

outlined, appears to be a sequential and logical progression to a

logical conclusion and product, in reality the process had

elements of serendipity, since the efforts documented in the first

two subtasks of this document contributed to the design process.

Likewise, it must be assumed that the residue of efforts expended

in the production of ARI-POM research reports can and will

contribute to the requirements definition.

What is necessary is the formalization of the process.

without which any data base redesign will result 7n a substandard

product. The completion of the data base design effort is

projected as Task 6 of Year Two of the Contract.

10 "

:.-.. ,-,. .., .,.:...:.... .. .... .- . .... .. . ...: . . .. %. .,.. V.. . . .. ,. . . . .. ,.., . . . ,.. .: .
1.1.4 List of Significant Reports and Products

A list of the four reports produced in the process of

performing Task One appears below. Copies can be found in the

indicated appendices.

1.0 Program GDETAP Documentation -

2.0 Capability to Analyze NTC Data

3.0 NTC Data Library

4.0 NTC Data Comparison: RDMS vs. CIS

V.

-7-

-

4.

4"-., 4(,**4
1.2 Task 2: Develop and Apply Concepts and Methods for

Measurement and Interpretation of Unit Performance

1.2.1 Intended Year One Outcome. The development of a method for

measuring and interpreting the performance of critical tasks for a

sample mission conducted at the National Training Center (NTC) was

the primary intended outcome for this task.

This outcome was to be achieved through the

accomplishment of the following seven subtasks in the order shown

in Figure I-1:

Subtask 2.1: Prepare a Research Plan for Task 2

Subtask 2.2: Describe Mission Essential Tasks for the Task Force

and for each Major Element of the Task Force

* Subtask 2.3: Identify Measures of Performance (MOPs) and

Measures of Effectiveness (MOEs) for use with NTC

Data for Each Activity or Task

Subtask 2.4: Develop and Apply NTC Data Files as well as

Extraction and Analysis Procedures for these Tasks

and MOPs/MOEs

Subtask 2.5: Produce Sample Performance Analyses and Profiles

Across and Within Battalions

Subtask 2.6: Develop Integrating Concepts, Procedures, or Models

for Interpreting the Performances

Subtask 2.7: Produce Reports

7! •.1 °'p

Figure I-I. Task 2 Sequence of Subtasks

1.2.2 Factors Influencing Task Performance.

'p p

As a result of specific instructions and guidance from ARI,

satisfaction of near term research requirements became a major

focus for this task. This change in emphasis was necessitated by

issues that had been raised after the contract was let concerning

the utility of NTC data for research. Therefore, the COTR (Dr.

Banks) determined at the beginning of the contract period that

12

s
immediate efforts should be expended to determine the usability

and value of NTC data by producing reports in the short term,

researchable issues of military significance which would have

interest for senior Army leadership. With this guidance, we began

to research the data sources to determine the feasibility for

producing such reports. The Take Home Packages appeared to offer

the most complete and readily useable data for conducting the type

of research desired by the COTR.

1.2.3 Det:riRition of Year One Progress.

Due to the above direction from the COTR. we undertook

Subtask 2.5 first. This effort was followed by work on the

remaining subtasks. The progress made on each of these subtasks

is discussed below.

Subtask 2.5: Using the data from the Take Home Packages, we

were able to perform an analysis of the performance of 56

battalions on the live-fire range. A comparison was made of their

performance over time and between the two types of task forces

(Armor versus Mechanized Infantry). Possible causes for

performance changes were investigated and reported. A second

analysis was conducted of the performance of 64 battalions during

force-on-force engagement simulation offensive and defensive

missions. In this latter analysis, a comparison of the friendly

and OPFOR losses by type task force for tanks and APCs was

performed. The impact of J-series organizations versus H-series

organizations on performance results was also investigated.

Possible causes for variation in performance were studied and the

results reported.

These two sample analyses constituted partial

fulfillment of the original Statement of Work. Additional

analyses were conducted as part of Task 3 and were incorporated

into "Lessons from NTC" as further illustrations of performance

analysis possible with NTC data. What remains to be accomplished

are sample performance analyses based on a model/methodology

designed to measure the performance of mission essential tasks.

Subtask 2.2: A preliminary effort was undertaken to identify

the mission essential tasks for the task force for the two

defensive missions of "Defend in Sector" and "Defend from a Battle

Position" and for two offensive missions of "Deliberate Attack"

and "Movement to Contact". FM 71-2J and military SMEs were used

to develop this preliminary list of critical tasks. It should be

noted that analyses were also conducted on mission essential tasks

as part of an "NTC Lessons" effort. This effort is described in

detail under Task 3 though its results were incorporated into the

Task 2 efforts.

Subtask 2.3: As part of the effort directed toward an

analysis of live-fire and force-on-force performance, as well as

development of "Lessons From NTC". initial development of measures

of performance and effectiveness was accomplished. Specifically.

13
J1..

measures of performance were identified which could be used in the

descriptive and inferential analyses conducted in these areas.

While these measures do not represent a complete or exhaustive set

of performance indicators, particularly in the area of critical

tasks. they do constitute a reasonable starting point for

measuring and understanding certain aspects of NTC performance.

Subtask 2.4: The problems indicated in the description of

Task I greatly constrained the potential application of the NTC

data files for extraction and analysis. However, two different

activities contributed to accomplishments for this subtask. First.

a data base was developed and used for both the live-fire and

force-on-force analyses. As described above, this data base

included the MOPs and MOEs identified and developed for these

analyses. Second, as part of the Task 3 effort, sample analyses of

fire control and fire distribution were performed using the NTC

data files. These analyses drew upon all data sources including

the graphic displays and the digital data from the CIS. The

resultant analyses employed both graphical and numerical summaries

of the NTC data and represent prototypes for future analyses of

mission performance. The extraction procedures created in the

performance of this effort have been documented and disseminated

to ARI for their use.

Subtask 2.6: In October 1985, a model was developed for

integrating and interpreting not only the critical task

performances but other organizational factors, such as cohesion

and leadership, as well as to show the relationship of these

variables to home station conditions. The model was produced and

briefed to the new commander of ARI, COL Henderson. A formal . -

report of this model and the integrating concept was prepared and

delivered to ARI in November 1985.

The COTR then requested a briefing be prepared for CATA

personnel which highlighted the integration and interpretation of

the mission effectiveness criteria and the NTC organizational

factors portion of the overall model. The briefing was conducted

in late November.

Subtask 2.1: In December 1985, a research plan, based on the

results and information obtained in the previous subtasks, was

prepared. The plan operationalized the portion of the integrating

model that has been briefed to the CATA personnel in November

1985. The research plan was presented to ARI in draft form in

late December 1985. It was subsequently revised and provided the

." basis for ARI's determination that because of CATA's interest in

application of the proposed measurement system, a change order

should be prepared to permit development of the proposed system

for all NTC missions rather than the prototype mission originally

called for in the contract. The plans for the first year of the

two year augmentation are discussed under Task 2A in the second

section of this plan.

14
A recap of the progress made on the Task 2 requirements follows:

Subtask 2.1. A research plan was completed and presented an

integrating model for NTC performance. The plan

included a level of effort that exceeds the original

Task 2 requirements. This was a result of

additional guidance from the COTR and an anticipated

augmentation to the original contract.

Subtask 2.2. Proposed efforts accomplished and exemplified by

"Lessons From NTC" report. Additionally, the

anticipated contract change order for Task 2 would

require a considerably expanded effort to be

performed in this area.

Subtask 2.3. Work done for Live Fire and Force-on-Force data

analysis as well as NTC Lessons' battalion

performance profiles.

Subtask 2.4. Work performed for Live Fire, Force-on-Force and

.,.7 NTC Lessons reports.

Subtask 2.5. Work accomplished for Live Fire and Force-on-force ,. and NTC Lessons analyses.

Subtask 2.6. D,'aft completed and submitted to ARI.

Subtask 2.7. See below for reports published. -, I.

9415,
1.2.4 List of Significant Reports and Products. As a result of

the above efforts, the following products were produced and are

included in this report under separate appendices:

5.0 Comparison of the BFVS and M113 Equipped Battalion Task

Forces on Live Fire Performance at the NTC

6.0 An NTC Live Fire Performance Analysis

7.0 A Briefing on a Concept and Methodology For Measuring Unit

Combat Effectiveness at the NTC

8.0 A Research Plan for Developing Concepts and Methods for j Measurement and Interpretation of Unit Performance

9.0 A Briefing On a Preliminary Analysis of NTC Force-on-Force . -

Performance

10.0 A Preliminary Analysis of NTC Force-on-Force Performance

11.0 A Briefing on the Performance of Battalion Task Forces on the

Live Fire range at the National Training Center (NTC)

12.0 A Research Plan for Measuring Unit Performance Effectiveness

13.0 National Training Center Research Issues

16,
1.3 Task 3 -- Develop Guides to the Use of NTC Information

1.3.1 Intended Year One Outcomes

The purpose of this task was to develop and produce guides to

serve as day-to-day reference documents for scientific and

military users of NTC information.

The potential impact of the NTC data on the military and

scientific community is enormous. The realization of its full

potential is, however, severely challenged by the lack of a guide.

or program. that would allow analysts to recognize what kind of

data is available and how to properly interpret the data.

The proposed approach for accomplishing the requirements for

Task 3 shows it to be a linear activity (Figure 1-2).

It was to begin with subtask 3.1, which required the

determination of major scientific and military community users

(and needs) for an NTC information guide. This information was to

become input to Subtask 3.2, during which the guides would be

designed and developed. In Subtask 3.3, the contractor would be

required to produce the requisite guides.

~E1

Figure 1-2. Task 3 Sequence and Relationship of Subtasks

The methodology for accomplishing this task would be a

straight-forward application of the three subtasks.

1.3 2 Events Influencing Task Performance

The initial activity for this task was the solicitation of

information from potential NTC data users about their needs.

Meetings were scheduled and conducted with the Infantry School at

Fort Benning. the Armor School at Fort Knox, and the Combined Arms

Center (CAC) at Fort Leavenworth concerning this issue. Feedback

from these meetings indicated a need and interest in NTC data but

17
lacked specificity about the form and content of tne information.

This situation was clarified at the TRADOC/FORSCOM coordination

meeting with ARI at CAC in the first month of the contract. It was

jointly determined that the Combined Arms Training Activity of CAC

should be provided with an example of how to use NTC data. The

media for this action was to be transmitted in the format of

"Lessons Learned." On March 15, 1985. the ARI COTR approved this

Task Three action plan to develop a "Lessons Learned" strawman.

The purpose of the task stated that a strawman was to be developed

for CAC which reveals lessons and insights derived from NTC data

and methods for extraction and synthesis of data into lessons and '

insights. The intended customers were to be brigade commanders

and proponents. Lessons and insights using NTC data and

experience were to be provided to brigade commanders and

proponents for the purpose of improving training, doctrine.

organization, and equipment within the Army. "Lessons Learned"

were to be written in classical form patterned after historically

acceptable, precedent setting documents. The focus is on those

areas that the brigade commander can influence by training,

organization. and training support conditions and resources. Some

"lessons" will require action by a proponent other than the

brigade commander. Recommendations of the "lessons" address

specific actions that the proponents should take considering the

particular "lesson" that has been identified. Provided within the

text are graphics, explanations, and examples of methods used to

synthesize information to assist CAC in developing follow-on

"Lessons Learned" documents.

On April 2. 1985. an In Progress Review was held at ARI.

Presidio of Monterey, California. The approved concept with

specific examples and insights was briefed at this meeting.

In addition to the focused task plan described above, a

second strand of effort was identified for Task 3. In response to

a request from the Operations Group at the NTC, activity was

initiated to design and produce training materials (guides) for

the NTC instrumentation system.

1.3.3 Description of Year One Progress

The detailed purposes of the tasks were: 1) to proliferate

the data and experiences emanating from the NTC into usable

lessons: and 2) to prepare guides for the use of NTC

instrumentation.

The Army has near term and long term needs fcr NTC data and

information. The Army as a whole has not profitec fully from the

experiences gained by units that have trained at !-e NTC. Three

years of training activity at the NTC have producec feedback to

FORSCOM units during this training period, Training Observations,

RED Thrust articles, Combined Arms Training Tips, and articles in o

TRAINER Magazine and service schools periodicals. These efforts.

however worthwhile, do not fully realize the capatlity of the NTC

instrumentation system and satisfy the needs of T;ADOC and other

18

.. . . . . . . . . . .
MACOMS. Because NTC capabilities have not yet been fully

exploited. opportunities for change in the way that the Army

trains, organizes. fights. and equips itself have been missed.

The major problem associated with the exploitation and

proliferation of NTC data, information, and experience is the lack

of knowledge among potential users. To remedy this problem it was

felt that an external source knowledgeable of NTC capabilities

must demonstrate the usefulness of NTC data, information, and

experience in a particular user's area of proponency. It was

determined that the preparation of NTC Lessons as a part of Task 3

would satisfy this condition.

The Army Research Institute (ARI), Presidio of Monterey.

California (POM) assisted CATA by developing a prototype or

Strawman "Lessons Learned" document which served as a guide for

semi-annual "lessons" which CATA will produce.

The methodology used for developing the "strawman" is shown

in Figure 1-3. "Lessons" employed all available NTC information

q_ to identify strengths, weaknesses, and voids in training,

doctrine, organization, and equipment. The following NTC products

were reviewed and analyzed to determine unit strengths and

weaknesses and voids in the support system:

History segment;

Task Force After Action Reviews;

Command communication tapes;

Segment video tapes;

Engagement Summaries;

Unit orders;

Research data base file reports; and

Unit after action reports.

The vast amount of available data on hand as a result of

three years of NTC training precluded the identification of every

trend supported strength, weakness, or void withir the "Lessons

Learned" strawman document. Also, in the interest of early

delivery of a product to CATA the content within t'e document was

" necessarily limited. Major critical events and operational and

organizational procedures were identified for analysis. These

were identified based upon a comprehensive review of NTC data and

information which determined the degree to which t-e NTC data and

information supported identification and analysis of unit

" performance.

19 ..

°.

%.7-."
zz

0

* w*

Na V

co

10
Strengths and weaknesses, and voids which address doctrine.

training . organization. and equipment were categorized within the

task force operating systems. These systems are maneuver, fire

support, air defense, intelligence, mobility - countermobility,

combat service support, and command and control. A separate

category of Nuclear. Biological, and Chemical (NBC) was also

included.

"Lessons Learned" was written in "Classical" form patterned

after historically acceptable precedent setting documents. The

focus was on those areas that the brigade commander could

influence by training. organization, and training support

conditions and resources. However, some Lessons required action

by a proponent other than the brigade commanders. The

recommendations of the "Lessons" addressed specific actions that

the proponents should take concerning the particular "Lesson" that

had been identified.

The accomplishment of this task resulted in a model document

from which CATA, CAC could develop and publish NTC Lesson Learned

Documents (and CATA is now doing so).

1.3.3 Development of NTC Operating and Training Products

The second strand of effort in Task 3 was concerned with the

design and development of products which meet the training and

operating needs of personnel in the Operations Group at the NTC.

As a result of a direct request from the Deputy Chief of

Operations Group (NTC). "The DeAnza Primer: A Basic Introduction

to the DeAnza Graphics Display" was developed to assist in the

training of incoming Training Analysis and Feedback (TAF) Center

Core Instrumentation Subsystem (CIS) personnel. "The DeAnza

Primer" is an instructional document which presents a series of

lessons and hands-on exercises that are designed to introduce new

TAF/CIS personnel to an integral component of the CIS electronic

monitoring equipment. the DeAnza Graphics Display tablet and

mon i tor

The lessons are presented in the form of a narrative tutorial

with hands-on exercises which enable incoming TAF/CIS officers to

learn the basic skills necessary for operation of the graphics

tablet The manual is comprised of six sections. Each section

covers a different set of related functions. Sections are

organized such that skills are learned sequentially (i.e.. skills

learned in Section I are required to learn the skills taught in S

Secticn 2) The exercises may be completed in a four and one half

our block of time and although it is necessary for the system to

be primed (i e . set up for the first exercise), close

supervision of the learner is not necessary.

Another product was also developed for the NTC Operations

3*o c This product was viewed as a management tool and focused

or tne steps involved with the initialization of the NTC

ist entation system The resultant report from this effort

21

' .-v.. .." ,i.i•" -"-..', .-L'.. ' .i -.".. -.".. - ' ' '..'.L , '. ',\- ,r ,= : '-.'Z ,'" r-:-

.-',-,
IN -------- r-rr

described not only the specific steps in the initialization

process but provided samples of all necessary forms and

indications of the roles of key personnel and groups. The report

was delivered to ARI in June 1985.

Also in November 1985, as a result of a coordination meeting

with the CATA chief of the Army Lessons Learned Cell, Fort

Leavenworth, BDM conceptualized a demonstration of a high

technology, NTC Lessons Training technique. The effort is titled

"What Now Captain?" and uses NTC instrumented historical battle

segments as a teaching tool. A demonstration tape was prepared

for the Commander, CATA, in February, 1986.

1.3.4 List of Significant Reports and Products

Products and reports which were produced under Task Three are as

follows:

14.0 A Detailed Description of the NTC System Initialization

Procedure

15.0 Lessons From the NTC

16.0 Interview Guide: Unit Home Station Training and Feedback

System

17.0 A Briefing on the use of Instrumentation to Improve Combat

Readiness

18.0 Research Plan for the Evaluation of Acoustic Quality and I

Research Potential of Communication Recordings from the

National Training Center (NTC)

19.0 The DeAnza Primer: A Basic Introduction to the DeAnza

Graphics Display

20.0 Script for "What Now, Captain?"

21.0 A Method of Analysis for the Investigation of the Bradley

Fighting Vehicle at the National Training Center

22

- ..
'I "A

1.4 Task 4 -- Determine Requirements for More Effective

Integration of NTC and Home Station Training

1.4.1 Intended Year One Outcomes

The purpose of this task was to examine current Army training

practices in order to identify strategies for:

1. Effective integration of NTC and home station training; and

2. Assessment NTC training benefits derived from current

practices.

One of the main goals of the NTC is to provide units with a

training environment which was more realistic and more combat

like than the facilities at a typical home station post. The NTC

provides a live and dedicated opposing force and instrumentation

for real-time casualty assessment which adds a strong element of

realism to unit training. Evidence that NTC has achieved this

goal is found in the reports of participants and trainers who

* describe the NTC experience as a difficult one which substantially

challenges the abilities of a unit and its leaders.

The fact that NTC is such a challenge can have either

positive or negative impacts on the readiness of a unit, depending

on the abilities of the unit itself. The training literature

clearly shows that training exercises must be presented at an

appropriate level of difficulty. If an exercise is too difficult,

little or no training benefit will occur. Instead trainers become

frustrated and demoralized, which may actually decrease readiness. 4.

Attempting to match NTC exercise difficulty to unit abilities lit

could be done in two ways. The difficulty of NTC exercises can be

varied, or the unit can be pre-trained up to an appropriate level.

However, any substantial reduction in NTC exercise difficulty to

accommodate poorly trained units would compromise the goal of

combat realism. It makes more sense to train units up to a level

which allows them to gain the most from their NTC experience and

maintain the level of NTC exercise difficulty as a standard which

units can train for. In this context, the effective integration

of NTC and home station training becomes an important goal. " 6

Current practice for integration of NTC and home station

training vary widely throughout the Army. Although the Army, by

means of a FORSCOM circular (359-83-10), prescribes certain

minimum NTC training activities, the actual extent of NTC/home

station training integration varies considerably because of local

training priorities, resource constraints, and assigned readiness

levels. For example, the 24th ID has a division training policy

designed to prepare units for NTC. Other units incorporate their

NTC experience in a less formal manner (e.g., using it to modify

*- their local training objectives), while still other units do not

explicitly address the NTC experience in their home station

training.

923

.. ;.. ,1.
Task 4 was to be conducted in a sequenced series of its

subtasks as shown in Figure 1-4.

'.0

4.1 .4 4. 4. 4.3 

Figure 1-4. Task 4 Sequence and Relationships of Subtasks.

In this approach, each subtask becomes a critical input to

its subsequent subtask. First, a research plan (Subtask 4.1) must

be developed to consider pre-NTC, NTC, and post-NTC training

conditions. This plan was to drive the assessment and analysis

efforts which would start with the development of the data

collection instruments and procedures (Subtask 4.2), then move on

to the actual collection of the data (Subtask 4.3), then analysis

of the data (Subtask 4.4), and ending with a final report (Subtask

4.5) on this task.

1.4.2 Events Influencing Task Performance

The necessary operating agreements between the Army Research

Institute - Presidio of Monterey and the Combined Arms Training

Activity (CATA) at Ft. Leavenworth Kansas were not completely

negotiated and signed until the last calendar month of 1985. This

set of agreements provides the necessary Army support so that ARI

can arrange for field support at the NTC during unit training

rotations and with units at home station prior to and after the

rotation. Without this agreement the unit contact aspects of this

task could not take place. This problem prevented execution of

Task 4.

Now that program support and a signed letter of agreement

have been completed the formal barriers have been removed. The

research completed in the first ten months has now provided a real

platform from which both tasks eight and nine can be completed

during the second year.

1.4.3 Description of Year One Progress

In executing Task 4. BDM performed the following activities.

Initial efforts concerned the identification and

investigation of two major research areas: unit training

activities and training-related conditions. These efforts were

24

. t,*U e -
'.1

aimed at dimensionalizing each area for possible use in the

conduct of the investigation of Home Station Training. The

activities undertaken in this report involved the examination of

the issue by several NTC SME's. The result was an identification

of factors or areas where the experts felt home station training

could translate into better NTC performance. The selection of

factors were restricted to those in the training activity area.

The second major area of activity for Task 4 was the

preparation of a research plan to address home station training rw

before and after deployment to NTC. A preliminary focus of this

plan was on the interrelated conditions impacting upon home

training. The conditions that affected the unit's ability to

train both at home station and at the NTC were seen as critical to

understanding effective integration of the NTC experience into

Home Station Training. The research plan for the original subtask

addressed both these conditions and the unit training activities.

1.4.4 List of Significant Reports and Products

22.0 Research Plan for the Evaluation of the Requirements for the

Effective Integration of the National Training Center

(NTC) and Home Station Training

C,.

4-%

I'..

25

....
',

.'

.

"->

.426

Yea, . o 4
INTRODUCTION

This section of the research plan presents a description of

the tasks and activities to be undertaken in Year Two of the

contract. Six tasks will be performed. The first five of these

correspond to the Year Two tasks from the SOW (Tasks 6 through

10). while the sixth is an expansion of Task 2 Year One being

conducted under as part of a contract augmentation. A brief

description of each task follows:

o Task 6 -- Analyze Unit Performance: This task has as its major

focus the design and creation of a research data

base for the NTC. As part of this effort, guides to

the use of this data base will be developed.

o Task 7 -- Develop Products to Support Training and Feedback at

the NTC: This task has as its primary focus the

design and development of guidelines for the After

Action Review (AAR) and the Take Home Packages (THP)

at the NTC. To accomplish this, the effort will

address the exploitation of the advanced automation

capabilities and the content and format of these

feedback devices relative to the measurement system

being developed in Task 2A.

o Task 8 -- Develop Products to Support Home Station Training

and Traini g Assessment: This task involves the

development and field-testing of data collection

instruments and techniques for use in describing

Home Station Training. The instruments and

techniques will draw upon the measurement system

being developed in Task 2A and upon a model of the

Army Training Process created as part of this task.

o Task 9 -- Develop Products to Support Feedback to Army Users

and Army-wide Utilization of the Products of the

Research: While the task above focused on Home

Station implications of NTC information, this task

is concerned with dissemination of NTC information

to the Army at large. As a first step in this

effort, a survey of users from across the Army will

be designed and conducted.

o Task 10 - Prepare Proposed Third Year Research Plan: This task

-_ parallels Task 5 of the first year. That is. it

requires the preparation of a plan which

incorporates the previous results of the contract

along with the originally proposed Year Three

activities.

o Task 2A -Develop Concepts and Methods for the Measurement and

9, Interpretation of Unit Performance: This task

represents an expansion and continuation of the work

bpgun under Task 2 in Year One. It involves two central

27
activities: the creation of a crite-on measurement

system for of combat effectiveness at the NTC and

the identification and measurement c4 critical tasks

at the NTC. The results of this effcrt will be used

as input to all Year Two tasks.

The accomplishment of the Year Two tasks wil' orovide several

important products to the Army and will greatly acvance ARI's

ability to measure and research battalion combat effectiveness.

Using the Unit Combat Effectiveness Model (UCEM) ceveloped in Year

One as part of Task 2, it can be easily seen how the Year Two

tasks advances the understanding of battalion performance at the - -

NTC.

Figure II-1 displays the UCEM model with the Year Two tasks Ww

indicated as to their major area of effort. Task 6. involving the

creation of a research data base, underlies all parts of the model

concerning NTC processes, performance, and factors contributing to

performance at the NTC. Task 7, concerning feedback methods at the

NTC, are depicted as part of the feedback loop in UCEM. Task 8,

Home Station Factors, is concerned with the model block entitled

"Home Station Organizational Factors". Task 9, Develop Products to

Support Feedback to Army Users, addresses the Army-wide

dissemination of NTC Lessons Learned and is concerned with the

furthest left portion of the model. Task 2A. developing measures

of unit effectiveness and critical tasks. will result in

specification and measurement of the right most block of the model

entitled: "NTC Mission Performance Factors". Task 10 will tie

all of these together for expansion and exploration in Year Three.

As can be seen from this figure and the indicatea Task Structure

overlay, the Year Two tasks represent a comprehensive research

effort of the UCEM model. In particular, it will -esult in a

measurement system for the criterion portion of t-e model. This S

will leave the relationship of Home Station and NTC organizational

and process factors to NTC effectiveness to be ex:i-cated in Year

Three.

Using the UCEM model as the unifying concept 'or the Year Two

efforts enables the individual research efforts to have a directed

and related impact on the problem of enabling the Army to

maximally benefit from the NTC. It also illustrates where the -

-ndividual tasks necessarily impact on each other and how they do

so. This relationship of tasks will further be ex:iicated in the

discussion of the individual tasks.

Tie orgarzation of the remainder of this sec:ion of the

-esearcn plan 7s ny task For each task. the disc-ssion will be

organized into four topics: purpose of the task. .nderstanding of

task, methodology fcr accomplishing the task. anc anticipated

reports and products. A milestone chart is Drov-:ed at the end of

each task discussion. At the end of the entire section, a

staffing ana ioadng chart is provided for all tasKs.

28

• "- . ". . . , ._ " - . ., , " , ' .' .• . o . ' . , , '. ..-. ,. ' . . . .'.'.
INI

EN Ott

22

E-4S

46p -

0 20c

U. ILJ

9;j 29LA V 0 %
11.6 Task 6 -- Analyze Unit Performance

11.6.1 Purpose of the Task.

The establishment of an effective NTC data base is key to the

.' analysis of Lessons Learned and the development of materials to

support unit feedback at the NTC. Completion of Task 6 will be a

major step towards this objective. Specifically, Task 6 is

concerned with the redesign, development, and incorporation of an ,

expanded NTC research data base, and documentation of procedures

to utilize the data contained therein.

11.6.2 Understanding of the Task

The initial design of an NTC database, as implemented by the

NTCDRS, has some serious shortcomings with respect to current NTC

operations:

4-.

(1) The data contained within the NTCDRS databases are

adequate to support only a small percentage of the research

* efforts possible with access to all NTC data.

(2) The data within the NTCORS tables are arranged so that

they are awkward to use. The development of useful database

queries is difficult and time-consuming.

(3) Design of the present database structure is deficient, in

that there are multiple occurrences of identical data, awkward

data relations or keys, and common inclusion of data that can be

derived easily from other tables.

In view of the model being developed under Task 2A, the

NTCORS will become obsolete without major changes and additions. Implementing even minor changes to the NTCDRS is difficult because

of the lack of documentation of the input files and the inherent

inefficiency of the table structures.

L.. The best way to produce a data base that is consistent with Zthe Task 2A model, that is useful in performing the kinds of

research that ARI-POM will be called upon to do in the future.

that can be adapted to meet the ever-changing challenges of the

research environment, and that can be maintained efficiently, is

to design and build a data base from the ground up.

11.6.3 Methodology for Task Accomplishment

The task will be comprised of three primary s'btasks:

S() Design and develop a useable NTC data base,

(2) Implement the data base, including retrofitting NTCCRS

data and incorporating data from additional sources, and

L,=.

Of 30
(3) Develop a researcher's guide to document procedures and

methods to use the data base.

11.6.3.1 Design and deve useab le NTC data base

The first subtask concerns the redesign and development of .

the NTC data base so that it properly supports the NTC researc,

objectives of ARI-POM. This is a long term process which will

consist of the following steps:

(1) Requirements Analysis. This step specifies the data

base. in terms of which data are required, including explicit

definition, accuracy, units. and derivation, if necessary. The

requirements analysis phase is driven by the research objectives,

so a major portion of the activity here is in proposing the

research course that will be supported by the data base, and in

defining the data elements required.

As part of the requirements analysis, a review of all

existing NTC data sources will be conducted and documented. This - .

will include both digital and non-digital sources as well as data

currently received at ARI/POM and data generated at NTC but not

currently received at ARI/POM. Thus, the essential data set

defined as a result of this analysis will include oaper data such

as the Take Home Packages, OPORDs, Scenarios. etc.. as well as

digitally captured data. Because of the diversity of the physical

characteristics of the data sources and data elements, it is

likely that the NTC data base will be a combination of computer

and non-computer storage. At present, it is anticipated that the ...

non-computer storage will be in the form of a rotation "log book"".

The log book would contain all paper data elements associated with

a rotation. Further, the log book would be supplemented by copies

of any analyses performed on the rotation. The result would be a p

continual expansion of the data associated with a rotation and an

assurance that no data or analysis information is unavailable for

future examinations.

(2) Design. This phase allocates the requirements into data

base terms, including choosing the database structure down to the

table structure. tables, and keys. As indicated above. the

database probably will have both a computer and ncn-computer

component. Also included in the design effort are complete

specifications of all data sources. down to explicit format

information, and design of the processes which wi l be necessary

to incorporate the data into the new data base structures.

(3) Development. During this phase, the processes necessary

to build the new data base are developed. The processes might

include computer programs. command files, manual entry, data

verification, derivation of some data elemnts, etc.

(4) Implementation Plan. This phase provides the transition

into a production status. and ties all of the seoarate entities

developed in the orevious step into a working system. In the

31 '

'k. !ftA -• 
ARI-POM environment, the implementation also includes the retrofit

of data presently in the NTCORS format into the new data base

format.

11.6.3.2 Implement the redesigned data base.

This subtask implements the redesigned data base by following

the implementation plan documented in the previous subtask. There

are two basic functions that must be completed during this

subtask:

(1) Convert data that was previously entered via the NTCDRS.

This function consists of using a conversion utility to reprocess

the NTCDRS data and produce a data base in the upgraded format.

It should be recognized that the translation cannot be totally

faithful, due to the lack of supplementary data that must be

factored into the new data base format, which is no longer

available for older rotations. For example, if the new data base

includes data from the ROMS log, these data are not available for

any rotations prior to 85-10, and for some rotations since then.

Some supplementary data will be available, such as THPs, and will

be incorporated as appropriate.

(2) Begin production on the most recent rotations. The

assumption is that all necessary data will be available for

rotations that occur after the data base design is finalized and

additional data collection requirements are coordinated with the

NTC. In this case, generation of the databases for current

rotations should be fairly straightforward. However, it is

recognized that actual production of the database will likely be

interactive in nature, particularly as results of new analyses are

added to the existing structure.

11.6.3.3 Develop Researcher's Guide to NTC Data.

Subtask 3 is concerned with documenting the contents of the

NTC data base and in providing explicit guidance in accessing the

data, manipulating the data, and generating reports from the data.

This subtask will be based on a compilation of outputs from the

data base design and implementation processes described under

Subtasks I and 2. It will also include the development of

training materials to guide researchers in all facets of NTC data

base manipulation.

11.6.4 Anticipated Reoorts or Products

Activities performed in support of Task Six activities are

list and described briefly below:

(1) An NTC Data Handbook, describing explicitly every facet

of the NTC data base, including each data element. table

structures and contents. The handbook will also contain the

results of the requirements analysis.

32

...-.-
(2) An NTC Data Manipulation Handbook, which describes how

*to access and manipulate data contained in the NTC data base.

This document will also describe a number of "canned" procedures

that will be made available to researchers to produce a number of

standard reports from the data base.

(3) A short, "hands-on" course to introduce the research

staff to the NTC data base. This course will be 4-8 hours, and

will be required prior to allowing access to the data base.

Subjects will include an introduction to the relational database

system. an overview of the NTC data base and its structure, an

introduction to the standards which must be followed in using the

data base, including file, table, and procedure naming

conventions.

(4) A periodic update of the data available to the research

staff. This will be in the form of a computer listing which lists

the segments available, some basic information about each one, and

where the data base is (disk or tape).

.-.

!j

_]I

.;.;'33
Milestones for Task 6 ~1986 :987

MAR APR IMAY IJUN JUL AUG SEP OC T NOV iDEC JAN FEB

Subtask 0 raft 

6. 1------------- ---o 

Requirements -'

Analysis I-_ .-

Drafti Final

6,2 .o--------- o

Design II

6. 3 - - -o o-- 

Develop- II

ment I"

(UPDATES)

6 .4 I I - 0 - - Implemen- I. 'tationI I I I

0i

-I.,

~..,

al

.. -3

* 34!
11.7 Task 7: Develop products to support training and feedback 

at the NTC. 

I I . 7. 1 

Increasing e•phasis is being placed on more effective 

integration of NTC and home station training and the NTC's 

potential for addressing questions concerning training techniques. 

equipment. organizations. and doctrine . The purpose of Task 7 is 

to develop a systematic approach to the preparation of NTC After 

Action Reviews (~ARs) and Take Home Packages (THPs) that takes 

full advantage the NTC Operations Group tactical experience. 

exploits advanced automation facilities and capabilities. and 

benefits fro• ongoing research in the area of Aray training . 

I I . 7. 2 

The Operations Group developed the current format for the AAR 

and THP in an uncertain environment that required both computersupported and manual materials to be assembled for each AAR and 

THP. Trade-off decisions and compromise were necessary due to the 

immediate concerns of shaping an effective process that presented 

units with material to appraise the tactical performance of their 

task force . The resulting format was the product of both an 

evolutionary process and a common sen~e solution to the tasks at 

hand. The capability to aggregate and incorporate diagnostic 

performance information into the feedback packages (i.e .. AARs and 

THPs) can be improved. In order to expand the current data 

capturing and aggregation capabilities with the result of improved 

NTC feedback processes. Task 7 must address two separate aspects 

of feedback development : 1) the format and content of the After 

Action Review ; and 2) the format and content of the Take Home 

Package . 

The AAR process is the most important training diagnostic 

function performed at NTC . As such, it is critical that it employ 

the full range of capability offered by the NTC instrumentation 

system as well as the observations of the OCs . Further , the 

content and structure of the AAR must be such that it contain the 

essential elements of performance which reflect the reasons and 

causes for mission outcomes. To accomplish this latter function. 

the AAR needs to incorporate the research results from a number of 

separate activities directed at leader performance , mission 

effectiveness. and critical task ~eformance . 

Leader performance has been an area of considerable interest 

and activity at the NTC. In conjunction with CAL. ARI has been 

involved in a study of leadership at NTC and has developed plans 

for enhancing the effectiveness of NTC OCs in reporting on leader 

performance. It is anticipated that part of the benefit to be 

accrued from this enhancement would be improved AARs in the area 

of leader performance . One of the focuses for Task 7 will be the 

investigation of how OCs can better observe leadership tasks for 

inclusion in a prototype AAR . This effort will capitalize on the 

35 
work performed both by the ARI Leadership team as well as the

contracted work performed for that team.

In addition to the leadership portion of the AAR, we will also

include the work from Task 2A concerning mission effectiveness and

critical task performance into the prototype AAR. By including the

results from this effort, it will provide the AAR with a three

dimensional structure of: leader behavior, mission outcome, and

task force performance. We believe that NTC performance and the

training implications from that performance can be fully described

by these dimensions. Thus, the prototype AAR will be approached

from this multidimensional perspective.

The second purpose of Task 7 is to develop concepts and

guidelines for potential incorporation into the NTC Take Home

Packages based on training and measurement concepts,

recommendations of the NTC and unit personnel, home station

training conditions and constraints, and results of performance

analyses.

The THP currently is a trend-related document which

summarizes improved and unimproved training and operational areas

experienced by the unit during NTC training. Recommendations for

further home station training emphasis are included for each

deficient operating system. The THP document is supported by

selected audio, visual, and computer-generated tapes.

Although the purpose of the THP is to change and/or reinforce

- home station training to meet levels of readiness proficiency

required by the wartime mission, THPs currently are not designed

as training schemes to be implemented upon redeployment from the

NTC. The purpose of Subtask 7.2 in Year Two is to develop concepts

for the THP based on training and measurement concepts developed

in Task 2A, a model of home station training developed in Task 8,

recommendations of NTC and unit personnel, and the AAR

guidelines developed in this task so that it will support home

station training by clearly documenting the performance diagnoses

and proposed solutions to deficiencies identified in the AAR

process.

11.7.3 Methodology for Task Accompl ishment:

Task 7 will require the performance of two separate but

related subtasks. Subtask 7.1 will result in the preparation of

guidelines for the AARs. Subtask 7.2 will produce concepts and

guidelines for Take Home Packages.

Four separate appraisals must be completed to address the

requirements of subtasks 7.1 and 7.2:

1) The current state of AAR and THP methodology must be

established. This should include a review of the organization and

functional responsibilities of field and computer staff officers.

V4 36

.N..

.,,. ..- .... .. . .. ... . , ,,...-
,-a

2) Guidelines and methodology for proposed AAR and THP

content must be established and approved.

3) The results of the previous year's research and other

training technology advances must be reviewed and utilized as

appropriate.

4) The capabilities of the NTC computer system must be

appraised at the time the research is started. This should

include down range data transmission and displays, data entry and

recapture at the TAF, and the range of data being captured,

stored, and recovered by the NTC system.

The two subtasks require the production of two documents:

Guidelines for After Action Reviews and a report containing F*o

concepts and guidelines for an improved Take Home Package. Having

direct input into these subtasks will be the measurement concepts

and methods for measurement of unit combat effectiveness developed

in Task 2A, the leader effectiveness measures developed under this

task, "lessons learned" documents developed in the first contract

year, and various home station factors addressed in Tasks 4 and 8.

11.7.3.1 Oevelop Guidelines for After Action Reviews:

The steps to be followed in accomplishing this subtask are

listed below. It should be noted that the activities directed at

developing the concepts for the Leadership portion of the AAR will

be performed separately though not independently of the remainder

of this subtask. This organization will facilitate the efficient : "*1

performance of this subtask in light of the varying interests of

the proponent agencies.

Leadership AAR Tasks: I.

1. Review existing literature and prior ARI research

concerning Leadership and Leadership Measurement.

2. Review, critique, and modify as appropriate framework

developed for leadership measurement at the NTC.

3. Establish appropriate leadership performance dimensions._

4. Develop strawman observation guides.

5. Interview OCs for feasibility of framework at the NTC.

*.-6. Revise and reproduce leadership guides for use in NTC NCO

focus rotation.

7. Produce report on development and use of Leadership

Observation Guides.

The tasks listed above will be accomplished within the

timeframe of the second year of the contract. The tasks will be

37 4

.,,.-. ., ,, ,. u .,,. .-..,, .......r..,_..s.. --,,. .,-.?_ ,, _,, .:.J, _ _. '_._. '_,.,'.,L,. ." "'-..A, :'- ..a E..,J. .'J'-- ' .L" , ' .. *• 1
performed as part of a larger ARI effort in support of an NCO

focused rotation at the NTC. The use of this vehicle. i.e. the

focused rotation, provides several advantages to the proposed

effort. Specifically. it will define and narrow the focus of this

initial work, it will result in the production of actual data

collection devices, and finally, it will allow for field-testing

of the devices. It is anticipated that the results of this first

year effort will allow for direct expansion to other levels in

the third year of the contract. input into the concepts and

guidelines for the AAR, and application to Task 2A.

Mission Effectiveness and Critical Task Performance for AARs:

1. Review existing AARs and reference material to

determine content coverage and appropriateness.

2. Interview NTC and Home station personnel concerning AAR

presentation and effectivenees.

3. Using training and measurement concepts developed as part

of Task 2A and Task 8, develop guidelines, concepts and

protoype AAR content and structure.

4. Present to ARI for their review and their solicitation of

CATA involvement.

Subtask 7.1 will reflect measurement concepts being developed

in the areas of unit and leadership performance and will also be

based upon inputs which can validate the training effectiveness of

the AAR process. Accomplishment of this subtask makes will make a

major contribution to the NTC and the Army as a whole.

I1.7.3.2 Develop Concepts and Guidelines for Take Home Packages:

In this subtask, concepts and guidelines will be developed

for potential use in the NTC Take Home Packages. As part of the

effort, we will investigate the military and authorized scientific

community to determine the true needs of those that use the THP.

To the extent possible, the actual data collection for this

investigation will be done in conjuction with the Task 8 and Task

9 efforts. In addition to the results of this investigation, the

results of Task 2A and Task 8 will be drawn upon to identify

concepts and guidelines for inclusion in the THP. It is

anticipated that these will parallel the information in the AARs.

From this output, concepts and guidelines for suggested

improvements in the THP will be revised. Steps to be taken in

developing the concepts and guidelines for the THP are:

1) Determine user needs

2) Survey sample users as to the strength and weakness of

the existing THP

3) Review input and design requirements

4) Propose THP guidelines and concepts

5) Submit THP concepts and guidelines for ARI review

38

.B iX .•I - oo . A 4 ".,° .Io* L% 2 . '% ' " "
Examples of the data to be gathered for incorporation in the

subtask are:

- who uses THPs?

- how and why are THPs used? - what are the characteristics of the users? - what is the environment the users operate in? - what has been the impact on training and analysis

from previous published THPs? - what has been the relationship between successful

NTC training and home station use of the THP prior

to deployment to the NTC?

- what has been the impact on home station training

methodology and BTMS as a result of THP use?

- have levels of performance improved as a result of

THP use? - what changes would unit personnel and users like to

see in the THP? - what Operations Group resources are available to

produce required THPs?

Expected data sources include CATA, TRADOC, FORSCOM, NTC,

FORSCOM units. Service Schools, and authorized analysis agencies.

The development of guidelines and concepts for the THPs is

essential to maximizing the diagnostic training benefit of the NTC

experience. Comprehensive and useful THPs will be primary

contributions to the Army feedback system and Army-wide

utilization of NTC data.

11.7.4 Anticipated Reports and Products:

S.' At the completion of this task, BDM will produce working

documents that address:

1) Format and content of the AAR based upon a diagnostic

multi-dimensional model of unit performance

2) Develop and test feasibility of enhanced observation

protocols for leadership tasks with OCs

• 3) An expanded concept for AAR presentations to fully use

all the technology available at the NTC

4) Format and content of the THP based upon a diagnostic

model of unit performance

4 39

'VV
Milestones for Task 7

1986 1987

MAR JAPR IMAY IJUN IJUL IAUG SEP IOCT INOV DEC JAN IFEB I p .:? 7.1 AAR I I I I II

-: Guidelines I I I I [I

I I Tech I

7.1.1 --------------- oRep I

Review current Submit

_ AAR Method- Propose I for

o'logy etc. Guidelines Review &

I I II' I Ito ARI I Approval .'.

e. 7.1.2 1 I ---------------o--------------0o

Develop and I - -

submit guide- I IF

lines and I 1Define II IV method for AAR ILeader I Produce II II

7Dimensions IGuides I I I I I

7.1.3 -- ---- o 0---------0 ------------------------ 0

Review & I IPrepare I I I I I I I I

Integrate I I Observation I I I I I

other research I JGuides I I I I I I

I I Tech

./7. 1.4-------------------------oRep I I I I I I

Appraise II

Computer/ I

Instru. Sys. IIFeedback ".I

Capabilities .I-

' 7.21 THP "'I- •. Guidelines I I I I I

7 2. 1 -------- -0

-.. Review Current I- . THP Methodology Propose Submit

Sample User Guidelinesl for

Needs IConcepts Review &

_II Ito ARI Approval

. 7 2 2 2 I-------------------o--------------o o 

., Develop and I I -

submit guide- II.I"I

lines and I

Smethod for THP I I

I(Data collection effort)I I I I

: 7.2.3 o-----------------------------

Review & To determine user needs

> Integrate I in conjunction w/ Task 8

-' Other Research II II I

I Tech J

2.4 oRep CI

Appraise I I II

Computer,' I ]

instru . Sys I I II I

i" Feedback I I

_ Capabilities I l II

40
- - .- -

11.8 Task 8: Develop Products to Support Home Station Training

and Training Assessment

11.8.1 Purgose of the Task

In order for the Army as a whole to derive maximum benefit

from the NTC, it is necessary that the training experience

afforded at the NTC be institutionalized by the units at Home

Station. Thus, the NTC experiences must be effectively integrated

into a unit's active training program subsequent to its NTC

rotation.

The purpose of this task is to develop guidelines for unit

training at home station prior to and following an NTC rotation.

The guidelines will include strategies for the effective ii

integration of NTC and home station training.

11.8.2 Understanding of the Task

The NTC provides units a training opportunity that is

unmatched in its challenge, scope, realism, and feedback. The

high fidelity combat simulation allows units to fight as combined

arms teams on an instrumented battlefield against an experienced

opposing force that employs Soviet doctrine and tactics.

Observer-controllers supported by the automated data base provide

immediate feedback to the units in the form of Mission After

Action Reviews. The After Action Reviews enable units to process

performance data, adjust procedures, and operate repeatedly to

correct deficiencies and improve performance while still at the

NTC. Given the fidelity of the NTC as a combat simulation and the

richness of the NTC training experience as an assessment and

action planning tool, and recognizing that this opportunity is available to a battalion just once in an eighteen month period, units should be at a level of readiness that will enable them to

derive maximum benefit from the NTC training. Their pre-NTC

training posture should be such that they can cope successfully

with the demanding scenarios and absorb and utilize the

performance data. Following the NTC experience, units should be

able to adjust training plans so that subsequent home station

training activities correct deficiencies and build on strengths

that were identified at the NTC.

Thus the effective integration of NTC and home station

training is reflected in an annual training plan and program that

enables units to prepare fully for the NTC. perform optimally at

the NTC and move to higher levels of readiness at home station

following the NTC experience.

Currently, there are no models or universal guidelines to

assist units in effectively integrating NTC and home station

training. As a consequence, training at home station and

performance at the NTC vary widely throughout the Army. A primary

objective of Task 8, therefore, is the development of products

which will support an integrated unit training program.

41

WD

, :j ,i.Z " -,, -. : Z4, .ZmyZ . Z " ,-'. ,'.' ./." ' .. , •. .- .. L I'- . "'_"
11.8.3 Methodology for Task Accomplishment

Investigation of effective NTC-home station training will be

accomplished over a two year period by following the subtask

sequence shown below. The Year Two subtasks will be accomplished

as contract Task 8 while the Year Three subtasks will be performed

as contract Task 13. Each subtask is described below so that a

comprehensive understanding of the investigation is afforded.

Year 2: 8.1 Develop Pre- and Post-NTC training constructs

8.2 Develop data collection methodology and plan

8.3 Conduct pilot data collection effort

Year 3 13.1 Collect pre-NTC, NTC, and Post-NTC data

13.2 Analyze data and develop report

13.3 Develop guidelines for unit training at home station

11.8.3.1 Develop Pre- arid Post-NTC Training Constructs

Using information generated by work on Task 6 and Task 2A,

* BDM will develop an array of NTC performance factors and clusters

associated with good performance at the NTC. This array will then

be used to develop constructs which identify and describe the Preand Post-NTC training and support mechanisms which contribute to

good performance at the NTC and subsequent efforts to correct

deficiencies and improve combat readiness. Then, by analyzing

these Pre- and Post-NTC training constructs and reviewing FORSCOM,

TRADOC, and division training publications we will identify and

target for data collection those relevant sources that will

provide maximum information for conducting correlational analysis

of home station factors and NTC performance factors. The

preliminary major areas of focus for collecting data are seen as: I

S1. Training

- Plans, schedules, and activities - Team/crew, squad, platoon, company, and combined arms

proficiency

- Facilities and support items

2. Personnel

- Fill

- Grades authorized vs. grades assigned

- Individual qualifications

- Time-in-position

- Turbulence

__3. Organizational Policy and Support

- Training Philosophy

- Command Climate

Concurrent with the work effort described above, BDM, in

conjunction with ARI/POM will identify the six battalions (in

three brigade rotations at the NTC) needed for the full data

collection effort. Preliminary coordination with FORSCOM and

TRAOOC will begin to assure that the battalions are available and

accessable during the study period.

i 42 • ,-

' ,.,',,. .. -.-. -, - ,-.'. . . .. -.... ... -. - .: . . ... ..- .-. ,.., .. .,. .- * .,, -. , , -. . ...- , .,,,- . ., ... -. ,.,- -:, ,." .-.
II.8:.3.2 Develo Data Collection Methodoloq and Plan

To attempt to establish linkages between the NTC training

experience and home station preparation and follow on training

activities, data will be collected on six battalions during two

time periods: 1) one to two months prior to the units' move to the

NTC. and 2) four to six months following the completion of the NTC

rotations. A third, and much smaller, collection effort will be

made on the last day of the NTC rotations to get commanders' and

leaders' immediate reactions to the NTC training, their

preparedness for it and their thoughts concerning follow-on

activities at home station.

The methodology for collecting these data, the collection

instruments, interview protocols, and a time-phased collection

plan will be developed in this subtask. All such items will be

prepared in draft form and submitted to COTR for review. In

addition, BDM and ARI/POM will obtain FORSCOM/TRADOC approval and

clearances for the data collection team's home station visits.

11.8.3.3 Conduct Pilot Data Collection Effort

This subtask involves the testing of data collection

methodology, instruments, protocols, and procedures. The BDM data

collection team will conduct one Pre-NTC home station visit, one

end-of-NTC rotation visit at the NTC and one Post-NTC home station

visit. The units visited during this pilot will not be among the

six selected for the full data collection study, and the three

pilot visits may be to three different units.

Based on results of the pilot, all components of this subtask

will be reviewed and modified as appropriate.

Following the pilot collection effort, BDM will prepare a comprehensive report summarizing the task efforts through that point.

11.8.3.4 Collect Pre-NTC_ NTC. and Post-NTC Data

Data for the six battalions in the study will be collected in

three phases.

The Pre-NTC Phase. The BDM data collection team will visit

the unit at home station and collect those data associated with

training and support mechanisms previously identified as

supportive of optimal performance at the NTC. This visit should

be timed to enable the team to collect data which accurately and

completely reflect the unit's state of preparedness just prior to

the NTC training. Thus, it would be desirable to plan its visit

no sooner than thirty days prior to NTC. Data will be taken from

existing records and reports, questionaires completed by most

leaders in the battalion and structured interviews of key leaders

- and commanders at company, battalion, brigade, and division

levels. Approximately five to seven days of on-site work will be

114 43
required.

The NTC Phase. Members of the data collection team will

interview the battalion commander, his principal staff officers

and each company commander during one of the last days of the NTC

rotation. The effort will be comparatively brief and will focus

on participants' reactions to NTC training experiences, their

views about how they could have prepared better and their thoughts

on what should be done as follow-on training at home station. The

collection team members will also coordinate directly with

appropriate staff of the NTC to assure that all relevant NTC

performance data are collected, packaged and promptly forwarded to

the ARI field unit in Monterey.

The Post-NT C phase. This phase involves a second visit to the

unit's home station. The focus of this visit will be to collect

data which reflect the extent to which the NTC training experience

was used:

a. To focus follow-on training plans and activities;

b. To correct deficiencies identified during the NTC

experience;

c. To sustain and improve on the level of preparedness as

reflected in the NTC performance.

Additionally, the team will collect information that will

assist in identifying those home station circumstances and

influences which help and/or hinder productive follow-on training

at home station.

This visit should occur four to six months after the unit's

return from NTC. By this time, the unit is likely to have:

returned to normal routine; had time to reflect on the NTC

experience; and made appropriate modifications to training plans

and schedules.

Approximately five to seven days of on-site work will be

required for this subtask.

11.8.3.5 Analyze Data and Produce Report

Data and information will be analyzed to reflect the profile

of a single unit and also a profile accross units. Analyses will

address the basic issues of training quality within the unit,

training quality at NTC, the training conditions effecting both

unit and NTC training, and the interrelationships existing between

NTC and unit training performance. Comparative analysis will

produce profiles of a unit's training condition indicating the

forces which established the condition. Profiles will be compared

among units to establish a common relationship of the impact that

resource managers have on trainers. Quality indicators of the NTC

training environment will be compared to the training profiles of

the individual units. Analysis will surface requirements which

show a need for greater NTC integration into unit training as well

44

• - "Zr
as demonstrating the benefits of NTC to unit training activities.

BDM will prepare a draft research report on the integration

of NTC and home station training. It is anticipated that this

research report will be one volume or section of a larger final

report which describes the entire research effort and the

interrelationships among the project areas. The report will

summarize the related literature, detail the research design and

procedure, present the results of the research, and discuss7

findings and conclusions regarding requirements for more effective

,' integration of NTC and home station training.

11 .8.3.6 Develop Guidelines for Unit Training at Ho me Station

Based upon the results from the analyses and recommendations

of the unit, NTC, and other appropriate Army personnel, guidelines T'.

will be developed which include recommendations for training type.

echelon, frequency, and sequencing to best enable a unit to profit

from advanced unit training at the NTC. Methods will be included

for prioritizing and adjusting the training to home station

constraints and conditions. These methods may include

descriptions of training options and their relative benefits and

costs (e.g., simulators and reduced caliber versus live fire,

large unit FTX versus CPX, battle simulations, fire control

exercises, small unit FTX versus drills, leader training, etc.).

This subtask will be a considerable effort. A single

guideline cannot serve as a panacea for all units that train at

the NTC. Every unit is different and every division has a

different set of training resources, constraints, missions,

leadership, and training environment. Every unit that trains at

the NTC has its own particular set of training objectives. The

scope of this effort increases with the introduction of air

cavalry, air mobile, and light infantry into the NTC training

arena. Therefore training guidelines must be tailored to meet the

needs of each battalion task force that is scheduled to train at

the NTC. It must be recognized that as senior commanders come and

go, so training philosophies and associated resources and

priorities change. While a training guideline may meet a need for

a short period of time, the specifics within the guideline will be

overcome by events. It is anticipated that guidelines will have

to be continually updated due to the dynamic nature of command,

training, philosophy, and resourcing.

The correlational analysis of home station factors and NTC

performance serves as the foundation for the development of

guidelines. Published Army training methods will Drovide the

structure for the guidelines. This is essential for

standardization purposes. Training needs will be balanced against

training resources and time available. Recognizing that there is

never enough training time available to bring units to level one

proficiency, integrated and concurrent multi-echelon training will

be inherent within the training cycle. The guidelines will

reflect the NTC training performance assessment methodology.

45

INA
Presently, assessments of unit and section performance are

performed at platoon, company, and task force levels. Platoon and

company assessments are based upon achievement of ARTEP

mission/task standards and prescribed doctrine and tactics within

the seven operating systems. These systems are command and

control, maneuver, fire support, intelligence, air defense,

mobility-countermobility, and combat service support. The

guidelines must relate directly to the NTC assessment system to

ensure compatible understanding and development of correct

training objectives by the unit training managers. Guidelines

will be highly descriptive and procedural. Types of training,

applicable echelon of training, frequency, and sequence of

- training will be included within the guideline. Methods for

prioritization and adjustments to training as caused by home

station constraints and conditions will be designed. These

methods will include descriptions of training options with

relative benefits and costs. The steps to be followed in

developing the guidelines are as follows:

(1) Conduct the correlational analyses of home station

factors and NTC performance;

(2) Assemble and research any literature to determine the

acceptable method(s) of training management;

%. (3) Determine training objectives;

(4) Determine training resources available at home station;

(5) Determine training constraints (e.g., money, time,

personnel, terrain, ammunition, level of proficiency

within echelons);

(6) Develop cost effective training options to meet training

objectives;

- (7) Develop and staff unit-particular training guidelines;

(8) Produce guidelines.

The data to be gathered will consist of the following:

(1) Historical NTC performance of the unit,

(2) ARTEP results,

(3) Army training methods,

(4) Training strategies,

(5) Training objectives,

(6) Constraints,

(7) Resources available, and

(8) Unit readiness profile.

Sources will include the correlational analysis. Army

literature, unit and NTC personnel, and unit training records and

reports.

The accomplishment of this subtask provides the training plan

by which training managers will be able to develop a workable and

viable training plan to improve combat readiness.

1S 46

. . . . . . . .. . .. . . . . .-S. . . *.-, . ... _ _ _ _
11.8.4 Anticipated Reports or Products

When the task is completed, BOM will have produced the

products and reports listed below:

Year 2: 1. An Applied Model for Evaluating Home Station Training

2. A Methodology for Evaluating Home Station Training in

the Context of NTC Preparatory and Follow-on Training

3. An Interim Project Report Describing Work to Date and

Pilot Test Results

Year 3: 4. A complete report of the comprehensive data collection

effort with correlational analysis of home station

factors and NTC performance on eight to twelve units

that trained at the NTC. i,.

5. A set of guidelines for training type. echelon,

frequency, and sequencing to enable a unit to profit

from the NTC experience.

47

* 47 ;,

-° . * " ." *. ' " V V. ",V ',', ° .-'." - .. .< . .. ,.'-. . .. :' - - . .. ..'','.-..-',";';', *. ..- . , ,. ' .-' -.-..*V ',- C ,-.--.,,--', . . ..- , ., ,". * .. ,. - .. ,
Milestones for Tasks 8 and 13

MAR IAPR IMAY IJUN IJUL IAUG SEP OCT NOV DEC IJAN FEB I

YEAR 2 (86-87) I I I

8.1 o--------------0 -Training l

Constructs i .. . I" I I II-II"II II

.8. 2 -- 0"oS Data I .'-

Col lection .,..

Methodology I I

& Plan II "

8.3 o--------------------------

CPlot I Collection I I I I I I I I I

8.4 ------------ 0

Report I I-- II -

13.1 1 . -

Data Col lection I

YEAR 3 (87-88) .,"I .

3 I I Io I .-.r

•13.--------------------------------- 1 1 1 1 -

Da ta I ,II-

Col lection I

13.2 I------------------------- - ---------- 01

Analysis/ I II

Report "'II

:3.3

Gu i de 1 i nes "I=

o Products/Deilverables

4..

j * 48~.*. . .. . . . ."... 
~.-.

II.9 Task 9: Develop Products to Support Feedback to Army Users

and Army Wide Utilization of the Products of the Research

II 9 1 Purpose of the Task

The purpose of this task is to perform a front end analysis .,

for a potential Lessons Learned information feedback system for

the NTC. Its focus, then, will be on the creation of a

methodology which might be employed in the creation and

dissemination of Lessons from the NTC to other Army proponents.

To accomplish this objective, the initial step is one of user " e. identification and definition of informational needs. Task 9 has

been designed to provide a comprehenive response to this objective

and will involve the creation and conduct of a user survey to

obtain the requisite information. It is anticipated that this

information will include a set of high priority issues as well as *

identification of alternative ways in which NTC data might be used '..'

to address these issues.

I I -9.-2 Understanding of te Task

This task fully supports the charge by the Army Chief of

Staff to exploit the NTC's full potential and spread the NTC

experience throughout the Army. After five years of intense

training experience, the production of feedback to proponent

agencies is just now beginning. Feedback to the units involved in

training has been immediate and comprehensive. Units receive

daily feedback as to their tactical proficiency as a result of

battle outcomes against the Warsaw Pact style OPFOR. After each

battle, the unit receives a detailed review of its strengths and

weaknesses as applied against prescribed tasks and standards

demanded by the type mission. The rotational units have been the

principal benefactors of training feedback. The brigade and

division trainers gain feedback as to the performance of the

battalions, companies, and platoons. and the respective leadership -.

of those elements through personal observation and attendance at

the After Action Reviews. Notwithstanding the fact that there is

* a requirement for improvements in the assessment and feedback

system, the units do receive subjective and minor objective

feedback of greater depth and significance than ever received in

the history of warfare training. While real-time feedback

requires vast improvement, the development of a feedback system

and products to Army users worldwide is just now beginning The

Army's Training and Doctrine Command has established an NTC

Lessons Learned division to capitalize on lessons from the NTC

However. the training development community will require

additional support from NTC to support quantum improvements in .. t r a i n i n g

The impact on the readiness posture of the Army and the

international perception of other powers as to the ability of the

Army to fight and win on any battlefield is directly influenced by

NTC feedback. The early exploitation of NTC information is of such

significance that feedback products must fully meet user needs

49 4.

°. --
The front end analysis represented in this task is designed to

assist in the identification and deliniation of these needs.

The year one efforts were critical in surfacing initial

military significant research issues. However, it is critical to

- continue to surface additional high priority issues. particularly

• -for users in the Army at large. To facilitate this end, the

" survey to be developed for use with respondents from relevant

proponents will solicit issues as well as their priorities and

potential ways in which they might be addressed by NTC data. The

NTC instrumentation system must be hardened and brought to its

full designed operational capabilities to support this task.

Refinements in data cleaning, analyses, and interpretation of data

must be completed to ensure the availability and uilitzation of

empirical data.

11.9.3 MethodologQ for Task Accomplishment

A basic methodology for accomplishing this user needs survey

and for the future development of feedback to the user is shown in

Figure 11-2. Although these two tasks are to be conducted during

separate contract years, their unitary purpose warrants an

identical methodology. The "Analyze" and "Design" requirements of

the methodology apply to this specific task. The "Develop" and

"Test/Evaluate" requirements will be addressed in Year Three.

11.9.3.1 Subtask 9.1 -- Conduct Detailed Survey of User Needs in

Training and Doctrine Development.

The initial step in this subtask will consist of a search for

potential Army users of NTC data and information. The search will

not be confined solely to the training and doctrine development

community, however. Users within the combat development community

who are responsible for requirements concerning weapons,

organizations. and equipment should also be identified. Our

current understanding of the material development cycle and the

current TRADOC-directed training development system/model will

produce the primary relevant users; the military-experienced

users responsible for developing weapons, equipment, and

organizations. Surveys and personal follow-up contacts will

reveal the exact format, detail. and media of NTC data required by

the user Beyond the NTC database, the research team will exploit

other NTC sources to include non-instrumentation sources such as

Port Irwin (NTC) base operations, historical data and home

staton training information. At the completion of this task,

tne research team will formulate final recommendations for future

changes in current procedures. where applicable.

The anticipated sequence of activities for the accomplishment

of this front-end analysis is:

0 Establish the appropriate target audiences for the

dissemination of NTC information in coordination with

50

'p..
I.- %1 3 . --' 3

(Aw

Ulf- 0 w0i

J9 4c

Iz I

MIM

IL U. - 2: c. 30.

- 0- : =e 0 

_. 0 + , +- a.o+_ .

-'-

ma Ob1 0

e 0

Sc."

"" - OL- Z.

./.. •._e I --

=1. Os -: *b_e 0 _ •m

,0 0

00

I- ca 3 b• 0- 

.* 0o 0

z S - I 1 - -

UJ I

.... w***::- .. : ,.

,c, 0 0_ cc e". X::

",,r"~~. '_ b - a ll'- 

6w• c

c cIS,,

e' ',,'¢ " " ". " "'., "'".". ',' '. -+. .. .".. .. --,,.- C 2 & 2 a e.. . . . .. . .. :,,, .,. .,, .,,.,..., ., 'E ,.:., . .... ,. 3 .... . .. ., .. .-M' .... ... . ... ... .,.. ... ..
and approved by ARI-POM. Other agencies will be

included as appropriate.

o Determine specific user, or potential user.

organizations of NTC and home station training

information through a top-down approach beginning with

the Army proponent for NTC matter, OCSOPS, then to HQ

FORSCOM and HQ TRADOC. Surveys may also be taken.

Information and products of Task 3 will be highly

useful in this endeavor.

o Validate NTC and home station training informational

access authority of these organizations through

OCSOPS.

o Determine current and future user informational

requirements. This may include type information.

purpose of informat'on, source, media. format.

procedures, and time schedules.

0 Ascertain current and future requirement

shortfalls. These become user needs.

0 Review NTC and home station training

information for compatibility with user needs. Where

data are not compatible, it will be necessary to design

a data collection, transformation, and analysis plan for

transforming the data into compatible format.

This will complete Subtask 9.1 and be followed by the

"Develop" and "Test/Evaluate" steps to be performed Year Three.

This task accomplishment will allow the Army's full

exploitation of high technology training in the areas of training

development, doctrine development, and readiness assessment

Included is the exploitation of information and data for weapons.

organizations. and equipment development.

11 9.4 Description of Anticipa ted Results or Products

A final report wil' be submitted which will describe the

methodology. findings. and recommendations of the activities

discussed above

'p.

P%

-,A2,
Milestones for Task 9 1985 1987

MAR IAPR MAY JUN IJUL IAUG ISEP 1OCT NOV DEC JAN IFEB 

9. 1

Conduct

Survey of

,User Needs

9 . 1 1 ------ o0

Establish

Appropriate

Target

- Audience

9 I 2 -----

Determine

User Organ- izations I I II I I

9.1.3 1---------------------------0

Determine

Current

Information

Requirements

9.,4 o-------------- o

Determine

Informational

Shortfalls

Draft Final

9 1 5 - -o

Review NTC I

& Home Station I

Tra n i ng 

Information .

for Compat- I

ability w'th I

User Needs I
II.10 Task 10 -- Prepare Proposed Third Year Research Plan

11.10 1 Purpose of the Task

The primary purpose of this task is to prepare a proposed

research plan for the third year.

11.10.2 Understanding of the Task

BOM is to prepare a proposed research plan for the third year

based on the experience gained in the first two contract years, an

understanding of the military and scientific areas of interst, and - the overall research technical objectives. Any proposed tasks

resulting from this effort must reflect its rationale and priority

of effort.

111.0.3 Methodology for Task Accomplishment

The methodology used for accomplishing this task will

parallel that used for Task 5. In developing a research plan for

the third year, BDM will first consider the accomplishment and

shortfalls of the second year's effort for each task. Then, it

will consider the impact that these findings will have on tasks 11

through 15 scheduled for the third year. Any additional tasks

will also be considered (BDM assumes the purpose and technical

objectives for the third year will remain unchanged. Therefore,

any required adjustments will be made with this in mind.)

11.10.4 0escription of Anticipated Results or Products

The research plan to be prepared for Year Three will be

structured in the same manner as the current plan.

At the time this document is prepared, the COR will be

presented a final draft and provided an opportunity to comment.

The third year research plan will then be finalized, and "

delivered

Mi lestones 'or -ask IC

4986 1987

'Ar APR :AY 2UN JUL AJG SEP OCT NOV DEC JAN IFEB I

I I I III .

': eDare Draft Final .. Yea 3 - --------- 0

Research I I i

P Plan II I

*I 54

*.... ~ '* **' 4 -
------------- p1 I.2A Task 2A: Develop Concepts and Methods for the Measurement

and Interpretation of Unit Performance

This task represents an augmentation and expansion to the

original contract. The work to be performed here is a

continuation of the effort started in Task 2. year one. The

present task has two distinct but related subtasks. The year two

work to be performed on each is presented separately below.

II.2A.1 Task 1: Development of Unit Combat Effectiveness

Criterion Variables.

II.2A.1.1 Purpose of the Task

The purpose of this task is to develop performance

effectiveness criterion variables which are needed to identify the

strengths and weaknesses in tactical doctrine and training.

Ii.2A.1.2 Understanding of the Task

The valid measurement of unit combat effectiveness during

peacetime has been a long-term Army goal. Much has been written

on this topic and many investigation have been conducted, all

directed at combat effectiveness. The need for valid indicators

of combat effectiveness is obvious. In particular, the Army has a

need to determine its readiness state so that weaknesses and

deficiencies can be identified and addressed prior to combat.

Clearly, a valid measure of effective combat performance is a

prerequisite for the analysis the strengths and weaknesses of any

unit.

Prior to the NTC, however, it was virtually impossible to

measure unit combat effectiveness because the units were not

exposed to combat like conditions. Further, the variation in home

station conditions and missions precluded the derivation of a

single definition and measure of effectiveness. With the

initiation of the NTC, the opportunity to empirically measure unit

combat effectiveness became a reality. Since the NTC provides a

limited set of conditions and scenarios as well as a variety of

*" means for collecting performance data, it is possible to define

and measure unit combat effectiveness.

The measurement of effectiveness at the NTC is seen as

involving three critical pieces of information. The mission or

nature of the task at the NTC must be understood so that

effectiveness is measured relative to its performance. Second,

data on the performance of certain critical tasks must be

collected and compared to appropriate standards so that

determinants of performance can be reviewed and critiqued. Last,

the outcome of these task performances must be viewed in terms of e.

absolute and relevant criteria. It is only when all of these

elements come together that one can measure NTC effectiveness.

The first step in the process of measuring unit combat

effectiveness is the determination of the effectiveness of mission 0

1 55

-..
performance. This concept is illustrated in Figure 11-3.

Developing mission effectiveness criteria requires a

determination of measures of effectiveness for each mission

performed at the NTC at the task force and company levels. The

METT-T factors provide a set of dimensions against which the

performance results could be compared to determine the

effectiveness level. The methodology for performing this task is

described next.

(RESOURCE HOME AT MSSION We^ ,.lSTATIO NY R ESULTS

I F'tlT ING * TAIOGll ,'

f t C TOIASCS$I? CATIO, 0 DOCT1l[

*~IC~E It o QU IPPIEN

.r ioss -. o OGAPILATIONS

IIOSC DVEO EFFIC.

Figure 11-3. Developing Feedback Based on Mission

Effectiveness Criteria

II.2A.1.3 Methodology for Task Accomplishment

The methodology for accomplishing this task is through

the undertaking of seven subtasks described next.

1) Subtask 1.1: Develop Measurement Model for

Assessing Unit Mission Outcomes at the NTC.

efetvns The ultimate purpose for measuring unit

effectiveness is to determine its antecedents in order to provide

direction for training and doctrine formulation for the kinds of

units being assessed. Implementation of this kind of an

explanatory research requires the development of a model or

conceptual framework which can be used to guide measurement

development and testing. In this particular subtask, the

requirement is to develop a model for measuring mission outcomes

and evaluating performance effectiveness. The development of this

model will continue the work begun under Task 2, year one, and

will be a refinement of the model depicted in Figure 11-4, below.

56. .
"IWY,O M SI SIRAMASESE WIDS ISINL1I M CIEI

o ". 4 1 , / Scr

/ r.",', 

V, I

Pi ... . SleM*41' a. , O P svl .SPeUO1WI. Of . Pl. t1e1'01 &le Of , Ig€ .. ''

D a...-

ig ur 114__ M d lfos es igMisoO to e

o - ,

CII oql" Ir, .

o I ,°.,o.- 

Figure 11-4. A Model for Assessing Mission Outcomes

2) Subtask 1.2: Establish Performance Conditions.

In order to establish performance standards, we

must first establish the varying conditions under which the task

forces perform their respective missions. To accomplish this

subtask, the battle scenarios used for each of the NTC missions

(and rules for preparing variations) as well as information on

other conditions likely to effect performance such as terrain,

weather, task organization, etc., will be collected, reviewed, and

documented. Our experience tells us that no two sets of

conditions are exactly alike, as a "new" scenario is prepared and

a new set of conditions occurs for each battalion task force.

Therefore, the review process will likely involve 60-90

variations. However, we anticipate that our investigation will

reveal a central core of common conditions that would have such

impact upon expected performance results that they must be taken

into consideration when determining performance standards. These

critical military conditions will be analyzed and specified and a

categorization of similar scenarios will be develooed. ARI will

request that CATA/NTC review the results of this subtask for

suggestions and completeness.

57.

957 ,

a,9:
3) Subtask 1.3: Set Mission Performance Standards.

In order to determine the effectiveness of a unit, its

performance must be compared to some standard or desired level of

performance. Such standards will be acquired from Subject Matter

Expert judgments for each of the task force missions and related

missions of subordinate elements through company team level, as a

function of the scenarios and conditions. These standards must be

expressed in measureable dimensions. ARI will ask CATA to review,

revise, and approve. Figure 11-5, below, is an example of this

requirement.

Establishing the mission standards is the preeminent and most

difficult subtask in this overall task effort. The concept of

establishing mission standards is extremely difficult technically,

as military mission outcomes are situationally dependent. This

problem would be unworkable if we were considering every

conceivable mission for every conceivable scenario in every

theater of the world. However, this task will be feasible by

limiting it to the NTC where the missions, scenarios, and Division

Operation Plans are relatively standardized and the terrain.

OPFOR, and externally imposed stress are a relative constant.

Off) PiS ION: CUM IN SEC1

MISION STND.

A:1~mug Pm~casSTDMS

TOAIN F00" ,MTRATIO Re mI 1 M TeAm ILIWA).

mhowis DkSTNU ?E oDM, INFLIC X1 CAS&MAt19S anim

S Y rollp- DEST"Ov I.e C110" INFLICT XI IM[STUIXTI2M W

C R Towos MNMIZE CASUAL-?:as S~UFF us o O ?u W X& a I LT:~

E I Iam m To U 0m.- CASUALIES.

0 &UIP- MINIMIZE LOSS 100.1e

L mcnl 'oemC To UNDC11TM SUVFC NO MAE TMAN I I LS GU

Y UIIUSI pmIgia.F SIAPOWS SYIMS.

NOT NOMLLY lC

TIME "PS ?am 9M

S INSRCT*OR

Figure 11-5. Example of Mission Performance Standards

58

58 .;
-- - -- -- ---- .. V . V . . ."

4) Subtask 1.4: Determine Measurement Reguirements

which will Result in Measures of Performance.

Once the dimensions of the standards are known we

must then determine the essential elements of information required

to produce MOPs that possess the same metric properties of the

standard so that direct comparisons can be made between them. We

will temper our approach by the availability and quality of the

NTC data and by the capability of the assessment system to provide

the required subjective and objective data. However, requirements

for data not presently available will be accompanied by discussion

of methods for collecting the required information. It will be a

feasible and practical approach and the results closely

coordinated with CATA/NTC. Figure 11-6, below, displays an

example of this requirement.

ATnlha DATA |IQGIROWNTS PASUA4JIS 0i PtNA uCg 1(PP)

I OF E(IM CASUALI(S[: ACTUA. MIsIsON RsusTs

E Aton X 100 CXAMPLC: 43%

4PERScPWLs AT START_______ _______________

m

E IOf IENEV WCAP0001 SYSTEM O(STOVV: AcT1MAL Miss aom Rc~suiTs

0 lE me r v i rfee rnT xI 10 a -1 EXAWPL(: 761

U W[APONS SYSTEMS AT STANT RT

C

E F moT

S a

N EQIP0 lENoT

'i

.,,-o

Y I ''

Figure 11-6. Example of Performance Measurement

5) Subtask 1.5: Design Criterion Performance Effectiveness Indices.

At this point the effectiveness of the unit

performance on a mission can be assessed by using the measurement

structure provided above. i.e., comparing the MOP with the

standard. However, for research purposes these data would be very

59

"% i

' - W r % , % , , , , . ,

." . . . .. ,..., - , .,%.. _.. .. .,,. , ,,.. ., ,,'',,' "". .- " " ," -.," " ," w ,-
.I 

unwieldy, given the number of missions and standards for each

mission. Therefore, a method will be developed for reducing these

scores into a more manageable and simplified effectiveness index.

The outcome of this subtask will be submitted to a review of

appropriate subject matter experts prior to finalization. Figure

. 11-7 shows an example of this requirement.

*1w

MISSION EFFECTIVENESS CRITRA -,

" ATTRIBUTE SCORINO PROTOCOL (SP) POITS WEIGHT OF WEIONTEO

SCO".o ATTA ISUT SCOrE

TEAIN 2

PTs 0 4 2 3 14

TROOPS 1 3 PTs

.E %.ILOss 0-39 -6 70-SO 81-89 90-100

E M PTS 3 1

S Y cuIP- 2 3 6 PTS

0 MwT %LOSs 0-39 '0-69 O 1-69 90"100

R

c

E o,

N

L- QUIP- 2

Y PWPOT

TIME

EFFECTIVENES Tot

INDEX weigh,:" scof#

Figure 11-7. Example of Criterion Performance

Effectiveness Indices

6) Subtask 1.6: Test Measurement System Feasibility.

CATA/ARI will be asked to collect observational

data in order to test the measurement system's feasibility. The

results will be used to refine the system.

7) Subtask 1.7: Prepare Technical Report. N

The measurement model and methodology developed for

its use at the NTC will be documented in a technical report.

S60 -

,..,, - -.
II.2A 1.4 Anticipated Reports and Products.

We will produce a measurement system for assessing the

effectiveness of a battalion task force in performance of its

assigned mission and a research tool for reducing that

effectiveness score into an effectiveness index which will

facilitate efforts in producing correlational analyses between

mission performance and their corresponding task performances in

the fulfillment of that mission.

II.2A.2 Task 2: Produce NTC ARTEP Supplement

I1.2A.2.1 Purpose of the Task

The purpose of this task is to expand and refine the

ARTEP specifically for use at the NTC.

1I.2A.2.2 Understanding of the Task

The mission effectiveness criteria measurement system

provides a basis for determining the contribution of

organizational processes to mission outcomes. To complete this

analysis one must be able to measure the performance of these

organizational processess. This relationship and requirement is

shown in Figure 11-8.

From a training perspective it is not sufficient to know

simply how well a unit performed a mission. rather it is necessary

to examine the components of performance to identify areas of

intervention. Through an understanding of the critical tasks and

their influence on mission effectiveness, one can begin to

determine the strengths and weaknesses of the unit's state of

training and the Army's battle doctrine. The resultant

information profile then allows training to leverage the Army's

readiness.

114,1/ Gha.PV

-IOCfSS AT PROCESS TC

STATION NT RErr .SULTS

IDENTIFY & MEASURE

ORGANIZATIONAL

PROCESSES AT ETC

Figure 11-8. Measuring Organizational Processess at NTC

61 JCL
The ARTEP is the heart of the Army's training program and as 

such should contain the critical tasks. conditions and standards 

for specific missions to which units. such as battalion task 

forces. can train. The limited and specific focus on NTC 

performance. here . allows for a more specific and narrow document 

than the current ARTEP. Oeta 1led analyses of tasks critical to 

NTC missions. under NTC-specific conditions. for all elements of 

the battalion task force must be developed in order to derive this 

NTC supplemental ARTEP . The completion of the NTC supplemental 

ARTEP will allow one to relate doctrinal task execution to unit 

performance effectiveness . Further . mission conditions will be 

considered in the establishment of the standards for the critical 

tasks included on the supplement . 

The methodology for accomplishing this task is through the execution of eight subtasks described below . 

1) ~YQ!~l~ 1~1~ Q!Y!12~ M!~lYr!m!n! M29!1 f2r ~Y~1Y~!ing ~ri!i~~l I~l~ f!rf2rm~n~! ~! !h! ~I~~ 

A model will be developed to provide the framework 

in which the critical tasks will be analyzed . This model, when 

operationalized. will provide a measurement system for use in 

evaluating critical task performance. The approach will be 

similar to that taken in Subtask 1.1 above resulting in a 

measurement structure resembling that in Figure II-9 below . 

.US'S ION : CO'DC) IN SEC101t 

TIISIC STNCMJS PVFCJIWG "'L§fC!QIOT. T8 EFJ'tt'TM)OS Oti'I"DIA 

\ 1 I (l) I)) , .. ) (}) (t.) (7) 

Ouu~ STA.CAitOI OaT a NAIUI'II Oil Scaltl .. P'olllfl ..... y., 

Tal&l -OUIIIUC.fl P'PIOIUWCI PitoToca Scala Tau 

' 

-- £_--- -£---- -£---£_. -£.-~-z 

Fi gure II-9. Example of Structure for Measuring 

Performance of Miss i on Crit i cal Tasks 

62 

(8) 

*a-;~u SC01t 

7 -

-7 

._ 
The key features of the measurement structure will include a 

list of critical tasks and standards based upon mission 

conditions. measures of performance. and measures of task 

effectiveness. It is anticipated that research will reveal a 

variability in the contribution of critical tasks to mission 

requirements within and across missions: i.e . . some tasks will be 

more important to mission accomplishment than others . Thus the 

relative weights of each critical task will probably have to be 

determined in order to equalize their ind i vidual contribution to 

mission results . 

2) ~~Q!!i! l~l~ ~~i~tiQ~ MiiiiQn ~ri!i~!l l!i!i i!nQ A~!l~i!i~il fQt !h~ l!i! EQt~~ QQ!n !Q fl!!QQn b~~~l fQt ~!~h ~I~ 

MiiiiQn~ 

Using military SHEs and doctrinal publications 

relating to task force operations. an analysis of the critical 

tasks will be performed for each mission at the NTC down to 

platoon level. It should be noted that critical tasks are viewed 

as any unit function essential to mission performance . In this 

report. leadership functions that are unit responsibilities and 

essential to mission success would be viewed as critical tasks. 

These leadership or commandership behaviors are seen as being 

distinct from individual leader characteristics or behaviors and 

rather as organizational processes. 

3) ~~Q!!i! l~~~ R~~i~! ~~rr~n! ARI~fi~ 

The most recent ARTEPs for Battalion Task Forces 

and below will be reviewed for accuracy and completeness based on 

the critical task list developed in subtask 2 . 2 above and 

conditions specified for each mission from Subtask 1. 2. Specific 

gaps or potential inaccuracies will be identified . ARI will 

present the results to CATA for review . 

The gaps identified in the ARTEPs in subtask 2 . 3 

above will be in terms of tasks. conditions. and standards which, 

when identified. will result in an NTC-focused ARTEP. In essence . it will represent a supplement to the current ARTEP for use at the 

NTC in accordance with the provisions in the ARTEP for CORs to 

adapt the written ARTEP to their own local conditions. This 

subtask. therefore. requires the identification of the tasks. 

conditions. and standards that are missing from the current ARTEP . The conditions will be based on performance conditions documented 

for mission scenarios while the standards will be established by 

SHEs . ARI will submit for review, revision, and approval by CATA. 

The expanded/refined NTC-focused ARTEP and its 

development methodology will be documented. 

63 
Two areas of potential major difficulty exist in the

execution of Task 2:

1) Developing a "supplement" to the ARTEP. Previous 'J

contract work with ARI on the subject of unit performance and

ARTEPs revealed inadequacies in using the ARTEPs as a training

document, particularly as they apply to the NTC. We expect,

therefore. that considerable "restructuring" of the ARTEPs will

result from this effort.

2) Setting performance standards to mission

conditions. If the research effort reveals a large number of

performance conditions for mission scenarios, The ARTEP supplement

and observer guides could become cumbersome and impractical

documents because of the multitude of conditions they would have

to address. At this time, we do not anticipate that we will find

an unmanageable number of mission- related conditons based on our

current knowledge of NTC scenarios. However, should the opposite

be the case, we are prepared to reduce them down to more

manageable numbers by developing a scheme for categorizing the

conditions.

II .2A 2 .4 AnticiRated Reports and Products.

This task will result in the publication of an NTCfocused ARTEP containing critical mission tasks, conditions. and

standards which, among other uses, will enable CATA to relate

critical doctrinal task execution to unit performance

effectiveness. This task will also provide a methodology for

developing ARTEPs.

.

.9"

' -.p,

.-

-- a
Milestones for Task 2A 1986 1987

MAR APR MAY IJUNEIJULYIAUG SEP OCT INOV DEC IJAN IFEB

Task 

Develop Unit

I

Combat Effectiveness

Criterion I I I I I

Variables

SubtasKs

% . . . o

Develop

Measurement

Model

1.2 ------------------- 0

Establish

Performance

Conditions

1.3 o--------------o

Set Perform- I

ance standards I

.14 ...- o

" Determine

Measurement

Requirements

Design Criterion Perform-

'" mance Indices ""

1 6 I --------------- 0o

.Test System

Feasibility

1 7 .. .

Prepare Technical Report

6.5

be

q

65
Milestones for Task 2A (Cont) 1986 1987

MAR APR MAY JUNE IJULYIAUG SEP OCT NOV DEC JAN FEB I

Task 2

Produce NTC

ARTEP Supplement I I

Sjbtasks I I

I I

----------------------------------------- 01

Develop

Measurement

Model

22 1 ---------- 0

Describe

Mission

Critical!

Tasks I

2 3 0

.Review

Current

ARTEPS

24 o

Draft TaskS. I I

Conditions. .

Standards I

a..P

4r%

-'%_

4"
TASK AND TIME MATRIX (PERCENT OF TIME) 

YEAR TWO 

TASK TASK TASK TASK TASK TASK 

NAME 6 1 8 9 10 2A 

T .J. RITENOUR 10 5 10 65 5 5 

PROGRAM MANAGER 

W.J . DOHERTY 10 30 15 15 15 15 

SENIOR SCIENTIST 

T. K. FORSYTHE 10 10 10 5 5 60 

TASK LEADER 

T.L. AVANT 5 10 60 10 5 10 

TASK LEADER 

J . A. BRISCOE 75 5 5 5 5 5 

TASK LEADER 

J .J . NICHOLS 10 65 10 0 5 10 

TASK LEADER 

RESEARCH PSYC 5 20 10 10 5 50 

MIL. ANALYST 0 10 0 0 5 85 

MIL. ANALYST 0 10 0 0 5 85 

E.A. KELLEHER 15 20 20 10 15 20 

ADMINISTRATION 

I. T.R. KEMPER 25 20 20 20 5 10 

RESEARCH ASST 

I • 

c 

67