— ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

T c hs*~ Paper 312

A PARTIAL VALIDATION OF FORECAST

ENGAGEMENT SIMULATION EXERCISE

OUTCOMES

Steven M. MedIIn

ENGAGEMENT SIMULATION TECHNICAL AREA

‘H1

.

_

[

~

iri)

U. S. Army

Research Institute for the Behavioral and Social Sciences

August 1979

Approv .d for public r.Iuus; di stributio n unilmitid.

__________________ 

~~~~~~~~~ I
BLANK PAGES 

IN THIS 

DOCUMENT 

WERE NOT 

FILMED 
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating A gency under the Jurisdiction of the

Deputy Chief of Staff for Personn el

WILLIAM L. HAUSE~ JOSEPH ZEIDNER Colonel , U S Army Technical Director Commander

NOTICES ‘

OISTRISUTION: Primary distribution of thss ‘sport has been made by A~ I. Please address correspondence cenesrnin~ dl$trlbufi

~

qt of reports to: U. S. Army ~sIserdt lnststuts for ffis SuItdooret and Social Scistiass. ATYN: PERI.P, 5001 Eigsnhovmr AVSci*. Alex.ndii.. Virgi ni. 22333.

FINAL DI~~OLTi~~d: This report .

~

‘sy bi destroyed often it is no ion sr ~~sdud. Pies., do not return it to

the U. S. Army PSISirCh Institute for the SUItIViSI-SI and Social Sdsnoss.

~~~~ The fin*n~s In ails report irs not to be ceflstnlsd as an official Dsp..tmsnt of tite Army position. unless to dssipo.ted by Other authoriasd documents.

111111111 ~~—‘-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ______ LI
A ,

~~-- - - - v—---

I, 

~~~~~~~~~~~~~~~~~~~ — —*.--- -~ - _______________

~~ Unclassified

SECURITY CLASSIPICATION OF THIS PAGE (Wha, Oat. Eaft,.d) _________________________________

~~~~~~~~~~~ Il ~~~ I IU ~~~~~~~ A ~~II

~ IJ 

~ READ INSTRUCTIONS i

~grv,~. IU’.,U ~~~~II I 

~~ I lUl ~~~~~~~~~~~ BEFORE COMPLETING FORM

1. REPORT NUMBER 2. GOVT ACCESSION NO S. RECIPIENT S CATALOG NUMBER

I Technical Paper 3B2 ____________________________

1. TITLE (aid Sublgtl.) . 

~~~~~~~~ REP~IRY ° 0 COVERED P ~ARrIAL ~ ALIDATION oF FORECAST ENGA(~

EMEN~~7 

interim

,SIMULATION EXERCISE OUTCOMES . 

~~~ / ~ 

NUMBER

7. AUTHO~~~~ S. CONTRACT OR GRAN T NUNBER(.)

~~~~~~~~~~~~~~~ M. Medlin 

0. PIRFOtoSING ORGANIZATION NAME AND ADDRESS 

____

tO. PROGRAM ELEMENT. PROJECT. TASK

U.S. Army Research Instituj ê f or the Behavioral~.

.. _____________

and Social Sciences 

~ 

(

~

‘

~ ‘ 2 762722A764 5001 Eisenhower Avenue, Alexandria, VA 22333 \..~.

a Q

II. CONTROLLING OFFICE NAME AND ADDRESS $2. ~~~~~~ ~~— 7

Deputy Chief of Staff for Personnel (jj~ 

Au~~~~ ~~79

Washington , DC 20310 1

~~. S

14. MONITORiNG AGENCY NAME C ADDRESS(U dSSl.rait from ControUlnj OWe.) IS. SECURITY CLASS. (of flit. a .po,t)

~~~~~~~~~ 

tS&~~~~; :~~ &tCATIONI0OWNGRA0ING

IS. DISTRIBUTiON STATEMENT (.fffiS. R~~ort)

Approved for p ic release; distributi~~j~~ imited.

/

$7. QISTRISUTION STAT ~

_f L. JL I It~ 11. It ((stint from R.port)

IL UPPtEN~~~TARY NOTES

IC. KEY WORDS (Caitffiu. on tivati. .ld. It n.c..omy aid Id.ntlfy by block .naib.e)

Forecasting

Performance benchmarks

Engagement simulation (ES)

2$ A~~.

~~Acr ~~~~~~~~~ om revises W _, N.~~

lfr by block nomher)

“rhe research evaluated the validity of board war gaming as a forecasting

technique to determine behavioral benchmarks. Using data from the Combined

Arms Test at Fort Carson , Cob ., March 1978, and from the Fort Carson Forecaiting Game, military judges were asked to distinguish between field exercise

data and forecast data. Judgments were made on maps of exercise maneuver

routes and tables suitmtarizing data on casualties suffered and weapon systems

inflicting casualties. The military judges were not able to disting~

’iish

~~~ 

W3 i°’nos ~~ MO~ U ~ Unc1a

~~~~~~~

*0

~~ 

\

~

__ - -- - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
‘

~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unclassified

UQ NTY ~1.A1mPSCATIQs( OP THIS PAGE(2$.. Data &lNes~~

Item ~O (Continued)

‘

~~~etween field and forecast maneuver routes, and they tended to classify

forecast casualty data as field exercise casualty data . Board war gaming

provides realistic process and product data that cannot be distinguished

from field exercise data. Although further validation research needs to

be conducted , board war game forecasting has great potential for use in

developing performance benchmarks against which unit performance can be

compared.

— —. i

i

.

j

~ iou F!r

i

~~~~~ GRA&I

DDC TAB

iinaxmouxlced

Justific3ti0~.__~~ —

___.. —

By ———— -

‘

pis tribuiW!L -

j~

vailabilY ’ $ ‘

Lvailr ’

~~0~’

~~~~ 

spec~a1

_

•\

__ -. --——

~~

. 

~~~.

.

. ~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

-- I~ c1aaaif L d
~~~~~~~~~~ - --

..

~~ - *-

Technical Paper 382

A PARTIAL VALIDATION OF FORECAST

ENGAGEMENT SIMULATiON EXERCISE

OUTCOMES

Steven M. Medlin

Angelo Mirabella, Team Chief

Submitted by: Frank J. Harris, Chief

ENGAGEMENT SIMULATION TECHNICAL AREA

Approved by:

A. H. Bimbaum, ActIng Director

ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria , Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

August 1979

Army Project Number Combat Performance Standards

20712122A754 and Evaluation

A.,oroved for publIc rdsass; distribution unlimited.

(ii

~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ [. 4
I

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military agencies by briefing or Disposition Form.

‘V
- - - - -

FOREWORD

The research presented in this paper was conducted under the Train—

ing and Education project in the Engagement Simulation Technical Area

of the Army Research Institute for the Behavioral and Social Sciences

(ARI) . The goal of this project is to provide quantitative methods for

evaluating unit proficiency. The means for achieving this goal include

basic research in criterion-referenced test methodology , measurement

and scaling models, and decisionmaking implications of test score

interpretation .

Related , ongoing programs within the Technical Area include evaluation of small combat units under simulated battlefield conditions (REAL—

TRAIN , ARTEP) , qualification of tank crews and platoon gunnery (IDOC),

and improvements in the reliability of ARTEP evaluation.

Anticipated future research under the Training and Education project

includes the development of a computer model for performance evaluation

and development of measurement , scaling , scoring , decisionisaking , and

quality—control models for use in performance evaluations when criterionreferenced testing procedures are employed .

ARI research in this area is conducted as an in-house effort, responsive to the requirements of Army Project 2Q762722A764. Without the

invaluable logistics support of the 4th Battalion of the 40th Armor

from Fort Carson , however, the project would not have been possible.

( *~ P

~

~‘u.~~ chnical Director

I -

~

!

~

— - - ~~~~~~~~~~~~~~~~~~ ~ . ~~~~~~~~~~~~ . - 

~~~~~~~ .~~~~~~~~~~~~ -
~~~~~~~~ ~~~~~~~~~

— - - —- . - — ——

A PARTIAL VALIDATION OF FORECAST ENGAGEMENT SIMULATION

EXERCISE OUTCOMES

BRIEF

Requirement :

To evaluate the validity of board war game forecast data for use

in determining benchmarks or standards against which unit performance

in engagement simulation exercises can be compared.

Procedure :

Military experts attempted to distinguish between engagement simulation exercise data from the Combined Arms Test at Fort Carson , Cob .,

March 1978, and board war game exercise data generated by the Fort Carson Forecasting Game . An inability to differentiate between field and

forecast data suggests that the two data sources can be considered to

be identical and, consequently, that generated data can be used to devebop behavioral benchmarks.

Findings:

Judgments were made on maps of maneuver routes and tables suainarizing casualties suffered and wea~~~ iystem inflicting casualty data . The

military experts were not abLe to correctly classify maneuver route information as - ‘1 ~

field) or simulated (game board) data. Furthermore ,

they tended I -

~issii ’ simulated casualty data as real.

Utilization of Findings:

Results from this study indicate that board war gaming provides

realistic process and product data that military experts cannot distinguish from field exercise data. With continued research on its validation , the forecasting procedure can be used to develop performance

benchmarks with which unit performance can be compared.

- 

- 

~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- - -~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . - - - - - - - -

- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ___

A PARTIAL VALIDATION OF FORECAST ENGAGEMENT SIMULATION

EXERCISE OUTCOMES

CONTENTS

Page

INTRODUCTION 1

METHOD 3

Participants 3

Design

Procedure 4

RESULTS 4

Maneuver Routes 4

Casualty Data 6

DISCUSSION 7

APPENDIX A. SUMMARY OF BACKGROUND INFORMATION ON PARTICIPANTS . 9 —

B. RESPONSE SHEET 11

C. MANEUVER ROUTES FROM CATEST FIELD EXERCISES 13

D. MANEUVER ROUTES FROM FORT CARSON FORECASTING GAME

EXERCISES 17

E. FIELD EXERCISE CASUALTY SUMMARY 21

F. BOARD GAME CASUALTY SUMMARY 23

DISTRIBUTION 25

LIST OF TABLES

Table 1. Distribution of number of correct responses 5

2. Frequency that real and simulated data were

classified real and simulated 6

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~ lli
-. ~~~~~~~~~~~~~~~ ~~~~ - 

- ~~-—-~~ ----- - ______—

A PARTIAL VALIDATION OF FORECAST ENGAGEMENT - SIMULATION EXERCISE OUTCOMES

INTRODUCTION

The Army Research Institute for the Behavioral and Social Sciences

(ARI) is conducting research on improved training and evaluation systems

for small combat units. One system currently under development is tactical engagement simulation (ES), a two—sided tactical field/maneuver exercise. To evaluate unit performance in ES exercises, performance benchmarks or standards must be defined. The dynamics of two-sided field

exercises do not permit exact, deterministic standards , however. The

standards must be in the form of probability distributions, tolerance

limits, or principle-derived sets of correct solutions to tactical

problems.

To establish standards in one or more of these formats, large

amounts of ES outcome data must be obtained so that the characteristics

of the distribution of ES outcomes can be determined. Because replication of field exercises is diff icult and costly, the distribution of

ES outcomes cannot be generated in field exercises. As part of the ES

research program, therefore , inexpensive simulations of ES exercises

are being developed to provide large amounts of valid ES outcome data.

In one of the current modeling efforts, the Combat Operations

Training Effectiveness Analysis Model (COTEAN) , the concept of situation—

specific forecasting is being used to develop an ES outcome data base

from which the characteristics of the distribution of ES outcomes can

be determined. Situation-specific means that the forecasting procedures

replicate the particular field exercise conditions as closely as possi—

ble ; a forecasting exercise uses the caine force ratios, weapons mix ,

terrain, weather conditions, and missions as the corresponding field

exercise.

The purpose of the forecasting efforts is to generate a data base

of ES outcomes from which expectations about tactical processes and

casualties can be derived for units participating in ES field exercises.

The methods being developed or adapted for generating the data base ind ude (a) military experts’ DELPHI , (b) board war games, and (c) coinputerized ES models. According to the COTEAM model , if the forecasts

obtained using these methods agree with those observed in ES field cx—

ercises , then these methods can be used to develop performance criteria

against which unit performance in ES exercises can be compared.

In assessing the similarities between forecasting data and observed

field data , differences can be determined either statistically or using

—- j— --—~—~

-

~

-—-——---- ~~~———----- ——~ —-

~—- . -~-- .

- ---~--~-—— --~~~— - —~—--.—-- -- .—. -—-- --S— -~-- - -- ~~~~~~~~~~~~~~~~~~~~~~~~~ --- -

~~

-

~ -.-
the Turing Test.1 The Turing Test states that if outputs from two different data sources are given to an information processor , and the processor cannot distinguish between the two data sources, then the data

sources are considered to be identical.

Recent work by ARI suggests that behavioral forecasting is a viable means of developing an ES data base. During the REALTRAIN validation of rifle squads at Fort Ord, Calif., in May 1977,2 military experts

made forecasts about the outcomes of specific ES exercises. The data3

indicate that experts can make this type of forecast and that their

forecasts are sensitive to such factors as the training and combatreadiness level of the unit being evaluated.

At the armor Combined Arms Test (CATE S

~ ) at Fort Carson , Cob .,

from January to April 1978, board war games were used to collect forecasting data about the ES field exercises. Using the Fort Carson Forecasting Game, a board war game specifically designed for the terrain

and scenarios used in the CATEST, board exercises identical to ES field

exercises were conducted. The results were assessed in terms of the

tactical maneuver routes, casualties suffered , and casualties inflicted

by each type of weapon system. The maneuver routes from the two types

of exercises were comparable; however, the field maneuver routes tended

to be slightly more complex than the board game routes. The casualties

suffered were almost identical for both types of exercises, and the

casualties inflicted by each type of weapon system were also similar.

The only sizable differences were the percentages of casualties inflicted

by tanks and casualties inflicted by artillery . These differences were

attributed to slight variations in the way ES rules were enforced in the

field exercises and in the board games.

The two studies (Mirabella , 1977; Medlin , 1979) established the

feasibility of forecasting as a means of generating an ES data base.

The data can be generated, and they closely resemble the data observed

in field exercises. But how closely do they resemble field data? Do • forecast data agree with f ield data enough to be used to develop

1

~

Turing , A. M. Computing Machinery and Intelligence. Mind, 1950 , 59 ,

433—460.

2

Banks, J. H., Hardy, G. D., Scott, T. D., Kress , G., & Word , L. E.

REALTRAIN Validation for Rifle Squads: Mission Accomplishment. ARI

Research Report 1192, October 1977.

3

Mirabella , A. Criterion-Referenced System Approach to Evaluation of

Combat Units. Paper presented at the 19th Military Training Association, San Antonio, Tex., October 19, 1977.

4

Medlin, S. M . Behavioral Forecasting for REALTRAIN Combined Arms.

ARI Technical Paper 365 , May 1979.
~~~~ ~~~~~~~~~~~~~~~~~~ • - ---•--~~~~~- . • -- - -

behavioral benchmarks or standards? Ultimately , the final decision is

a subjective one, but further research can help to establish the validt ity, or the ~izni1arity, between field and forecast data. As Medlin

(1979) noted, because of the severe restrictions on sample size, no inferential statistics could be used to assess differences between the

two types of data. Furthermore , since much of the data collected is

“process” data (for example, on maneuver routes, firing positions, and

overwatch positions), the results of statistical analyses may be mis—

leading even if such analyses can be performed . Any of several maneuver

routes, firing positions, or overwatch positions may be reasonable for a

given exercise . Statistical analyses may show significant differences

that are not in fact meaningful . Thus , it is not important to demonstrate

statistical differences; rather, it is important that the forecasting

procedure should generate expectations that are not significantly different to an information processor or a military expert from the outcomes

observed in field exercises.

The present research attempted to partially validate, or establish,

the similarities between forecast data and ES field exercise data by

subjecting the data to the Turing Test. The study was to determine

whether military officers can distinguish between forecast and field

data (for example, the maneuver routes, casualties suffered , and casu—

alties inflicted by weapon type) from the CATEST field exercises and

from the corresponding board war gaines. As the Turing Test is applied

H in this context, if military officers cannot distinguish between generated and observed ES outcome data and cannot distinguish between the

two sets of process data (one forecast and one collected in field exer—

cises), then the two data sources are considered to be identical. This

result would provide further support for the use of forecast data as a

means of developing performance benchmarks or standards against which

unit performance can be compared .

METHOD

Participants

Sixty—six military officers from the maneuver arms units at Fort

Carson , Cob ., served as participants for the study (see Appendix A).

The officers participated in the experiment as part of their required

weekly leadership training class.

Design

Boc~

-klets were constructed containing a background information

sheet (Appendix A), a response sheet (Appendix B), three maps of maneuver routes from CATEST field exercises (Appendix C), three maps of

maneuver routes from the corresponding Fort Carson Forecasting Game

exercises (Appendix D), a table sununarizing CATEST field exercise casualty data , i.e., casualties suffered by weapon systems inflicting

L 

~
-

~~~~~~~~~~~~~~~~~~~~~~~~

—

• casualties (Appendix E), and a table sunnarizing forecast casualty data

(Appendix F) -

Excluding the background information and response sheets, the material in each booklet was randomly ordered to eliminate possible order

or sequential effects. The background information sheet obtained information that might be related to an officer’s ability to distinguish

field exercise data from board war game data ( for example, rank , combat

experience , REALTRAIN experience, board war game experience, years in

service , and education) .

The response sheet provided space for the officer to indicate

which map or table he was considering and whether he thought the data

were real (field exercise data) or simulated (board war game data) . The — response sheet also provided a line marked from zero to 100 in incre—

ments of 10 on which the off icer was to indicate “how much confidence”

he had in his decision about whether the data were real or simulated.

Finally, the response sheet provided a space in which the officer could

indicate whether or not he was guessing, or , if not guessing, to indicate which factors influenced his decision. The six maps of maneuver

routes were copies of observed field exercise data or the generated game

data for the same exercise. The two tables of casualties were the summary table for the CATEST field exercises and the summary table for the

Fort Carson Forecasting Game exercises.

Procedure

Each officer received a booklet containing all the necessary materials. The experimenter described the task to the officers and ex1 plained the background information sheet , response sheet , map data , and

casualty data in considerable detail. Questions were answered as coinpletely as possible without compromising the integrity of the experiment.

Each officer then filled out the background information sheet , studied

the map and casualty data , and filled out the response sheet. When all

officers had finished and all booklets were collected , the experimenter

debriefed the participants.

RESULTS

Maneuver Routes

Each participant was asked to indicate whether each of six maps

of maneuver routes depicted real (field exercise) data or simulated

(game board) data . Half of the maps were from field exercises and half

were from game board exercises. If participants were not able to distinguish between the two types of data , they should guess correctly on

three of the six maps. The mean number of correct responses was 2.99.

Out of 396 responses (six for each of the 66 officers) , 197 were correct ,

Qnly 1 less than the 198 expected by chance . Table 1 shows distribution 

• - — ~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- -- _____ 
I

I

of the number of correct responses. Most of the participants (50 out

of 66) responded correctly on two, three , or four of the six maneuver

route maps.

• Table 1

Distribution of Number of Correct Responses

I correct responses Frequency

0 1 —

1 7

2 18

: 3 19

4 13

5 3

6 5

Because of the difficulties inherent in trying to “accept ” the

null hypothesis (see Greenwald5 or Bakan6 for a discussion of the issues

involved) , confidence intervals for the “true” mean number of correct

responses were constructed . The 99.5% confidence limits are 2.54 and

3.44 , the 99% confidence limits are 2.59 and 3.39, and the 95% confidence

limits are 2.71 and 3.27. Thus, one can be 95% certain that the “true”

population mean lies between 2.71 and 3.27 .

Even though the overall mean of correct responses is 2.99 , it is

possible that sane systematic relationship exists between the real and

simulated data and the responses of “real” and “ simulated” given by the

subjects. In the worst possible case , participants could correctly

identify all the real exercises and misidentify all the simulated ex—

ercises, yet the observed mean would still be 3.00. As Table 2 shows,

there is no systematic relationship between the source of the data and

the participants’ responses about the source of the data (X2 1.70,

df — 1, n.e.).

5

greenwald , A. G. Consequences of Prejudice Against the Null Hypothesis.

Psychological Bulletin , 1975, 8(61), 1—20.

6

Bakan, D. The Test of Significance in Psychological Research. Psychological Bulletin , 1966, 66(6) , 423—437.

~~~• •~~~~~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~
• • —- ---

~

Table 2

Frequency That Real and Simulated Data Were

Classified Real and Simulated

Real Simulated

Classified as real 105 106

Classified as simulated 93 92

The above results indicate that, overall, the participants cannot

distinguish between real and simulated maneuver routes. Certain individuals, however , may be able to discern one type of data from the

other because of combat experience , years in service , or some other

experiential background. Statistical analyses reveal that the number

of correct responses does not differ as a function of rank , combat experience , REALTRAIN experience , board war game experience , familiarity

with the exercise test lanes , years in service , or education.

Each participant also indicated how certain he was of his response

and if he was guessing. The number of admitted guesses does not differ

as a function of any of the background information , and the mean conf idence response of the participants differs marginally (t -2.18, df = 64 , p < .033) as a function of REALTRAIN experience only. Officers

with REALTRAIN experience were more confident of their responses than

were participants with no REALTRAIN experience. Furthermore, neither -: confidence nor admitted guessing varied as a function of the correcthess

of the response.

Casualty Data

Participants were also given two tables, one from field exercise

data and one from forecast data, that provided summary information on

the casualties suffered and the casualties inflicted by each weapon system. The participants were asked to indicate which table contained data

f rout field exercises. If they were unable to distinguish between the

two types of data, the participants should have guessed correctly half

of the time. Only 53 of the 66 participants responded to this question;

of the 53, only 20 correctly determined the field exercise casualty

• data (X2 = 3.57, df — 1, p < .062). Thus, there is tendency for the

participants to classify the real data as simulated. None of the background data was related to the participants’ ability to correctly alascity the casualty information.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •—~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •--

~~~~~~~~~~~~~
________ - 

~

•——- •

~

1

DISCUSSION

As part of ARI’s research on improved training and evaluation systems, an evaluation system for two—sided tactical field/maneuver exercises is being developed. To evaluate unit performance and assess the

units’ combat readiness and/or training deficiencies and proficiencies, performance standards or benchmarks must be defined. The COTEAM model

uses the concept of situation—specific forecasting to provide benchmarks.

Because of the nature of the two—sided tactical play, :lowever, the standards cannot be exact or deterministic; they must be in the form of

probability distributions, tolerance limits, or principle—derived sets

of correct solutions to tactical problems. After these expectations

about casualties and tactical processes have been established, ES outcomes can be compared to them to assess deficiencies and readiness levels.

This study was designed to partially validate a board war game forecasting procedure by determining whether military officers can discriminate between field exercise data and data generated in a board game . Using the field exercise and generated forecast data from the CATEST at

Fort Carson, Cob ., military officers attempted to distinguish the field

maneuver routes from the simulated maneuver routes.

The results suggested that officers could not distinguish the simulated data from the real data, and that a participant’s experiential back- • 1 ground was not related to the number of correct responses obtained. Furthermore , neither estimated confidence in the response nor admitted

guessing were related to the correcthess of the response. R.EALTRAIN experience increased a participant’s confidence in his response, but no

other background data were related to confidence or admitted guessing. When considering the casualty data, the officers correctly identified

the field exercise data in only 20 of 53 cases.

In informal postexperimental question-and-answer periods, the officers said the ES field exercise data seemed unrealistic. They doubted

that infantry and grenades could destroy tanks, TOWs, and APCs as was

evidenced in the field exercises (Appendix E). They also felt that tanks

had too little impact on the field exercise casualties.

The game board data may have seemed more realistic because the officers were not aware of “realistic” casualty data or because the REALTRAIN casualty assessment rules were not appropriate.

In either case, more realistic casualty assessment techniques for

ES exercises will be provided in the Multiple Integrated Laser Engagement

Simulation (MILES) system, and the forecast game will generate data more

closely resembling the field exercise data by strictly adhering to the

MILES casualty rules. These developments should eliminate the major discrepancies between the field and forecast casualty data.
~ 

— — •- —

‘ I ’

The results of this study suggest that military officers cannot

distinguish field exercise data from forecast data generated in game

board exercises . According to the Turing Test , therefore, these two

data sources may be considered to be identical. Game board exercises

generate data that are similar enough to field exercise data to be used

to determine the characteristics of the distribution of ES outcomes

which, in turn , can be used to define performance benchmarks or standards

for units participating in ES field exercises.

Although the initial steps in validating the board war game forecasting procedures have been taken, more research is necessary before

forecasting can be espoused as a valid, reliable means of generating

data for the purpose of establishing behavioral benchmarks or standards.

• The effects of different levels of training (for the forecasters) on

forecasting must be assessed and compared to the effects of different

training levels (for the unit) on ES field exercises. If forecasting

is to be used to provide behavioral standards, the forecast data should

reflect the same types of differences due to training level as are cvi—

• dent in field exercises. As a long-term goal, it would be fruitful to

develop learning curves for ES field exercises and forecasting exercises

and then compare the two.

A short—term research effort can be directed to developing a relatively large data base for a few selected exercises and then comparing

the results observed in the field with the generated expectations. Provided sufficient manpower resources are available, considerably more

data than maneuver routes and casualties can be collected and compared

with the ES field exercise data. These data may include detection distances and locations, firing patterns of vehicles, indirect fire placement, or communications.

It is then of considerable interest to determine how much and what

types of information are necessary for military officers to distinguish

between field and forecast data . Determining this additional information will result in a better understanding of the dynamics of tactical

behavior and an improved set of forecast procedures. Successive itera—

tions and modifications of the forecasting procedures will help to de—

velop a realistic data base from which performance standards can be

derived .

I 

_ •• a_•

 •~ _______
~ 

~~~~~~~~~~~~~~~~~~~~ •

~

v~

’-

~

___’ 

_

~~~~~~ -_..w. - -

~~~~~ - -~

APPENDIX A

SUMMARY OF BACKGW)UND INPORI4ATION 3N PARTICIPANTS

I

Rank: 2LT 1LT CPT MM LTC COL

22 22 13 3 3 3

Combat experience : YES NO

12 54

REALTRAIN experience: YES NO

40 26

Board war game experience: YES NO

51 15

Familiarity with test lanes: YES ~~ 29 37

Years in service : LESS THAN 1 1-2 3-4 5-6 7-12 MORE THAN 12

2 28 16 7 7 6

Education: H • S • COLLEGE POST-GRADUATE

5 7 4

T
- - - ~~•

-•••••-

~~ .~~~~~ -- -

~~

_ _ _ i

~

-- -

~~~ ~~~~~~ - - •—.‘.,.- - ‘r~-

~~ •

~~~~

•

~

•

• p•• —

~~ 

-- -

~~~~~~~

-—- - --—— - -

~~~

- - •

~~~~~ 

______

IT

I- e —~ 1.4 0

O s

41 0 w~ 4J >

41 114

1.4 114

4~~ 4

00 00 00 0 0 00 00 00 0 0

0 4) 0 41 04 ) 0 41 0 41 0 41 0 4 ) 0 4)

‘-I ’d .-4 ’Ø ,4’ .14 .4 .14 •r. .14 .14 ~ .14

11.1 114 114 144 ‘44 ‘4.4 ‘44 144

o c o~~ o~~ o~~ o~~ o~~ o c o

H 0 N O N O NO (11 0 NO NO NO NO~~ C) 0 0 0 C) 0 0 C)

Li o~~

. 0 0 0 0 0 0 0 0 m

41 .14

041 0 0 0 0 0 0 0 0

14 ~~ 14 ~~ ‘ ~~ ‘ ~~ ‘ ~~‘ 1’ ~~ ‘

4 1 0

U) 0 0 0 0 0 0 0 0

I IL I I I

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - •- -~~~~~~~~~ -— —---
-

~~~~

__ _ _ _ _ _ _

APPENDIX C

I

. MANEUVER I

~X7~ES FI~ M CATEST FIELD E]~~RCISES

~~~~~~~

ci 

~~~~~~~~~~ 

: ‘ ~~~~~~~~~~

~ 7 .i L

~~ ~ 

L.

~ ~~ 

:: ‘

~:* L~

LJ ~ • ~~~~~~ 

%‘.

~ 1 ‘

~

A~~%~~~ :

~~~ l ~~~~~

‘ 

S .

~‘ ~~ ~~~~~~~~~~~~~

I” ~~~~~~ ~~ g

/ .

~~.~~ • ‘ f r’ C U 

~~ 

I

. ~ / ~~~~~~~~~ ~~~~~~~ ;5 

~~ ~~~~ t • S. 

• ~ ~~~~~ ‘

I ,;‘1; a

~’ — ~ ~i — I

• — • — • .‘ . . 1.l’••

~ I .

~

. J ‘/

~~: / (

~~~~\ (

~ ~~

I

1’ AUrsssoT 

I !E

I .-,

l ~~~~~~~~~ k r

~ ’i ?orcsz

________

~~~~~~~~~~~~~~~ 

-

~~~ •— —•— ~~~~ ~—•- —•——
I

- . • — 

•

~~ -

• ~~~. 

~~ —

~~~

• ••.

‘ 

— .

~

. - ~~~~~~~~~~~~ • 

~

I’•r_.•s •f ~~~~~~~~~~~~~ ~( ~~~~ I -

~~~t~ j=,;~ —~~

• ~~~~~~~~~ I

~~ ~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~ J.4

~ ~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~~~~ . ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ k-. - -•-. ~— s.~~~ .- -

5:

~ 0 ‘N t

~ 

- 

U ~~ ~ ~~~~~~~~~~~~~~~~~~~ / -

-

~~~~~~~~~~

~~~~ ~~

4.

~-4~J i

~ ~ h?>? L/.

;7 )

~~ / ~~~~~~~~~~~~~~ ~~~~~~~~~~~ ‘~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~— ‘

~~~~~

-

~~

- ? r-.. •.

~~~~ 

,:

~~‘~ —~

r ~

. 

~

5 ~~~~ PC T~ 

~~~~ ~~~~~~~~

1

~~~~~~~~~

~~~~~~~~~~~~~ ~~~~~~ 

;

~~_c4 l

~~~

-

~~~~J~ 

~~~~~~~~~~~~~~~~~~

—

~~~~~~~~~ -1~~~~~~~~~ P(

Y

~~~~~~

~~~ 114 It ~~~~~~ \ \‘!

ç ( ~ ‘C

• %.:&—~

y (t

~~~ -~

( 

~~~~~~~ ~~~~ ~~~~~~~~ \‘:

~~

:

~ 1 ‘ - : . ..

\ . ~\ \,

~ \ / T.._,

~

:1. .— •‘. ‘\

~~~~~~~~~~~~~

...- ‘

•

•.

~ •

• ‘

~~P~

”

~~ ~~~~~~~ ~~1J1

~ ~~~~~~~~~~~~~~ - / f.1

~ 

~~~~~~~ I ‘

~~~~~~~~~~ t

~

IL ~-

~

J 7 ( ~~~~~ J 7

~~~~

‘

• 1.J~~ (( 1 ) ,c 

~~~~~~~~~~~~~ ç

~~~~~~~~~ r- - _ .- ,:,. ~ ~~

-

~~C~~~NV 

ii ‘

~~‘T~~~~~~~~~~~~~~ ~’ ~

L-?~k1(tiJ 1M~~~~~~~~~~~~~~~ ) 

‘

~~~~~~~~~~~~~~~~~~~~~~~~~ ,i~~~~~~~

•

•

~~~~~~~~ ) ~~~~J U~ ~~~~~ ‘

~~~1~ ~

ff /

~ .

~~ I .. .

. . \

~~ .‘

~.,

I PC ~~~~~~~ c ~~~~~~~~~~~~~~~ Ic 4 ‘

~~ c

~~~ ’

4s

~~~~~~ ?j’ \ U if/ ~~ ~ •.

ur.sso~ : - - - - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ J ~~~~~~~~~~~~~~~~~ \\ ‘—‘• “I -

~~~ “\ I V ~~~~~~~~~~~~~~~~ ~~~ ‘(‘ ~~~~~~~~~~~~ ~~ ‘‘ ‘

~ ‘ 

\___ ~~~~~~ 1

~ . 

~~~~~~~~ ‘~~

‘I) ,

~~1,

, 

~~~ 

7V 43 • 

, ‘

~, ,c

4
1

~

1 ~ , -

~~~~~~~~ •

•

~?~

:

~~~~~~~~~~~!k~~T ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .1 ~~~~ 

~~

‘L( ‘

~

- ‘1⁄4 “ I 

- ~~~~~ 

— ~~~ 

—‘

• ~~~-k-;

~ .

, 

~~~~~~~ 

S~~’

~

?N1

~ 

‘

~ - - ~~~Y”~~~)”~~~~~~’

~ N --—- \‘ ~

•

.

‘•.

• 

‘~~ • • ••—

~

it “

. S. ~~ AUrsssor: ‘

: ~~~~~~~~~~~~ / I(A.~L11(.

~~~~~~~ ~~~~~~~~~~~~~~~~~ 

— —

‘

~b.~• —

— ~~~ — - 

— — \

~~~~

• 

~~~~~ - — 

~~~

... . 

~ _•

~~~~ - .r

r 

~~~

JJ!I1

_ _
- ~

-., .••

APPENDIX D

MANEUVER W~

UTES F~~

)M TORT CA~~0N FORECASTING GAME E~~RCISES

• 

. --. .- • -. ~~~~-• • . I- ~~~~ •— -

1

~ — -

~ r — —~ .

-— ‘•-.

~~ 

— ~~~~~~ ~~. ,

3~

Z

~ .

.~~~~~~~~~~~~-

xT

~~ 

‘IS

~

.<\ !-&

~~~~~ 

1w V~~~~~~~\ ~~~~j

~~2_ ~

L

~fr~

) ~~~~ [

~~ O,

)Y ~~~

:

~ i~~~~~~ ? a - ~~ ~~~*kT • ~ ~~~~~~ • ~

)

~

*

~

ç ~~~~~~ ~ ~~

“ 

~

‘ ‘ ‘

~~~~~~

•

~~~

-

~~~

-

~~~~~~~~

~~ ,

i PC “

~

L

~~

’ •

~ /I

~ %~~ I I ~ f %\( .-~ ‘ ~~

~

j. 

‘

~

c ~~~~~~~~ 

I

, 

~~ \

‘

\

~~~ i~~ ) 

WN (J ~ ~~~

~~~~~~~ I

t;; .1 “

~ ~~

W\ %~~~~~~ ‘I

, 

~ ~y( 

~

-

~

< ,p ~~~~~~

- ~ —‘

~~ ~~~~~~~~~~~~~ .

.

~~ ~~~~ ____ t ~~C’ ..,. - 1I

j

.

~~ / i. .

;

. 

~ . 

— ~~ ~~~~~~~ ~~~~~~~~~~~~~~~~~ -

~~~ r’ ~~~— - ~~ ‘

~

~~ ~~~~~~~~~~~~~~~~~~~~~~~ 11 \&L •

- — .•— ~~~- •— ~~~~~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ . ~~~~~~~•
- ~~~~~~~~~~~~~~

I

p • • . • ..:. - ~-

. .- . .. •

~ • / —. 

~ ç - :- ~~~~- - :—- — ,. . ~ -

— -, - I ~~~‘

i 

~~~ ~~~~~~~~~~~~~~~~~ 

\ L c

~

—~

- ‘

~~ \ ‘

~-~~: 

~z~~~~~\ k ~~~~~~~~~~~~~~ ~~~ r

~~~~~~~~~~~~~~ itI~ - ‘

~~~~~~~~~~

1A 

~ ~~ 

;

. J ~~ 

~~ ~ ~~~~~~~~~~~~~~~~~ ~~~ - 

~

3 \Z’ \r 

~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

‘

~~

—~~ ? \r’ ~~~~~ -

~ ‘Ic’ ~ 

friw ~ ir ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ \ I 

~~ ‘

I 

~

—-X

~

\( “~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~ ti ~~~.) ,

-

~~

‘ ~~~~

\‘ .

~? . ‘~~~ _T ~~~~~~~~~~~~~~~~~~ t\ c :’ 1..& ;I

~~~~~~~~ ~~~~~ ~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ jf)Ct~~1⁄2\4~ ‘

?

~~ ~

JI c. . ~

, 

S ‘

~~ 

-

~ 

) ) ~ ~

\ 

~~ 

4

4 

- = _

~

3

~ ~

7;’

~

’\c -,

~~

. r;’ 

~ j ;;./ . 

.

~~~ ‘7 

V ~~~~~~~~~~~~~~ ~~~~~~

•

~~~~: 

~ °

~ 

o — 

,

~~

• .•• •

~~~~,> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- 

I

~~~~~~~ 

- 

• .

~

A AUrS SSOT: 

~~~~~~~~~~~~~~~~~

‘

. 

- , 

.

• US POrCS :..

— ~— —- •—--— - -— à ~~~~~~~~~~~~~~~ -- - - -
—~~

•

• -

~~~~

-.-- --— 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —

~~~~~~~~

--•-

~~

- 

~~~

-‘

~

- - — —

I H

F 

‘

~

f.• ~~~~~

,•..

r

” r—”

-

~ ..

~ 

~

J,

” • ç’

~

. -

I 

.

• - - . -

~ II . r ~

\ ~~~~~~~ ‘

~“ jj-

~~~

.

~t~

r .

. 

.

~~~~

I , 

2:-::E

~

) ~~ ‘ 

4 -

7 ~~~~~~~~~~~~~ m)~-... ~~~~~~~~~~~~

‘

~~~

‘ 

~~~~~~~~~~~

~ / ‘It, ~~~ ~ ~~~~~~~~~~ ~~~

~~~~~~~ 

I :

~~

ic\J ~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~ ,

‘

, 

~~~~~~~~~~~~~~~~~~~~~~ US Fore.: — -

~~~~ p ~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .

i

~~~~~~~~~~~~~~~ 4~~~~~.
—I - 

• -.--

~-~~~~~~~~~~~~~~~~~ ••

•

~~~- 

‘

~~

-

~~~~~

- —.— • —--

N ~~ ‘ m U) ‘0 ea in

I N ‘14 .14 N

• U) .14 ~

D N •-I in 0 ~ ‘ N

Z N

I frI

-

11 • b

~ El

I El N i—I

~~ I I

4) ..,

J 0~~~~~~~~~ 4

i—I i—I N 0 0 0 0 N

~~ Ø’I in it) ~

D 0 r14

I) _

- - (‘I N .14 -4 N 0

--- -

~~~~

; -

U) E4/ ~~~ r

~ U)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

H

0

- øs U)

1! 

14

Fl P4

U) ~~

U ~4 ~~ ‘ N ‘0

L. ri ‘14

)4 1)

01 m ~~~ in ,—4 ~4 ~ 4 ‘—4 It)

I:. Fl 

~~ Fl 0 Fl .14 ‘4) P4 N 0

El N N Fl Fl ,4 U)

in t

~ N 4) ~4

~ 

I;- 

I ~~~~~~

- 

• ~~~~~~~~~~~~~~~ . •~~~ ~~~~~ 

I
-

- ~~~~~~~~

“- ‘

~r~

.__ • — ~~ • —

N .—4 .-4 in N

I I- • -iJ F’ 14• 4) - U ..4

-

~~ 

14 4) — — — — —

O mNN N

: 1.1

F’

I

U) 

—

in 

— — — 

in N 

— —‘0 U) N N H Fl

41

I

‘0 ‘0 0s Fl ‘0 tn

U) 0 N U) 1’ N El Fl N N r4 U)

N ‘0 In U)

~~~~~~~~~~~~~~~~

iidd -

~~~~~~~

-

~~~~

_

~~~~~~~~~~~~

-

~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ _____

•

~~~~

• . .
- - ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ •

~~f••,~~~~~~~~~~~ -’m~~ u.”-- - - --~~~~~~~~~~~~~~~~ —~~~~~~--——--— ,~~~~~—

•1

- - - 

- 

-

DISTRIBUTION

ARt Distribution List

4 0450 (M&RA) 2 HOUSACDEC, Pt Ord, ATTN: Library

2 HODA IDAMI-CSZ) 1 HOUSACDEC. Ft Ord. ATTN: ATEC-EX—E -Hum Factors - 

- I HODA (DAPE-PBRI 2 USAEEC. Ft Benjamin Hurison, ATTh: Library

1 HOOA IDAMA -AR I 1 USAPACDC, Ft Benjamin Harrison, ATTN: ATCP-HR

1 HODA IDAPE-HRE-P0) 1 USA Comm—Elect Sd., Ft Monmouth, ATTN: ATSN-EA

1 HODA (SGRD-ID) 1 USAEC, Ft Monmouth ,ATTh: AMSEL- CT-- HDP

1 HODA IDAMI-DOT-C) 1 USAEC, Ft Nonmouth . ATTN: AMSEL—PA- P

1 HODA (DAPC-PMZ-A) I USAEC, Ft Monmouth . ATTN: AMSEL—Sl—CB

1 HODA (DACH-P?Z-A) 1 USAE C, Ft Monmouth, ATTN: C, Fec) 0ev Br ‘

~

; 

- 

- 1 HQDA (DAPE-HRE) I USA Materi als Syt Anal Agcy. Aberdeen , ATTN : AMX SY—P

• 1 HODA (DAPE-MPO-C) I Edgewood Arsenal . Aberdeen. ATTN: SAREA---BL --H

1 HODA (DAPE-DW) I USA Ord Ct, & Sd., Aberdeen, ATTh: ATSL—TEM-C

1 HODA (DAPE-HAL) 2 USA Hum En~ Lab, Aberdeen, ATTN: Librvy~0ir

I HODA (DAPE-CPS) I USA Combat Arms Tng Sd. Ft Benn.ng. ATTN: Ad Supervisor

I HODA (OAFD -MFA) 1 USA Infant ry Hum Rich Unit . Ft Banning, ATTN: Chief

I I-b OA (DARO-ARS-P) 1 USA Infantry Sd, Ft Banning, ATTN : STEBC-TE-T

1 HODA (DAPC-PAS-A) I USASMA. Ft Bliss . A’rTN:

I HODA IDUSA-OR) 1 USA Air Del Sd., Ft Blit , ATTN: ATSA CTD - ME

I IIODA IDAMO-ROR) I USA Air Del Sd. Ft Bliss , ATTN : Tech Lib

1 HODA IDA SGI I USA Air Del Sd. Ft Bliss . ATTN: FILES

1 HC)DA (DA1O-PI) 1 USA Air Def Sd. Ft Bliss , ATrN: STEBD—P0

1 Chief . Consult Div (DA-OTSG). Adeiphi , MO I USA Cmd & General Stf College, Ft Leavenworth, ATTN: Lib

1 Mil Asst. Hum Ret , ODDR&E . OAD (E&LS) 1 USA Cmd & General SIt College, Ft Leavenworth, ATTN: ATSW—SE—L

1 HO USARAL . APO Seattle . Afl N: ARAGP-R 1 USA Cmd & Gsneral Sit college , Ft Leavenworth, ATTN: Ed Advisor

1 HO first Army, ATTh: AFKA.Ol-TI I LISA Combined Arms Cmbt 0ev Act , Ft Leavenworth, A1TN: OepCdr

2 HO Fifth Army. Ft Sam Houston I USA Combin ed Arms Cmbt Div Art. Ft Leavenwomfi. AITh: CCS

1 Dir , Army Stf Studies Ole, ATIN: OAVCSA (DSP) 1 USA Combined Anne Cn*t Div Act. Ft Leecen aurth. ATTN: ATCASA

1 Ole Chief of Sit , Studies Ofc 1 USA Combined Arms Cmbt 0ev Act . Ft Lesvenwo~

th, AflN: ATCACO—E

I (ICSI’C-R, ATTN : CPS/OCP I USA Combined Arms CiTib, Div Act . Ft Lesvenwørth , ATTN ATCACC—CI

1 The Army Lib , Pentagon . ATTN: RSB Chief 1 USAECOM. Night Vision Lab, Ft Belvoir , ATTN: AMSEL—NV—SD

I The Army Lib . Pentagon. ATTN : ANRAI. 3 USA Comput.r Sys Cmd, Ft Belvoir, ATTN: Tacit Library

1 Ofc, Asit Sect of the Army (R&D) 1 USAMERDC. Ft Belvoir , ‘

,TTN: STSPB—0O

1 TeCh Support otc. ojcs 1 USA Eng Sd., Ft Belvoir, ATTN: Library

1 USASA. Arlington, ATTN: lARD-I 1 USA Topographic Lab, Ft Belvoir, ATTN: ETL TO—S

1 USA Rsch Ofc , Durham, ATTN: Life Sciences Dir 1 USA Topographic Lab, Ft Belvoir. AIM : STINFO Canter

2 LJSARI EM , Nalick . An N: SGRD-UE-CA 1 USA Topographic Lab, Ft Belvoir, ATTN : ETL -GSI.

- -~ I USA! rc, I I Clay t on, Al IN: ~If IC MO ~ I USA l.ih liyrnr.’ CIr & Sd., Ft H,s,,chucj, ATTPI: CT)) MS

I USAIMA , Ft Bragg. ATrN: ATSU-CTD-OM 1 USA Intelligence Ctr & Sd., Ft Huichuca. ATIM: ATS—CTD—MS

I USAIMA, Ft Bragg, ATTN: Marquat Lib I USA Intell igence Ctr & Sc).. Ft Huachuca, ATTN: ATSI—TE

1 US WAC Ctr & Sch, Ft McClellan , ATTN: Lib I USA Intelligence Ctr & Sc)., Ft Huichuca. AIIM: ATSI—TEX --GS

I US WA C Ctr & Sch, Ft McClellan, ATTN: Tnq Dir I USA Intelligence Ctr & Sc)., Ft Huachuca. AIIM: ATSI—CIS—OR —

I USA Quartermaster Sch, Ft Lee. ATTN: ATSM-TE I USA Intelligence Ctr & Sd., Ft Huadiuca, ATIM: ATSI-CTD--QT

1 Intelligence Material Des Ofc , EWL, Ft Hol.bi rd 1 USA Intelligence Ctr & Sch, Ft Huachuca. ATIN: AT5l—CTD--CS

1 liSA SE Signal Sc)., Ft Gordon, ATTN: ATSO-EA I USA Intelligence Ci, & Sc)., Ft Huechuc., AIIM: DAS/SRD

I USA Chiplaii, CIr & Sc)., F. Hamilton , ATTh: AT$C-TE-RD 1 USA Intellig ence Ctr & Sd., Ft Huechuca, ATTN: ATSI-TEM

I USATSCH, Ft Eustis, AIIM: Educ Advisor 1 USA IntellIgence Ctr & Sd., Ft Huachuca, ATTN: Library

1 USA War Colleqe, Carlitla Barracks , AIIM: LIb 1 CDR. HO Ft Huichuc., ATTN: Tech Ret Div

2 WRAI R, Netsr optyth iat,y Div 2 COR, USA Electronic Prvg Grd, ATIM: STEEP--MT—S

1 DLI. WA. Mnnlerey 1 HO, TCATA. ATTN: Ted. Library

1 USA Conce pt Anal fr~cy, Bethesda, AIIM: MOCA-MR I HO, TCA~

’A, ATIN: AT CAT OP O. Ft Hood — 1 USA Concept Anal Agcy, Bethesda. AIIM: MOCA.JF I USA Nectufting Cmd. Ft SherIdan. ATIM: USARCPM-P

I USA Arctic Test Ct,, APO Seattle, ATIN: STEAC-PL.Ml 1 SenIor Army Mv., USAFAGODIT*~ . Elgin AF Asia FM NoB

1 USA Arctic Test Ct

~

, APO Seattle, ATIN: AMSTEPL.TS I HO, USARPAC. OCSPER, APO SF 59555. ATTN GPPESE

1 USA Armament Cmd, Redstone Mend , ATIN: ATSK.TEM 1 Stlmson Lb. Arademy of Health Sciences, Ft Sam Houston

I USA Armament Cmd, Rode Island, ATTN: AMSAR•TOC 5 MarIne Corps Inst., ATTN: Dean-MCI

I FAA.NAFEC. Atlantic City, AIIM: LIbrary 1 HO. USMC. Commandant AIIM: Code MTMT

I FAA-NAFEC . Atlantic Cliy. ATTN: Human Engr Br S HO, USMC, Calniflandint AIIM: Cods MPI-2O~

2S

1 FAA Aeronautical Ct,. Oklabcm. City, ATYN: AAC.440 2 USCG Academy , New London, ATm: Admission

2 USA FM Arty Sd., Ft Sill, AIiM: LIbrary 2 USCO Academy. New London. ATIM: Library

I USA Armor Sc)., Ft Knox. ATTN: Library I USCO Training Ct,, NY , ATT’N: CO

I USA Armor Sd. Ft Knox, AIIM: ATSS.DI-E 1 USCG Training CU, NY. AIIM: (due Sec Ofc

I USA Armor ScSi. Ft Knox, AIIM: ATIS-DT- TP I USCO, Psychd Ret It, OC, AIIM: OP 1

~

S2

1 USA Armor ScSi, Ft Knox. ATIN: ATS$-CDAD I HO Nld-Rt;t9i B~, MC Det, Deentico, AIIM: PU Die

j

~

_

~

_j 

~~~~~ -___~

- —— • - - -• --• - - —-- -- —----~-p---—---—-- --•-- - —----•- _ _ _s

.~~ I&__ ____ — -------

~~•-—-~ -.- - -
- ~~:-

~~~~~~~

____________ - ~ - - -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ — -

~

-

~ -

I

I US Marine Corps Liaison Ole, SIMC, Alexandria. AIIM: AMCGS-F 1 Del & Civil Inst of Enviro Medicine, Canada

I USATRADOC. Ft Monroe. ATIN: ATRO-ED I AIR CRESS, Kensington , AITM: Info Sys Br

6 USAIRADOC, Ft Monroe , ATTN: ATPR--AD I Militaerpsytc&ogiik Tjeneete, Copenhagen

1 USATRADOC. Ft Monroe , ATTN: ATTS-EA 1 Milit ary Attache. French Embassy. ATTN: Dcc Sec

1 USA Forces Cmd, Ft McPherson. AITN: Library I Medecin Chef, C.E.R.PA.—Araanal. Toulon/Naval Fiance

2 USA Aviaflon Test Sd, Ft Rucker, ATTN: SIEBG—PO S Prin Scientific Off . Appi Hum Engr Reds Div. Ministry

1 USA Agcy for Aviation Safety , Ft Rucker. ATTN: Library of D.fenae, New Delhi

-1 USA Agey for Aviation Safety, Ft Rucker, AIIM: Educ ~~~~~ I Per; Rich Ole Library, AKA. Israel Defense Forces

1 USA Avljti on Sch, Ft Rucker , AIIM: PD Drawer 0 1 Ninlstiris vie. Defansle, 000P/KL Afd Sociaal 4

1 HOUSA Aviation Sys Cmd, St Louis , ATTN: AMSAV—ZDR Psychologische Zakan. The Plague, Netherlands

2 USA Aviel,o,i Sys Test Act., Edwaids AFB. AIIM: SAVTE—T

1 USA Air Del Sd., Ft Slits, AflN: ATSA TEM

I USA Air Motility Rsch & Dcv Lab. Moffett Fkt, ATTN: SAVOL -AS

1 USA Aviation Sd., Ret Tng Mgt. Ft Rucker, AIIM: ATST—I—RTM

1 USA Aviation Sc)., CO. Ft Rucker, ATTN: ATST-D-A

1 HO, DARCOM. Alexandria. ATIN: AMXCD—TL

I HO, DARCOM . Alexandria, ATTN: CDR

1 US Military Academy, West Point, ATTN: Serial, Unit

1 US Military Academy, West Point, ATTN: Ole of MiIt Ldrthp

I US Military Academy. West Point, AIIM: NAOR

1 USA Standardization Op. UK, FPO NY, AIIM: NASE-GC — -

1 Ole of Naval Rids, Arlington, AIIM: Code 452

3 Ofc of Naval Rich, Arlington, ATIN: Code 468

1 Ole of Naval Rids, Arlington. AIIM: Code 450

I Ole of Naval Rids, Arlington, ATTN: Code 441

1 Naval Aerotpc Med Rae Lab, Pensacola, Afl’N: Acous Sc). Div

1 Naval Aerospc Med Re, Lab. Pensacola, ATTN: Code L5I

1 Nsval Acroapc Med Ret Lab, Pensacola. ATI’N: Code LB

1 Chief of NasPers, ATTN: Pert-OR

1 NAVAIRSTA, Norfolk. AIIM: Safely CIt

1 Nay Oceanographic. DC, AIIM: Code 6251, Charts & Tech

I Center of Naval Anal, AIIM: Doe Ci,

I NavAirSysCom, AIIM: AIR—6313C

1 Nay BuMed, AIIM: 713

1 NavH.llcoplacSubSque 2, FPO SF 96601

- - I AFHRL (FT) Williams AFB

I AFNRL (Tfl Losts -y AFB

1 AFHRL (AS) WPAFB. OH

2 AFHRL IDOJZ) Brooks AFB

1 AFHRL (DOJN) Lackland AFB

1 HOUSAF IINYSO)

I HOUSAF IDPXXA)

1 AFVIG (A D) Randolph AFB

3 ANAL (MEl WPAFB, OH

2 AF Inst of Tech, WPAFB, OH. AIIM: ENE/SL

1 AIC (XPTDI Randolph AFB

1 USAF Aerotaed Li)., Brooks AFB (SUL—4), AIIM: DOC SEC

- : 1 AFOSR (NI). Arlington

1 AF Lag Cmd, McClellan AFB, AIIM: ALC/DPCRB

1 Air Force Academy, CO. AIIM: Dept of Bel Sen

5 NavPers & Oev Ctr, San Diego

2 Navy Med Neuropsych iatric Rich Unit, San Diego

I Mae Electronic Lab, San Diego , A11’N: Ret Lab

1 Nay TrngCen. San Diego. AIIN: Code 9000’— Lib

1 NavPcstGraScts. Montetey, ATTM: Code B6Ae

I NevPostGraScf.. Monterey, AIIM: Code 2124

I NavlrngEqulpCtr, Orlando, AT’TN: Tech Lib

I US Dept of Labor. DC. ATIM: Manpower Mmin

I US Dept of Justice, DC, ATTN: Drug Enforce Admit.

I Nat Bur of Standards, DC, AIIM: Computer Info Section

I Nat Clearing Maui, for NH-Info, Redsvllle

1 Denver Federal Ct~. Lilcewood. AIIM: BLM

52 Defense Documentation Center

4 DIr Psych, Army Hq, RussO Ole., Canberra

1 Scientific Adv,r, Nil Id, Anny Nq, Russell Ofcs, Canberra

I Nil and Air Attache, Auitrlan Embassy -

I Centre di Redserctie Des Facteurs, Humeisse de li Detente

National,. Brussels

2 Canadian Joint Staff Washington

1 C/AIr Staff, Royal Canadian AF. AIIM: Pert fed Anal I,

3 ChIef, Canadian Del Rich Staff, ATm: C/CPOS(W) 

—

4 BritIsh Del Staff , BrItish Embassy. WashIngton

I -_- _ ~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -~~~~~-- - - 

-