/ I

~~~~~

,

Technical Paper 349 , [

_

AD

SELECTIVE FEEDBACK AS A TRAINING AID TO

ON-LINE TACTICAL DATA INPUTTING

Paul A. Gad., Alleon F. Ft aide, and Irving N. Alderman

QcM~~ HUMAN FACTORSTECHNICAL AREA 1Q ~~~~~

~

ir

Research %nstitute tor the 

U. S. 

Bt

Army

’haviora

~

l and So~ial Sciences

Nov.mb.r 1978

~~~~~ Aøprov, f public r.l.ss s ;~~ist rlbu$â QsI unlI mlt,d.

_____ 

a j~ 

6~~U
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WILL IAM L. HAUSER

JOSEPH ZEIDNER Colonel , US Army

Technical Director Commander

NOTICES

DISTRISUTION: Prim ry distribution of this report has been made by ARI. Plus. sddrus correspondsnc.

concerning distribution of reports to: U. S. Army Rei.erch Inititute for the Ssh.v i or.l and Social Sciincss. ATTN: PERI-P, 5001 Eiwnhowsr Avenue, Ali

~andria, Virginia 22333,

FINAL 04~~O1ITloN: Thu report ipoy be destroyed whe n it is no longs, needed. Plea, do riot return it to

the U. S. Army Rsissrch Institute for the Isheviorel and Social Science,.

~~~~~ The findings in this report ire not to be construed as an official Deportment of the Army position,

unites so des ignated by other auth orized documents.
- —.-.- ~~

.- .—- -. .r.- ~

-. .

--.

Unc .Lass lt lea

SECURITY CLASSIFICATION OF THIS PAGE (lfls.,s Os. Eni .,.

~~

~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ READ INSTRUCTIONS ,

~~~r ijI~ U t#’J~..,UM~~ fl I ~~ U U’.’I~ r ~~~~ BEFORE COMPLETIJ4G FORM

.fISEPoRT MuMBt~ ____ ..... ~_ ~~~~~~~~~ 2. GOVT ACCESSION NO 3. RECIPiENT’S CATALOG NUMBER

(

~ ~~

Techflj C~~ _______________________________

I!. TITLE (li

~~LbAH.) —p-’ 

~~~~~~~~~~ 5. ‘PYPE OF REPORT & PERIOD COVERED

ii ‘

~~~E~~~~~V~IjEEDBACK AS A ~~AINING~~ID ~

o / -- (

~ J~~l-LINE TA~TICAL DATA INPUTTING, -~~ 6. PERFORMING ORG. REPORT NUMBER

~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

.. ~~~~~~~~~ . S. CONTRACT OR GRANT NUMB~~

R(.)

~

Pau1 A~jGadei Alison F.~ Fie1dstf vin~ N.

\.

~~d JAlderm I _______

L PERFORMING ORGANIZATION NAM E AND ADDRESS *0. PROGRAM ELEMENT. PROJECT . TASK REA & WORK UNIT NUMBERS

U.S. Army Research Institute for the Behavioral —-.-~~ - -

and Social Sciences Q763743A774

5001 Eisenhower Avenue , Alexandria, VA 22333 -

II. CON1~

ROLLlNG OFFICE NAME AND ADDRESS ~~,

. t

~~ RgpoaT~

DA4._.—.- .-.

U.S. Army Combined Army Combat Development / )November~-1978

Activity (CACDA) ‘

~~~~ . 4~

-.iu~ e.sIier r

~ oca --~~ Fort Leavenworth, KS 30

14. MONiTORING AGENCY NAME & ADDRESS(U dlff•omI frcm ControllI ng Of (Sc.) IS. SECURITY CLASS. (.1 SkI. r.porf)

Unclassified

—— I_S.

. DECLASSIFICATION/DOWNGRAOING

SCHEDULE

15. DISTRIBUTION STATEMENT (of Ski. R.port)

Approved for public release; distribution unlimited.

~~~~~ 

(

~~T~

-TT

~j *7. 01ST -IO*--S-rw~

I tMv.NT?Tth~ .b.Ir.ci .ntet.d In Block 20. If dlff•r.ne from R.porf)

~

IS. SUPPLEMENTARY NOTES

tC. KEY WORDS (ConIhi u. on rev.,.. .Id. If n.c.. •.ry aid Id.ntIfy by block number)

Military information systems Training

Man-computer interaction Adaptive training

Input procedures Response-sensitive strategies

Computer—assisted message input

50 ‘

~ ABSTRACT (Caiffiiu. a, r.vie,e *I

~~ If n

~co~e y aid IdentIfy by block nomb.r)

Training as Message Input Output Device (MIOD) operators was given to

53 Army enlisted personnel under one of four training methods that were based

on different instructional strategies. Performance of each participant was

assessed during training and under one of two operational configurations of

the Tactical Operations System (TOS).

Results showed that the response—sensitive training strategy was effective in reducing training time wi1

~

hout loss of performance accuracy during~~

~ 

21- O (contin~ed) ~~~.. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (

~~~~~~~~~~~~~ SSl~~~~~~~~
Unciassttied

~~

.Ie ,

~~~*Ty CLASSIFICATION OF THIS PAOE(Wl .ni Dci. Ento r.d)

20. Wontinued)

transfer—testing . None of the training methods significantly reduced the

~rnount of undetected errors that enter the system through manual data ~~~~~~

0~

ACCfSSION for

NT IS W~ 

j :~ ~‘:

~on L

ooc e.ii s

~ -~~ o ~NAfIP ’OfiNC D 0

BY

O$TRI~

JT~ /AYA~c.::r ‘

~

r$

~ 2 ~~~~~ __.__ - . ~~~~~

Ti

iii t

111
- 

~

- _— .-, —.:~- .-. . ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .... .

~~~~ ~~W - —

Technical Paper 349

SELECTIVE FEEDBACK AS A TRAINING AID TO

ON-LINE TACTICAL DATA INPUTTING

Paul A. Gade, Alison F. Fields, and Irving N. Alderman

HUMAN FACTORS TECHNICAL AREA

Submitted as comp lete and Approved By:

technically accurate, by:

Edgar M. Johnson A.H. Blrnbaum, Acting DIrector

Technicil Area Chiet ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

Joseph Zeidner

TECHNICAL DIR ECTOR

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

Nov.mb.r 1978

Army Project Number Command/ControP

20753743A?74
~

: ~

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

_____- -

~~~ —_ ~~- ---- ~~ ---~~ ~ --- _~~~~~~ _ _ -~~~~ _~~--- _ J—

_

~ -~~~

_ ._~~~_~ ~~~~~~~ ~~~~~~~~~ -_ —- ——----—-~~ _ i_ _ _ _ -- - - _ - - _ ----_.— —
FOREWORD

The Human Factors Technical Area is concerned with the demands of

increasingly complex battlefield systems that are used to acquire , transmit , process , disseminate, and utilize information . This increased complexity places greater demands upon the operator interacting with the

machine system. Research in this area focuses on human performance

problems related to interactions within command and control centers as

well as issues of system development . It is concerned with such areas

as sof tware development, topographic products and procedures , tactical

symbology, user—oriented systen~ , information management , staff operations and procedures , decision support , and sensor systems integra tion

and utilization .

An issue of special concern within the area of user—oriented systems has been the improvement of manual data input procedures , especially

in the Tactical Operations System (TOS). The main source of information

for tactical data systems is manual data entry—-a slow , error—prone process. The capability of tactical data systems such as TOS to support

command staff actions with accurate , complete , and timely informa tion

depends on the performance of the human operator who must manually enter

information into the system. Previous ARI research on data entry has

resulted in simplified message forma ts , improved reference codes , and

aids for on-line preparation and verification of message entries. Although progress has been made , data entry remains a major system bottleneck . The research reported here demonstrated the effectiveness of a

response-sensitive training strategy in reducing the amount of time required for operator training .

Research in the area of user—oriented systems is conducted as an

in—house effort augmented through contracts. The current report resulted from an in—house research effort initiated under the program direction of Mr. James U. Baker and responsive to requirements of Army

Project 2Q763743A774 and to special requirements of the U.S. Army Combined Army Combat Development Activity , For t Leavenwor th , Kans. Special

requirements are contained in Human Resource Needs 75—150 , “Online Aids

for Tactical Data Inputting .”

OS PH Z R

e nical Director
~ — —------.-

SELECTIVE FEEDBACK AS A TRAINING AID TO ON-LINE TACTICAL

DATA INPUTTING

BRIEF

Requireme n t:

To evaluate alternative on-line computer—assisted training strategies for improving performance of the Message Input Output Device (MIOD)

operator in the Tactical Operations System (TOS) .

Procedure :

MIOD training was given to 53 Army enlisted personnel under one of

four training methods: minimum feedback , edit feedback , remedial feedback , and response-sensitive feedback . The first three methods were 2

based on the traditional linear instructional strategy model , and the

fourth method was based on a response-sensitive model of instruction .

The transfer of this training was tested under two different operational

configurations of TOS : no feedback and edit feedback .

Findings:

Use of a response—sensitive instructional strategy markedly reduced training time . No differences in data entry speed or accuracy

were found among the four training conditions , or for participants

tested in each of the two operational configurations. Mean entry accuracy was more than 80%. Most of the errors (more than 90%) were types

that could not be detected by computer edit routines.

Utilization of Findings :

A response—sensitive training strategy is an effective method for

reducing MIOD training time without sacrificing inputting accuracy .

The decision to adopt a response-sensitive training strategy for specific training should consider the trade—off between the added cost of

training development and the cost savings of reduced training time .

Development of more elaborate edit routines may help reduce errors

bu t probabl y will no t elimi nate them . Improved forma ts and defi nition

of legal entries may provide an effective , simple method to reduce input errors.

________ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SELECTIVE FEEDBACK AS A TRAINING AID TO ON-LINE TACTICAL

DATA INPUTTING

CONTENTS

Page

INTRODUCTION 1

METHOD 3

Participants 3

Apparatus 3

Procedure 3

Experimental Design 6

RESULTS AND DISCUSSION 7

Entry Time 8

Entry Accuracy 11

SUMMARY AND CONCLUSIONS 18

REFERENCES 19

APPENDIX . DESCRIPTION OF DATA ENTRY FORMATS 21

DISTRIBUTION 29

LIST OF TABLES

Table 1. Sununary of error feedback features 5

2. Experimental design 6

3. Analysis of covariance for mean time per entry 9

4. Mean item entry time in seconds for type of message

and mode of training 10

5. Analysis of covariance for first entry accuracy . . . . 12

L 

6. Mean percentage accuracy scores for training and

transfer—testing sessions as a function of

training and transfer—testing feedback modes 13
CONTENTS (continued)

Page

Table 7. Mean percentage accuracy scores for type of

message and mode of training 15

8. Mean percentage testing accuracy scores for

type of message and type of transfer—testing

feedback 16

9. Analysis of covariance for first versus last

attempt accuracy 17
SELECTIVE FEEDBACK AS A TRAINING AID TO ON-LINE

TACTICAL DATA INPUTTING

INTRODUCT ION

The primary function of the Army ’s evolving semiau tomated Tactical

Operations System (TOS) is to provide commanders and their staffs with

current , accurate battlefield information on which to base tactical decisions. The effectiveness of TOS in providing timely and accurate

information is heavily dependent on the performance of the Message

Input Output Device (MIOD ) opera tor who mus t man ually enter tactical

information into the system . Dur ing the evolution of TOS, it became

apparent that data inpu tting was perhaps the most crucial componen t of

the system. Delays or errors in data entry .

~an have a devastating impact on the quality , completeness , and timeliness of the subsequent

processing of information by the system. Data entries also need to be

relatively error—free , since more than 80% of operator errors probably

will enter the system undetected (Strub, 1975).

Previous research (reviewed by Alderman , 1975) focused on improving the data inputting performance (reducing time and errors) of

MIOD operators by reallocating the inputting functions/tasks and development of job/training aids . However , if data entry time as well as

the number of data entry errors are to be reduced further , MIOD operators must be trained to become more proficient.

The problem of training a human operator for effective on—line

data input to the TOS is threefold : initial training of the MIOD operator , optimum transfer of new skills to operational settings , and

main tenance of the high levels of proficiency developed during the

first two phases. One approach to training MIOD operators is to develop training modules that can be embedded within the TOS software.

Modules could be used to train operators and to maintain trained operators at a high level of proficiency . The effectiveness of an embedded training module is highly dependent upon development of optimizing training strategies. Such optimizing strategies should be

arranged to accommodate individual differences among learners (e.g.,

educational history , curren t skill level ) as well as quantitative and

qualitative differences in the material to be learned (Atkinson & Paulson , 1972).

A response-sensitive instructional strategy provides the basis

for an instructional system that accommodates differences in individuals as well as differences in the material to be learned (Atkinson ,

1972). Briefly , this. strategy uses a trial—b y— trial response history

to assess the student’s current state of learning so that further instructiona l items can be selected to optimize learning . To illustrate

how this stra tegy works , assume you wish to train potential MIOD

1

hh- , . —~~~~ -

~~~~~~~~~~~~~ - - -- -~~~~~~~~ - - --- -~~~~~-- --—--- - - -~~~~~~~~

—

~~~~~~~~~~~~~~~
operators to use the specific message input format Ed (see the Appendix for format listings) . During the initial stages of training ,

you would begin to select items for further training based on the operator ’s performance history. For examp le , suppose af ter giving three

Ed format messages you observed that a particular student never made

a mistake on the sections labeled AGENCY and SOURCE , but performed inconsistently on other sections of the format. Under a traditional or

linear strategy you would continue to give this student practice on

all items . Under a response-sensitive strategy you would reduce the

frequency of student practice (possibly to zero) on the sections 1abele~ AGENCY and SOURCE and concentrate the training on those items where

performance has been inconsistent.

Strub (1971) has shown that inputting accuracy can be increased

when MIOD operators use a cathode-ray tube (CRT) to input data directly

into the TOS computer system. He also found that entry times could be

decreased by providing a computer-generated format on the CRT as an

aid to data entry (Strub, 1975). Strub (1975) suggested that an automated training program incorporating these inputting aids with a

response—sensitive instructional strategy might be used to train MIOD

operators more effectively. The present experiment was designed to

investigate the relative effectiveness of following Strub ’s suggestion

of using these inputting aids within the framework of a responsesensitive instructional strategy to train MIOD operators.

It was hypothesized that use of a response-sensitive instructional

strategy in conjunction with computer-generated feedback would have two

desirable effects on MIOD operator training . First , it was expected

that such a strategy would allow subjects to progress through the training material at a faster rate than would other more conventional strategies. Second , it was thought that a response—sensitive strategy would

produce better transfer of training to operational environments thai

~ would other training strategies. This enhancement of transfer of training was expected to result in greater input accuracy scores and/or faster

entry times during transfer—testing .

The objective of the research , then , was to assess the impact of

response—sensitive training on the speed and accuracy of tactical data

inputtiny by MIOD operators in a TOS environment.

L 2_ ~~~~~~~~ -~~~~~~~ -- .

- .- . - -
METHOD

Participants

The participants were 71 enlisted personnel who volunteered to

serve for 7 to 8 hours a day for 2 consecutive days. To approximate

the capabilities necessary for assignment as an MIOD operator , par—

ticipants were requested who had General Technical (GT) scores of 105

or higher and who had normal vision or vision correctable to 20/20.1

Apparatus

The experiment was conducted on-line in four separate work stations. Each station contained an IBM 1050 printer/keyboard terminal

and a CDC 210 CRT display and associated keyboard entry panel.2 The

1050 terminals printed out free—text messages that simulated battlefield information . Messages consisted of two general types , those

concerned with enemy activities and those concerned with the activities

of friendly forces. The CRT’s displayed the formats into which participants entered information translated from the free—text messages.

All information about enemy activities used the same format. Friendly

activities , however , required that the participant choose one of nine

other formats based on the subject matter of the free—text message .

With slight modifications , the messages and the formats used in the

present experiment were the same as those used by Strub (1975). Copies

of the 10 formats (9 friendly , 1 enemy) appear in the Appendix along

with a list of the order in which they appeared in the experiment.

Participants were given a notebook containing a detailed set of instruction s that outlined , step by step , each task they were to learn .

In addition , participants were given another notebook tha t contained

an entry—by-entry description of each of the 10 message formats and a

list of legal entries for each of the format entry items .

Procedure

Each par ticipant took part in a training session and a transfer—

testing session , which were conducted on 2 consecutive days . After

receiving a brief description of the experiment , par ticipants were

randoml y assigned to one of the fou r work stations and given notebooks

that contained detailed description s of their tasks , legal entry tables,

and a scenario that described the simulated combat environment in which

they would be operating.

‘

No at tempt was made to ver ify GT scores wi th participant records.

2

Commercial names are used for purposes of clarity only and do not indica te endorsemen t by the Department of the Army or ARI.

- L - --.--~~~-- - ~~~~~~~~~ -~~~ ----- ~~ ±_~~_

~

_

~~ --- —~~-- -~~~~~~~ .- - -- - -- - - - “-
Participants were given a brief keyboard orientation that included

a practice drill using the various keyboard functions. This was followed by a 12-item typing exercise to familiarize them with the typewriter output and CRT keyboard inputting procedures. After the participants had completed both the keyboard dr ill and typing exercise and had

read the detailed instructions for the exper imen t , the experimenter

demonstrated the method to be used for inputting free-text messages

into the computer. The demonstration consisted of a step-by-step message analysis and a detailed entry procedure for a sample friendly message. The procedure was demonstrated to each group of four participants

on a large piece of posterboard tha t had the appropria te forma t inscribed

on it. Participants returned to their work stations to enter the sample

message . If participants tailed to identity the correct entry, it was

supplied by the experimenter witn an explanation . Entries might be correct, incorrect, or illegal (not a member of the appropriate legal entry

table).

Participants were required to complete 21 messages during training :

15 training messages on the first day and 6 on the second day . Each

participant received one of the following four types of training feedback :

1. Minimum Feedback. Participants received messages on the lcMer

part of the CRT whenever illegal or incorrect information was

entered. The error message informed them that their last entry

was in error and that they should continue to the next entry .

Par ticipants could not go back to correct their mistakes .

2. Edit Feedback. An error me3sage was provided when an illegal

entry was made . The participant was instructed to refer to

the appropriate legal entry tables and to reenter the information before proceeding to the next entry item. Incorrect

entries in legal format produced no feedback .

3. Remedial Feedback. If an entry was incorrect , an error message containing both the incor rect and the correct entry was

displayed at th bottom of the CRT. A correct entry was required before the participant could proceed to the next entry .

4. Response—Sensitive Feedback. This condition was identical to

the Remedial Feedback condi tion wi th the following exception :

After the participant had entered information correctly into

a par ticular elemen t of a par ticular type of format three

consecutive times , the compu ter would fill in the corr ect

answer fo r the par ticipan t thereaf ter when tha t element of

the format was encountered (auto-fill). The figure of three

consecutive correct entries was chosen as the criterion for

receiv ing auto—fill based on the performance of 20 pilot subjects. This figure represents the median number (rounded to

the highest whole number) of consecutive errors made by the

pilot subjects on all common en try items occurring on the f i rst

4
21 messages. Because none of the friendly formats occurred

more than three times during the training session (Appendix),

auto-fill was possible only for enemy messages.

After completing the training , participants were briefed about the

type of feedback they would receive during transfer-testing. Each participant received one of two types of feedback during transfer—testing :

1. No Feedback. Under this condition the participant was never

given information about the correctness or legality of his

entries. All entries were accepted by the computer.

2. Edit Feedback. This condition was exactly the same as that

for Edit Feedback training.

Participants then completed as many messages ~s possible, up to the

maximum of 43, in the testing session which lasted the rest of Day 2.

A summary of the error feedback features for each of the feedback

conditions is given in Table 1.

Table 1

Summary of Error Feedback Features

Feedback Error Correction Correct Autocondition report required entry given fill

No feedback - - - -

Minimum + - - -

Edit ~a 

+ - -

Remedial + + + -

Response—

sensitive + + + +

a

Errors in Edit Feedback are detected by computer edit and validate

routines only.

S
Experimental Design

The experimental design , shown in Table 2, was a 2 x 2 factorial

with two appended control groups, yielding a total of six independent

gioups. Each of the 71 participants was randomly assigned to one of

these groups.

Table 2

Experimental Design

Transfer-Testing Mode

Training Mode No Feedback Edit Feedback

Experimental condition

Remedial Feedback ~a. 10 n = 9

Response—Sensitive Feedback n = 8 n = 10

Control condition

Minimum Feedback n = 7 -- Edit Feedback -- fl = 9

a

Number of participants in each group who completed both phases of the

experiment (N = 53).

Independent Variables. The two between-group independent variables were Training Mode and Testing Mode . The Training Mode had four

conditions: two experimental (Remedial Feedback and Response-Sensitive

Feedback) and two centre... (Minimum Feedback and Edit Feedback). Data

from the control conditions were combined for analysis and the resulting combination referred to as the Combined Control Feedback condition.

The Testing Mode had two conditions: No Feedback and Edit Feedback .

In addition to the two between-group variables, there were two

within-group independent variables: Message Type (friendly and enemy)

and Sessions (training and testing).

pçpendent Variables. The three dependent variables were mean

time per entry , mean accuracy of first entries , and mean accuracy of

final data entries.

6
~~~~~~~ -

~~~- ~~~~ -~~~~~~~ - .. - --- - - -~~~~

, - -- .-- - - -

Mean time per entry, as a function of training and testing feedback conditions, was the average time to complete each entry item of

a single message.

Accuracy of first data entries during both training and testing

was the accuracy of the first response a participant made for each

entry. Accuracy was measured as a percentage :

Number of correct items

Number of items possible

Accuracy of final data entries was measured during testing only.

If a participant made an error that was caught by the computer (e.g.,

a misspelling) and that the participant then corrected to a computer—

acceptable form , only the final response was counted as correct or incorrect. Thus, this analysis explored the quantity of errors that were

not detected by computer validation. If o response was correct the

first time, of course , the first was also the final entry.

RESULTS AND DISCUSSION

To assess the relative effects of the response-sensitive instructional strategy on speed and accuracy of performance during training

and transfer of training , the following dependent measures were analyzed :

mean time per entry , mean accuracy of first entries, and mean accuracy

of final data entries. Mean time per entry and mean accuracy of first

entries were analyzed for both training and transfer—testing phases of

the experiment. Mean accuracy of final data entries was analyzed only

for the transfer—testing phase . Since the mean time per entry during

the 12—item typing exercise (given at the beginning of the experiment)

showed a significant within-cell correlation with each of the dependent

measures , it was used as a covariate in all analyses.3

All covariate analyses and all subsequent simple effects tests

were accomplished using a significance level of p = .05. All simple

effects testing was carried out using either analysis of variance or

Tukey ’s HSD test where appropriate (Kirk, 1968).

3

Typing-exercise entry time was positively correlated with mean time

per entry (r = + .57) and negatively correlated with mean accuracy of

first data entries (r = — .43) and mean accuracy of final data entries

(r = - .44). Typing-exercise accuracy did not show any significant

within cell correlation with any of the three dependent measures

(r = — .09, r = +.lO , and r +.05 for entry time , first accuracy , and

final accuracy , respectively).

7

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Entry Time

The results of a four-way analysis of covariance performed on the

time taken per entry are presented in Table 3. The four factors analyzed were type of training feedback , type of transfer-testing feedback , type of message (enemy versus friendly) , and sessions (training

versus transfer—testing). The data from 18 participants were excluded

from all of the data analyses because six participants failed to complete the first 31 transfer—testing messages, one failed to complete

training , one became ill, one failed to return on the second day , and

nine were eliminated due to system malfunctions. Table 2 shows the

number of participants remaining in each of the training/transfer feedback combinations. A chi squared test was applied to the number of participants eliminated from each of the groups to determine the likelihood

that the loss of data from any of the groups was due to a systematic

bias. This test showed that the probability of such a systematic bias

was low ( X 2(2) = 2.26, p > .25).

Data from all 21 training messages were used whenever tra ining was

analyzed . Only the data from the first 31 transfer-testing messages

were selected for analysis because (a) most participants failed to cornplete all 43 possible messages, and (b) the first 31 messages included

participant responses to all 10 possible formats.

Training. The mean entry times for type of feedback given and

type of message received during training are presented in Table 4.

The results show unequivocally the potential time—saving characteristic

of the response—sensitive instructional strategy used here . During

training, Response—Sensitive Feedback produced a mean savings of 4.09

seconds per item over the Combined Control Feedback conditions on Enemy

messages (p < .01). Response—Sensitive Feedback also produced signif i—

cant time savings on Enemy training messages when compared to Remedial

Feedback ; the mean time per entry was 3.82 seconds faster under Response—

Sensitive Feedback (p < .01). Remedial and Combined Control Feedback

conditions did not differ significantly from one another on Enemy

training messages.

Because auto-fill could not occur on Friendly messages , it is

not surprising that the Response-Sensitive Feedback condition did not

differ significantly from the Remedial Feedback condi tion on mean item

entry times . Remedial Feedback produced significantly faster en try

times than did Combined Control Feedback on Friendly training messages

(p < .05), but Response-Sensitive Feedback did not . This result was

unexpected , because the Response—Sensitive and Remedial Feedback groups

received identical feedback on Friend ly messages. Failure of ResponseSensitive Feedback to produce faster en try times on Friendly messages

may have been due to the d i f fe r ences in feedback experienced by this

group between 1’r iendly messages without auto-fill and Enemy messages ,

on which auto—fill was available. The fac t tha t Enemy messages were

processed significantly faster than Friendly messaqes (X 10.04

B

~ 

--.--
Table 3

Analysis of Covariance for Mean Time per Entry

Source df MS F

Between subjects

Training Feedback (TF) 2 19.59 <1

Transfer-Testing Feedback (TTF) 1 6.11 <1

TF x TTF 2 34.84 1.58

Errorb 46 22.06

Within subjects

Message Type (M) 1 82.34 l5.45***

M x TF 2 37.85 7.lO**

MxTTF 1 .45 <1

MxTF x TTF 2 20.48 3.84*

Error1 47 5.33

Sessions (5) 1 1135.62 12l.58***

S x TF 2 50.62 4.29*

SxTTF 1 .01 <1

SxTF x TTF 2 10.69 <1

Error2 47 11.81

M x S 1 244.31 60.52***

N x S x TF 2 46.85 ll.6l***

MxSxTTF 1 .01 <1

MxSxTF x TTF 2 8.24 2.04

Error 3 

. 

47 4.04

*p < .05.

**p < .01.

< .001.

9

~~~~~~ 

________ _______________ 
-- .- ~~~- —-- —-~- -

~~

- .- -.- -

~ 

. -

sec/item versus X = 16.59 sec/item) in the Response-Sensitive group

(F( l, 94) = 82.67 , p < .001) lends support to this hypothesis. There

were no significant differences in entry times between Enemy and

Friendly messages for any other feedback group.

Table 4

Mean Item Entry Time in Seconds for Type of

Message and Mode of Training

a Message type

Training Mod e n Enemy Friendly

Training Session

Combined Control 16 14.13 17.47

Remedial Feedback 19 13.86 14.16

Response—Sensitive Feedback 18 10.04 16.59

Transfer—Testing Session

Combined Control 16 9.12 8.43

Remedial Feedback 19 9.55 8.65

Response—Sensitive Feedback 18 10.15 8.96

a

Number of scores combined across transfer—testing modes and participants.

The effect of Response—Sensitive Feedback on training time can be

summarized as follows. Compared to the Combined Control group, ResponseSensitive Feedback produc~ d a mean time savings of 40 minutes over the

course of the 21 training messages. It also resulted in a mean savings

of 37 minutes over Remedial Feedback training.

Transfer—Testing . The time savings produced by Response-Sensitive

Feedback training did not transfer to the simulated operational environment. In fact, no significant differential effects on transfer—testing

entry times were produced by any of the training feedback conditions.

As Table 4 shows, all participants made their en tries at abou t the same

rate during transfer-testing , and there were no significant differences

in entry times between Enemy and Friendly messages .

10 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~ —~

.—-.—:.. .—. - —, —r——~

——— -’— 

. 

!

~

uuI IuIIIII

~

These results contradict those of Strub (1975), who found faster

inputting speeds for Friendly messages under all conditions. However ,

Strub used mean time in minutes per format, whereas we used time in

seconds per entry as the measure of input entry speed. Use of the entry

time per format measurement necessarily makes input speed appear slower

for Enemy messages since the Enemy format contains on the average 12

more entries than the Friendly formats. If one applies the time-perentry analysis to Strub’s data, the differences in entry speed between

Enemy and Friendly messages virtually disappear, with Enemy message

entry times averaging slightly over 1.5 seconds slower than Friendly

message entry times.

It should be noted that type of transfer-testing feedback did not

influence response times. The error detection routines used in the

Edit Feedback condition did not significantly affect inputting speed.

Entry Accuracy

First—Entry Accuracy . A four—way analysis of covariance was used

to assess the effects of training and transfer—testing feedback conditions on accuracy of performance.4 The results of this analysis appear

in Table 5. The four factors analyzed were type of training feedback ,

type of transfer—testing feedback , type of message, and sessions.

Because a response-sensitive training strategy enables students

to bypass material already learned and to concentrate their efforts

on items not yet learned , it was assumed that the use of this strategy

would facilitate the transfer of MIOD training to operational testing

environments. Table 6 shows that the results of the present experiment

do not support this assumption. Only the Edit Feedback trained and

tested participants showed even a hint of positive transfer of training as evidenced by a small, nonsignificant increase in performance

accuracy.

unresolved error in the computer program used for automatically

scoring participant entries produced some falsely incorrect entries

and generated several instances of false error feedback during training. The incidence of such errors was small, and the effects on the

performance of the participants involved was judged to be minimal.

Specifically, there were 63 such cases of computer-generated error in

the 44,679 entries made during training (.14%). Prior to computing the

accuracy scores for each participant , however, all entry errors were

manually rescored and corrected for computer error .

L __ ___ 

I’

- i ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
- - —--- - - . ~~~~~~~~~— —-----.- 

Table S

Analysis of Covariance for First Entry Accuracy

Source df MS F

Between subjects

Training Feedback (TF) 2 1294.27 9~ 39**

Transfer—Testing Feedback (TTF ) 1 399.49 2.90

TF x TTF 2 140.42 1.02

Errorb 46 137.85

Within subjects

Message Type CM) 1 1165.84 60.40**

M x TF 2 159.77 8.28**

M x TTF 1 39.76 2.06

Mx TFxTTF 2 4.22 < 1

Error1 47 19.30

Sessions (S) 1 730.20 5l .l7**

S x TF 2 102.85 7.21**

SxTTF 1 .17 < 1

S x TF x TTF 2 83.34 5.84**

Error2 47 14.27

MxS 1 1.60 < 1

MxSxTF 2 12.15 <1

M x S x TTF 1 137.90 7~ 77**

MxS XTP x TTF 2 13.19 <1

• Error3 47 17.75

< .01.

**p < .001.

12

__ __ _ — .-- •

~~~
--

Table 6

Mean Percentage Accuracy Scores for Training and Transfer—Testing

Sessions as a Function of Training and

Transfer—Testing Feedback Modes

Session

Training Mode na Training Transfer—Testing

No Feedback Transfer-Testing Mode

Minimum Feedbackb 14 79.36 75.04

Remedial Feedback 20 69.35 87.10

Response—Sensitive Feedback 16 86.41 81.72

Edit Feedback Transfer-Testing Mode

Edit Feedbackb 18 81.72 82.61

Remedial Feedback 18 89.96 85.86

Response—Sensitive Feedback 20 91.94 83.76

a

Number of scores combined across message types and participants.

condition .

Under No Feedback testing conditions , both Minimum and ResponseSensitive Feedback training groups showed a significant decline in performance during transf~

..r-testing (F(l , 47) 12.33, p < .01). The decline in performance by the Remedial Feedback training group was not

significant.

Under Edit Feedback testing , both the Remedial and ResponseSensitive Feedback training groups showed a significant decline in

performance during transfer—testing (F(l , 47) = 10.60, p < .01 and

F(l , 47) = 46.89 , p < .01, respectively) . The Edit Feedback trained

and tested participants showed a small , if nonsignificant , gain in

performance accuracy during transfer—testing . This result is not surprising , since training and transfer-testing feedback conditions were 

.-

~~~~~-•-- .~~~~~~~~~~~~~~~~ - -~~

,•-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -

~~~~~~ -~~

--

~ • .. ---~~~-

~~~~-— --•

identical , and one would expect positive (or at least nonnegative)

transfer to occur .

Table 6 also shows that neither Remedial Feedback training nor

Response—Sensitive Feedback training provided participants with any

special advantage when tested under the Edit Feedback compared to the

No Feedback condition, the differences in performance in each case being trivial and nonsignificant . The Edit trained and tested participants were clearly superior in performance to those receiving Minii~u ’

Feedback training and No Feedback transfer-testing (F(l, 93) 5.68,

p < .05). This result was probably due to the slight positive transter

found in the Edit trained and tested participants combined with the

general decrement in transfer-testing performance experienced by all

other participants. The accux~~ y scores in Table 6 show that there

were no significant diff erences in tra ining performance between No

Feedback and Edit Feedback transfer-testing conditions for any of the

training groups. This finding is important for two reasons. First ,

the lack of differences between the two Remedial Feedback train ing

groups (F (l , 93) < 1, p > .05) and between the two Response-Sensitive

training groups (F( 1 , 93) = 3.42, p > .05) indicates that no systematic

confounding o9curred in the initial assignment of participants to these

groups. Secon d , the absence of differences between the two control

groups (F(1 , 93) < 1, p > .05) during training suggests that Minimum

Feedback and Edit Feedback are similar in their influence on response

accuracy.

Since transfer—testing feedback conditions were not in effect

during training and since the Minimum Feedback and Ed it Feedback groups

did not differ significantly during training , the transfer-testing

factor was ignored in analyzing the effect of type of train ing feedback on performance accuracy during the training session . Minimum

Feedback a

~

id Edit Feedback groups were considered to be a single group

(i.e., Combined Control) with an overall mean of 80.69 . The mean performance of Remedial and Response-Sensitive Feedback groups was virtually the same during training (89.64 and 89.48, respectively). Both

groups show significant ’

~ superior performance compared to the Combiicd

Control group when tested with Tukey ’s HSD test (p < .01 in botP cases).

During transfer-testing , the performance accuracy for Remedial and

Response—Sensitive Feedback trained groups was also higher than their

respective control groups under both No Feedback and Edit Feedback

testing conditions . However , the only statistically significant differ ence was for the Remedial Feedback tra ined group compared to the

Minimum Feedback trained group under No Feedback testinq (p .01).

Although Remedial and Response-Sensitive training strategies were mild ]y

eff ective in improving performa nce accuracy dur ing train ing, little of

this positive influence was transferred to either of the two testing

environments used here.

14

~ 

•
—, .——— —.-- .—-----•.-,•- —————— ,‘-—.w——--—-~

• .•

Strub (1975) found that participants completed Fr iendly message

formats with greater accuracy than Enemy message formats. Our results

agree with Strub’s in that the overall accuracy on Friendly messages

(87.32%) was significantly higher than on Enemy (82.65%) messages

(F(1 , 47) = 60.40 , p < .001). The analysis of covariance revealed a

significant training feedback x message type interaction (F(2 , 47) =

8.28, p < .001). This result, coupled with the fact that the training

feedback x message type x sessions interaction was not significant,

indicates tha t performance accuracy on Enemy and Friendly messages was

differentially affected by the various training feedback conditions .

Furthermore , this effect was essentially the same during transfer testing as it was during training . Table 7 shows the mean accuracy scores

on Enemy and Friendly messages for each of the three training feedback

conditions. Subsequent analyses showed that all three training feedback

groups performed more accurately on Friendly than on Enemy messages

(F(l , 47) = 57.06 , p < .001; F(l, 47) 17.32 , p < .001; and F(l , 47) =

7.59, p < .001 for Combined Control , Remedial Feedback , and ResponseSensitive groups, respectively). The results of HSD tests revealed

that on Enemy messages both Remedial and Response—Sensitive groups performed significantly better than the Combined Control group (p ~ .001

in both cases), but did not differ signi ficantly from each other in

performance accuracy . On Friendly messages , both Remedial and ResponseSensitive groups performed more accurately than the Combined Control

group , although the difference was significant only for the Remedial

group (p < .05). As on the Enemy messages , the two groups did not differ significantly from each other in performance accuracy. Remedial

and Response—Sensitive Feedback training led to reduced errors on both

Enemy and Friendly messages. However , Remedial Feedback was somewhat

more effective than Response—Sensitive Feedback in eliminating errors

on Friendly messages. This result must be interpreted cautiously since

auto—fill was not given on Friendly messages.

Table 7

Mean Percentar,a Accuracy Scores for Type of Message

and Mode of Training

a Message type

Training Mod e n Enemy Friendly

Combined Control 32 75.84 84.14

Remedial Feedback 38 85.98 90.17

Response—Sensitive Feedback 36 85.20 87.13

a

Number of scores combined across transfer-testing conditions , sessions,

and participants.

15

L~~~~~~~~~~
_____________ 

~~~~~~~~~~~~~~~~~ ~~~~~~-

~~-.~~~~ —--- ~~—-- - -~~

-v

~~ -

~~~~~~~~ a~- .:~~~~~~~~~ -

As can be seen in Table 8, the type of transfer-testing feedback

also affected performance on Enemy and Friendly messages differently .

Under Edit Feedback , performance was significantly more accurate on

Friendly messages than on Enemy messages (F(l , 94) = 36.97, p < .001) .

Performance did not differ under the No Feedback condition . These differences are primarily due to the lowered performance on Friendly messages found under No Feedback testing as compared to Edi t Feedback

testing. However , this difference was not statistically significant .

Table 8

Mean Percentage Testing Accuracy Scores for Type of Message

and Type of Transfer-Testing Feedback

a Message type

Transfer—Testing Feedback n Enemy Friendly

No Feedback 25 81.03 82.96

Edit Feedback 28 80.57 87.56

a

Number of scores combined across training feedback modes and participants.

First Versus Final Entry Accuracy . A four-way analysis of covariance was used to assess the relative effectiveness of the various training feedback conditions in reducing detectable and nondetectable error

rates. The results are presented in Table 9. The four factors analyzed

were type of training feedback , type of transfer-testing feedback , type

of message , and error detection accuracy. Error detection accuracy was

measured by comparing first and last entry accuracy during transfertesting. Differences b. iween first and last entry were observed only - •

in those participants receiving Edit Feedback during transfer-testing ,

since they alone were allowed to correct illegal entries made on f irst

entries. Most of the errors made in the testing situation were not

caught by the Edit routine. Although the difference in performance accuracy between the first entry and the entry finally accepted by the

computer was signif icant (F (l , 47) = 69.48, p < .001), it can hardly

be called important (mean accuracy = 84.12% and 85.21% for first and

final entries, respectively) . This performance difference of 1.09%

means that only 7% of all errors were detected by the Edit routine.

The lack of a significant error detection x accuracy training feedback

interaction (F(2, 47 ) < 1, p < .05) shows that all of the training feedback conditions were equally effective (ineffective?) in reducing final

error rates. Since none of the interactions involving either message

type and/or error detection accuracy were significant , it appears that

16

______ ________ .—~~~~~~~~~~~~ -~~~~~ ---- - . - - — -• ‘—-- --•
_____________________________________ -

the error detection routine was equally effective on Enemy and Friendly

messages.

Table 9

Analysis of Covariance for First Versus Last Attempt Accuracy

Source df MS F

Between subjects

Training Feedback (TF) 2 1022.37 6.33*

Transfer—Testing Feedback (TTF) 1 588.33 3.65

TF x TTF 2 335.45 2.06

Errorb 46 161.39

Within subjects

Attempts (A) 1 15.61 69.48**

AxTF 2 .11 <1

A x TTF 1 15.61 69.48**

AxTFxTTF 2 .11 <1

Error1 47 .22

Message Type CM) 1 1112.43 29.67**

M x TF 2 101.68 2.71

M x TTF 1 343.09 9.15*

MxTF x TTF 2 7.66 <1

Error2 47 37.49

AxM 1 .22 3.58

AxMx TF 2 .14 2.24

AxMxTTF 1 .22 3.58

A x M x TF x TTF 2 .14 2.24

Error 3 47 .06

*p < .01.

**p < .001.

17

___ _ _
SUMMARY AND CONCLUSIONS

In summary, a response—sensitive training technique, covering only

those format items in which errors occur, is effective in reducing

training time. This technique is also at least as effective as other

training strategies in reducing error rates. However, the decision to

adopt a response-sensitive strategy for a specific training problem

should consider the trade—off between the cost of training development

and the benefit of reduced training time.

The type of error detection routine provided by Edit Feedback was

only minimally effective in reducing errors and was not enhanced by any

of the training methods used. The results of the present experiment

suggest that training probably will not help catch many of the errors

that escape the usual operational error detection routines. Although

more sophisticated error-detection routines might help reduce such errors, a simpler and perhaps more effective way co eliminate the errors

would be to clarify the meaning of the terms used as legal entries in

the various formats. For example, there was confusion among participants

about the terms Observ~tion and Reconnaissance as descriptors for information sources. The distinction between other items, such as “Mines”

versus “Demolition Equipment” and “Demolished” versus “Destroyed ,” was

also unclear. If these terms were changed or clarified , a fairly high

percentage of otherwise undetectable and unavoidable errors might be

eliminated.

18

• —~~~~~~~~~---- - —.-~ •---• .-. - - -- •- —~~ --- ~~

. -. —~~~~~
REFERENCES

Alderman , I. Tactical Data Inputting : Research in Operator Performance

and Training . In Proceedings of Second National Symposium on

Management of Data Elements in Information Processing . Gaithers—

burg , Md.: National Bureau of Standards, 23-24 October 1975.

Atkinson , R. C. Ingredients for a Theory of Instruction . American Psy— chologist, 1972, 27, p. 921—931.

Atkinson , R. C., & Paulson, J. A. An Approach to the Psychology of

Instruction. Psychological Bulletin, 1972 , 78 , No. 1, p. 49—61.

Kirk, R. E. ~~çperimental Desi9n: • 

Procedures for the Behavioral Sciences. Belmont, Calif.: Brooks/Cole Publishing, 1968.

Strub , M. H. Evaluation of Man—Computer Input Techn iques for Military

Information Systems. ARI Technical Research Note 226, May 1971.

(AD 730 315).

Strub , M. H. Automated Aids to On-Line Tactical Data Inputting . ARI Technical Research Note 262, February 1975. (AD A0l30350).

19 

-—.~~ ----- . —------ ----------—.. -- -.- ~~- • ..-
APPENDIX

DESCRIPTION OF DATA ENTRY FORMATS

Table A-l

Format Index

Designator Description

Ecla Enemy situation/action

UC1 Front line trace/coordination point

UD1A Command post/center of mass

UE3. General on post/combat on post

UF1 Assembly area/operational area

UG1A Line of departure/phase lines

UGIB Axis of advance/direction of attack

UG1C Objectives

UG1F Movement control points

UGh Fire control lines

a

All information about enemy activity was entered on the EC1 format.

The remaining nine formats were used to describe friendly activities

only.

~~~~~~~~~~~~~~~~~~~~ 

.

--.

~~~~~~~~~~~~~~~

•

-

-.

~~~~~

-

~~~

-- ------ 

-

~~~

-- -

. -

.

~~~~~~~~~~~

-

L 21
~~~~~~ ~~~~

Table A-2

Training Formats

Message number Format required Message number Format required

1 UC1 33 UG1F

2 UE1 34 UF1

3 UE1 35 EC1

4 UE1 36

5 UD1A 37

6 UG1A 38

7 Ed 39 “

8 ““ 40 UE1

9 “ ~‘ 41 “

10 “ “ 42 Ed

11 “ II 43

12 44 “

13 j

~ “ 45

14 “ 46

15 ~~~~~II 47

16 “ 48

17 “ ‘

~ 49

18 “ “ 50 TJG1C

19 UG1C 51 UG1A

20 uGlA 52a UG1B

21 UG1B 53 UC1

22 UGh 54

23 uci 55

24 UGh 56 UD1A

25 “ “ 57 Ed

26 UD1A 58 “

27 UE1 59 “

28 ,, ,, 60

29 “ 61

30 EC1 62 “

31 “ 63 II

32 ,, ,, 64 “

a

participant responses to items beyond message number 52 were not used

in data analysis. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-

~

LLfl ’

~ ~~ flCit~L

~

~~~ 

PA~~ ~~ ~~ IQQ - ‘

~~~~

1EO

~ ~~~ 

Ju’~~~~

Table A-3

• Format Scorable Entries

Format designator Number of entries

Edl 45

UC1 24

UD1A 21

UE1 25

UF1 31

UG1A 46

UG1B 47

UG1C 59

UG1F 51

UGh 46

Data Entry Format--printouts

~~

.C1 ENSIT UMTM ML.~ SAG~ MS(,’4JR( )Pê~Et.( )SC I’Y ( )W~

I6( )RFS1~~( )

)RPT—ID( )

~( 1 FNCY( ) SOUHC~~( /

)EM—IJ N1T( / / -‘SUM J———A (. I V — ~— IiIUAN T— D ~Sr R1PT ION— — — -———— L .OCAT ION— / / / /

/ / / /

/ / / / )

/ / / /

/ / / /

~ A~*NT ( / / ) R ~~iAT—TO(

W~ P4AUK5

)IS1P( ) INT EI

~

€ST ( )

_ ___
--‘ .

~~~~~~~

---—-—--.

~~~~~~

ThIS PAGE IS BEST ~UAL1T’f FP.LCTICABLZ

B’1

~~ ~~X ~~ Nzsa~

) TO DDC

Data Entry Format--Printouts (continued )

‘JC i FLT CO~~~U L)A IA t4ESSAG~ )PkEC ( )SCTT( )OIUG( )PFSTR( )

JNTT~~ID— U~~~IF—NA M~.( / / / /

FLT— COOW I) i ( )fLT—COORL)-2 (

I L r — COO ~~ — i( )FLT — rOO RD— 4 ( )

CO— ON— L IN~~~POSIT ( /

~

4N—tjN—1 I~~.~~ OSL I C / )

LT—COOR I )— PF (

L T~ C0OWL )—PT TIM~~(

p

~T-COOMU-PI TIME~

(

~

JU1A CP—CUM UATA MESSAbI

)PkLC ( )SCTY ( )ORIG ( )RFSTN (

IJNIT— 1D— Ok—1F— NAML ( / / / / — — I TEM— /LOCAT IUN/——— —T 1M E——— /

/ )

/ )

— ALT—CP ( /

CUP— CUM ( / )

PLN—COM C /

tiE i GOP—CUE’ OA IA ME~

SSt~6E

)PREC ( )SCTY ( )URIG( )pFST I

~( )

JNj T~~

IL)—tJR—IF—NAME ( / / / /

t,OP—NA M~ I

LOC ( )

TIM~~ FP ( )TIME~ TQ (

COP—NAP4 ~ (

LUC (

TI Pft .’fR( )TIME.,.TO ( )

24

~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Q~~~ ’

~ 

l’

~~

~t

~

iS PAGE ~° .

FROI& CO?’

~ ~

Data Entry Format--Printouts (continued)

UFi OP AW EA DATA MESSAbE

)PkEC ( )SCT Y( )ORI6( )RFSTRC

(J N1T—1 Q— O R TF ..NAM ~. ( / / / /

)E FF —T TME ( )

)P—APEA— j .” NAME ( )

LOC (

)

~

4.EMI~

W P(S (

)P—A ~ EA — ,)— NAM E (

LO C (

, )

HE M AP S C

U~

5u

~ 

CO(

~ 

MtM SUHES DAT A MESSA(1~ 

PHASE LINE S

) PRE~

C( )SCTY )OR IG( )PFST P (

IJNTT—1 D—Q H..TF— NA ML ( / / / /

PL— J —NA M E

LUC( )

TLMF ( )

PL— 2—NAM E (

LOC (

T I’* (

PL—3— NA M~~(

LOC ( I ) )

PL—4— NAM F.(

LUC ( I I

T 1 ME (

~~Ll)—NA ME C

1.0 CC I

IIMI ( )

_______
—~~~~~~~~~~~~~~~ -

~~~~~ -~~~~-~~~~~ -- •- . • • •.---

THIS PAGE IS BEST QUAI~

tfl PRACrICARLE

~~~~~~~ ~O D~.C

Data Entry Printouts (continued)

u

~

ifti CON MEAS DA IM MESSAG I . AXIS WV • D IR ATTACK

MSG—NN ( ) PHE(. C )SCTY ( )ORIG( )RFSTK( )

U NTT — Lt )—ik ” ~ F” NAM~

.( / / / /

~

X TS— 0f—ADV— 1 — NAM E ( )

LOC ( • )

LUC ( • )

,

~X I S~~

CJF— gDV—~~

—NAML ( )

LUC ( •

~0IH~ 0F—MT )—NAMI .C )

l.OC ( . •

)

LUC ( • • — I) — 0F—A tK— ~ —NA ML. (

L0(.(

tit, 1C CON Mt.A 5 L i A IA MESSAG i O~ JECT tvE S

4SC~— NN ( )PWE ( ( )SCTY ( )OR IG( )PFSTH )

i J N I T—IO— O P ~— 1I~—NA ME ( / / / /

)TIME (

10C C

I • )

HF SI’—UNI I C / / / / / )

)TIME (

LI-IC C

I I . )

~~SP —UN1T ( / / / / /

‘HJ—3— NA M E ( )T IME (

L O CC , • I I )

~~s~ —uN I r ( / / / / / )

26 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ -. - -~~~~~~•~~~~~~- - . .-~~~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~~~ ••
~~•- --

•. -

~~ -U- ~~~~~~~~~~ - . - - 

.

THIS PAGE ~~~~~~~~~~~~~~~~~~~~~~~~~~~

FROM COPY FURN-ISH~ ) TO DDC ...._.—

Data Entry Format--Printouts (continued)

U61F CON M EAS DATA ME SSAGI . MUT CIL PO TN TS

t4SG—NR ( )PRECC )SC1Y ( )OHIG ( )PFST~~ INTI— IO—O k—T F—NAME ( / / / / )

— 1TLM—N k—LUC. .A1 10N——L TEM —N ~~.LOCAT I0N

—S P—1 C / )— SP— 2 ( /

— SP — IC / )—SP— 4 ( /

— HP— i C / )— IIP—~~( /

— RP— U / )—W P— 4 ( /

LOP— I ( / ) LUP—2 ( /

LIJP—3 ( / ) LUP—4 ( / ) ( KP— 1 C / ) CKP—~~C /

CKP— ~( / )CKP— 4( /

— PP— i C / ) — PP—~~( / — PP— IC / )—P P— 4 ( /

~

iiiI CON M tMS UAIA MESSA~w FIRE CONT POI LINE S

)PRE (,( )SCTY ( )OWIGC )PFSTR (

IJN IT—1U — OR—Tf—NA M E ( / / / / )

I ScL—LOc

)IIME (

—NFL—LOC ( •

—— ‘ )TIME ( )

—FCL—LOC ‘ •

)TIME (

~PH—NA M E ( )IOC I 

I I )

‘— — — HHQ—NAME ( )IOC ( s——— I • I )

27
DISTRIBUTION

ARI Distributi on List

4 OASD IMItRA) 2 HQUSACDEC, Ft Ord , ATTN Library

2 HODA )DAMI-CSZ) 1 HOUSACDEC, Ft Ord. ATTN : ATEC -EX—E—Hum Factor,

HODA (DAPE-PBR 2 USAEEC. Ft Benjamin Harrison, ATTN: Library

1 HODA (DAMA .AR ) 1 USAPACDC, Ft Benjamin Harrison , AnN: ATCP—HR

1 HODA (DAPE-HRE-PO) 1 USA Comm— Elect Sch , Ft Monmouth, ATTN: ATSN--EA

1 HODA (SORDID) 1 USAEC . Ft Monmosith, ATTN: AMSEL—CT—HOP

1 HODA (DAMI DOT-CI 1 USAEC. Ft Monmouth , ATrN: AMSEL—PA-P

1 HODA (DAPC PMZ-A) 1 USAEC, Ft Monmouth, ATTN: AMSEL—S l—Cb

1 HQDA (DACH-PPZ-A) 1 USAEC , Ft Monmouth, ATTN: C, F~

scl Day Br

1 HODA (DAPE.HRE) 1 USA Materials Sys Anal Agcy, Aberd een . ATTN AMXSY— P

1 HODA (DAPE .MPO .C) 1 Edgewood Arsenal, Aberdeen, ATTN: SAREA—BL-- H

1 HODA (DAPE .DW) I USA Ord Ctr & Sch. Aberdeen ATTN: ATSL—TEM—C

HODA IDAPE-HRL) 2 USA Hum Engr Lab. Aberdeen, AT1N: Library/Dir

I HODA (DAPE-CPS) I USA Combat Arms Tng Bd, Ft Bennin g, ATTN Ad Supervisor

I HODA (OAFD-MFA) 1 USA Infantry Hum Rsch Unit, Ft Benning, ATTN: Chief

I HODA (DARD-ARS-P) 1 USA Infantry Bd. Ft Banning, ATTN: STEBC—TE—T

1 HQDA (DAPC-PAS-A) 1 USASMA. Ft Bliss , ATTN: ATSS—LRC

1 HQDA (DUSA-OR) 1 USA Air Del Sch. Ft Bl iss. ATTN ATSA—CTO—ME

1 HODA (DAMO -ROR ) 1 USA Air Del Sch. Ft Bliss , ATTN: Tech Lib

1 HODA )DASG) I USA Air Del Bd. Ft Blits , ATTN: PILES

1 HQDA IDA1O-PI) 1 USA Air Del Bd , Ft Bliss . ATTN: STEBD—P O

I Chief, Consult Div (DA OTSG). Adelphi, MD 1 USA Cmd & General Stf College, Ft Leavenworth, ATTN: Lib

I Mil Aist. Hum Res, 000R&E. OAt) (E&LS ) I USA Cmd & General Stl College. Ft Lesvenworth, ATTN: ATSW —SE—L

1 HO USARAL , APO Seattle. ATTN ARA GP-R 1 USA Cmd & General Stf College. Ft Leavenworth. ATTN: Ed Advisor

1 HO F,r~t Army, ATTN : AFKA -Ol.TI I USA Combined Arms Cmbt 0ev Act, Ft Leavenworth, ATTN: DepCdr

2 HO Fifth Army. Ft Sam Houston I USA Combiri.d Arms Cmbt Dew Act . Ft Leavenworth, An N: CCS

I Dir, Army Stf Studies Ofc . ATTN OAVCSA ~

os~ 1 USA Combined Arm, Cmbt Dew Act, Ft Leavenworth, ATTN ATCASA

1 Ofc Chief of Stf , Studiet Olc 1 USA Combined Arms Cmbt Dee Act, Ft Leavenworth, ATTN ATCACO --F

I DCSPER. ATTN: CPSIOCP 1 USA Combined Arms Cmbt 0ev Act , Ft Leavenworth, AIIM: ATCACC—Cl

1 The Army Lib , Pentagon , ATTN RSB Chill 1 USAECOM. Night Vision Lab , Ft Bulvoir , ATTN AMSEL—NV—SD

1 The Army Lib , Pentagon. ATTN: ANRAL 3 USA Computer Sys Cmd , Ft Belvoir , ATTN Tech Library

1 Ofc, Asst Sect of the Army IR&O) 1 USAMERDC , Ft Belvoir, ATTN: STSFB—0O

1 Tech Support Ok, OJCS I USA Eng Sch, Ft Belvoir , ATTN: Library

1 USASA. Arlington, ATIN: lARD-I 1 USA Topographic Lab, Ft Belvoir , ATIN: ETL -TO—S

I USA Rscf. Ofc. Durham, ATTN: Life Sciences Dir I USA Topographic Lab, Ft Belvoir , ATTN: STINFO Center

2 USARIEM, Netick , ATTN SGRO.UE.CA 1 USA Topographic Lab, Ft Belvoir. ATTN ETL—GSL

1 USATTC. Ft Clayton, ATTN: STETC-MO.A 1 USA Intelligence Ctr & Sch, Ft Huachuca , ATiN: CTD—MS

1 USAIMA. Ft Bragg, ATTN: ATSU-CTD.OM I USA IntellIgenCe Ctr & Sch, Ft Huachuce. ATTN: ATS—CTD—MS

1 USAIMA, Ft Bragg, ATTI’s: Marguat Lib 1 USA Intelligence Ctr & Sch, Ft ltuacfluce, ATTN: ATSI—TE

1 US WAC CU & Sch. Ft McClellan , ATTN : Lib 1 USA Intelligence Ctr & Sch. Ft Huachuca, AT1N: ATSI—TEX—GS

1 US WAC CU & Sth, Ft McClellan, ATTN: Tng Dir 1 USA Intelligence Ctr & Sch, Ft Huachuce, ATTN: ATSI—CTS—OR

I USA Quartermaster Sch, Ft Lie. AIIM: ATSM.Tt I USA Intelligence Ctr & Sch, Ft Huechuca , ATTN ATSI—CTD—DT

1 Intelligence Material Div Ofc, EWL. Ft Holabird I USA Intelligence Ctr & Sch, Ft Hu.chuca, ATTN: ATSI—CTO—CS

1 USA SE Signal Sch. Ft Gordon, A IIM: ATSO-EA 1 USA Intelligence CU & Sch, Ft Huachuc ,, ATTN: DAS/SRD

1 USA Chaplain Csv & Sdt, Ft Hamilton , AIIM: ATSC.TE.RD 1 USA Intelligence CU & Sch, Ft Huachuca . AIIM: ATSI—TEM

I USATSCH, Ft Eust,s , ATTN: Educ Advisor I USA Intelligence CU & Sch. Ft Huachuca, ATTN: Library

1 USA Was Colleqe, Carlisle Barrecks , ATTN: Lib I COP, HO Ft Huachuca, ATTN: Tech P.1 Div

2 WRAIR , Neu,opsychsatry Div 2 COP, USA Electronic Prvg Grd . ATIN: STEEP—MT—S

1 DLI, SDA, Monterey I CDR , Project MASSTER , ATTN: Tech Info Center

I USA Concept Anal A~~y. Bethesda, AIIM: MOCA.WGC I Hq MASSTER , USATRADOC . LNO

1 USA Concept Anal Aqcy, Bethesda, ATTN: MOCA.MR I Reieerch Institute, HO MASSTER . Ft Hood

I USA Concept Anal Agcy, Bethesda, AIIM: MOCA-JF I USA Recruiting Cmd, Ft Sherdian , ATTN: USARCPM-P

I USA A,t ic Test Ctr , APO Seattle, ATTN: STEAC.MO.ASt. 1 Senior Army Adv ., USAFAGOD/TAC , Elgin AF Aus Fid No, 9

1 USA ArI,. TesI CI., APO Seattle, ATTN: AMSTE.PL.T$ 1 HO USARPAC. DCSPER , APO SF 96558, ATTN: GPPE-SE

I USA Armament Cm.), Redstone Arsenal, AIIM: A’ISK.TEM I Stimson Lib , Academy of Health Sciences, rt Sam Houston

I USA Armament Cmii, Rock Island, AIIM: AM$A R-T DC 1 Min e Corps Inst., ATIN: Derm-MCI

I FAA NAFEC. Atlan t ic City, ATTN t ibrary I HOUSMC, Commandant , ATTN: Code MIMI 51

I FAA MAP EC AlIens,. City AiIM: Hum Enp Br 1 HOUSMC, Commandant. ATIN: Cod.’ MPI 20

I FAA Ae,on*st,cal CI’, OkIif ~

om. CIty, AIIM: AAC-440 2 USCG Academy, New London, ATTN A,ln,

~

ssio n

2 USA FId Arty Sd,, Ft Sill , ATTN: Library 2 USCG Academy, New London ATTN: Library

1 USA Amine 5th, Ft Knox , ATTN Libr ary I USCG Tr.ining Ctr . NY , A’fl’ N: CO

I USA Armor Scfs, Ft Knos, AIIM: AT$S.Dl.E I USCG Tr.lning Ctr , NY, ATTN: Educ Svc Ofc

I USA Armor ScSi . Ft Knox , AIIM: ATSe OI.TP I USCG, Psycliol Ret Br, DC, ATTN: GP 1/62

I USA Armor Sth. Ft Knox . AIIM: ATSB.CD.AO 1 HO Mid—Range Br , MC Oat . Ouantico, ATTN: P&S Div

_ ~~~~~~~~~~~~ 1

:A~J
I US Marine Corps Licision Ofc, AMC, Alex andria , ATTN : AM CGS—F 1 Del & Civil Inst of Enviro Medicine, Canada

1 USAIRADOC, Ft Monro., AIIM: ATRO-ED 1 Al

~ CRESS. Kemlngt on, AIIM: Info Svs Sr

6 USATRADOC. Ft Monroe. AIIM: ATPR—AD I Mllliaerpeykolcgisk Ijensate. Copebagen

1 USATRADOC. Ft Monroe, AIIM: ATTS-EA I Mlllta’y Attache, French Embassy. AIIM: Doc Sec

I USA Forces Cmd, Ft McPherson , AIIM: LIbrary 1 Med.cin Chef. C.E.R.P.A.—Arsenel, Toulon/Naval France

2 USA Aviation Test Sd, Ft Rucker , ATTN: STEBG—P0 I Prin Sclenfific Off, *pp~ Hum £n~ Rids Dlv, Mlni*ry

1 USA Agcy for Aviation Safety, Ft Ruckar, AIIM: Llb.’jy . Of Defense. New Delhi

I USA Agcy for Aviation Safety, Ft Pucker, ATI’N: Ed

~~ ~~~~~ 1 Pert Rich Ofc Ubrary, AKA, Israel Defense Forces

I USA Aviation Sch. Ft Pucker , AIIM: P0 Drawer 0 1 Minleterls van Detent’

s., DOOP/KL Afd Sociaal

1 HOUSA Aviation Sys Cmd, St Louis. ATTN: AMSAV—ZOR Psydsologlache Zakan, The Hague, Netherlands

2 USA AvIatIon Bye lit Act., Edwards AFB, A’rfN: SAVTE—T

I USA Air Del ScSi. Ft Bliss, AIIM: AISA TEN

I USA Air Mobility Rich & Dew Lab, Moffett Fld, AIIM: SAVOL—AS

1 USA Aviation Sch. Ret Tng Mgt. Ft Pucker, ATTN: AIST—T—RTM

1 USA Aviation ScSi, CO. Ft Pucker, AIIM: AI$T—O--A

1 HO, DARCOM. Alexandria, AIIM: AMX CD—TL

I HO. DARCOM . Alexandria, ATT N: CDR

I US Military Academy, West Point, ATTN: Serials Unit

I US Military Academy, West Point, AIIM: Ofc of Milt Ldnshp

1 US Military Acadun,y, W’st Point, AIIM: MAOR

1 USA Stanclerdi,alion Op. UK, FPO NY. AIIM: MASE—GC

1 Ott: of Naval Rtcl,, Arlington. ATIN: Code 462

3 Ofc of Naval Rich, Arlington, AIIM: Code 458

1 Ofc of Naval Rss h, Arlington. ATTN: Cods 450

1 OIc of Naval Rich, Arlington. ATIN: Code 441

I Najal Aerospc Med Ret Lab, Pensacola, AIIM: Acous Sch Dlv

1 Naval Aerospc Med Psi Lab, Pensacola, ATIN: Code L51

1 Naval Mrospc Med Ret Lab, Pensacola , AIIM: Code L5

I Chief of NavPers , AIIM: Pers.OR

1 NAVAIRSTA, Norfolk, ATIN: Safety CV

1 Nay Oceanographic, DC, ATIN: Code 6251. Charts & Tech

1 Center of Naval Anal, AIIN: Doc Ctr

1 NavAirSysCom AIIM: AIR—53I3C

1 Nav BuMed.ATTN: 713

1 NayHelicopterSub$qua 2, FPO SF 96601

1 AFHRL (FT) William AFB

1 AFHRL (TI) Low~y AFB

I AFHRL. (AS) WPAFB,OH

2 AFH RL (flOJZ) Brooks AFB

1 AFHRL (OOJN) Lackland AFB

1 HOUSAF (INYSDI

I HOUSAF (D PXXA )

1 AFVTG (AD) Rando’pts AFB

3 AMRL (HE) WPAFB, OH

2 AF I nst of Trch , WPAFB , OH, ATIN: ENE/SI,.

1 ATC (XPT DI Raiulolpt AFt)

1 USAF ArrnM.:.l Liii, Brook s AFB (SUL -4), AIIM: DOC SEC

1 AFOSR (NLI, Arlington

I AF Log Cmd, McClellan AFB, AIIM: ALC/DPCRB

1 Air Force Academy. CO. A’fl’N: Dept of Sal Sen

5 NavPers & 0ev Ctr, San Diego

2 Navy M,’tl Neuiopsyr,hiatnic Rich Unit. San Diego

1 N

~

iv Electronic Lab. San Diego , AIIM: Rat Lab

1 Niw Irnit(’en. Sari Diego AIIM: Code 9000—Lib

1 MavPuvlGraSch. Monterey, ATIN: Code 5SAa

1 NavPostGsaScli . Monterey , ATIN: Code 2124

I Nec TrngEqtiipCtr. Orlando. ATTN: Tech Lib

1 US Di ps ni Labor , DC, AIIM: Manpower Admin

1 (iS Dept of Justice. DC. ATTN Drug Enforce Admin

1 Nat Ru, of Standards, DC. ATIN: Computer Info Section

1 Nit CIr,ariivt H,,,iio for MH-

~Into, Rockvllle

1 O,’,uver Firiloral CIr, Lakewood, AIIM: BIM

12 Dnhm,n Do,,,inontatIon Center

4 Dir Psy ch , Army Hq~ Russell Ofcs, Canliorru

I Sr.i.,,iIi (ic Arlv”,t Nil Sd, Army H~, Russell Ole., Canberra

I 1,5,1 aw l Air AtI.,i:l,o, Asittr,an Sinliasty

I ~~~~~ .5- R,’,-I.r,ch,, Des F,w:r,,u,rt , Humalne da le Defense

Nuti,,n.,l.’

, II, ,,:.‘,sk

7 Caai.xlsa,i .Ini~~I Still Wwrhsutgto.i

I C/As , Ss51f , Royal Canadian AT . AIIM: Pars Ste Anal Br

3 Clod . Canadian I)e$ Rich Staff . AIIM: C/CRDS(W)

4 fls$t.sh l~ ’t Staff , British Embassy. Washington

30

_ _ _