~~~ LcaI~~~~327 _

~~ 

TA COMPARATI yE ANALY SIS OF METHODS FOR

~ 

/ TACTICAL DATA INPUTTING . —

-7

/ ~~~~~~~~ Ailson F .

~

FieIds , Rictiard EiMalsano/and

Charles F.~Marsha Il /

~

~ C)

~~LLJ

BATTLEFIELD INFORMATION SYSTEMS TECHNICAL AREA

C..

~ 1/ -

/

—~~~~~~ ;

)

(

~~~~

-

- - 

~~ ~~ 

31 i

~

18

~~~

U. S. Army

Research Institute for the Behavioral and Social Sciences

i/i

Approve d to, public release; distribution unl rn I l i d .

•

1 Qii 0 14 ,,.

~~~~~

.-

~~

-—-———

~~~~~~ 1

,

±I
U. S. ARM Y RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating A gency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WILLIAM L. IIAUSEI(

JOSEPh ZEII)NER Colonel, US Army

Technical Director Commander

NOTICES

DISTRIBUTION Primary distribution of thit report has bee n made by API. Please eddrest correspondence concerning distribution of reports to: U. S. Army Research Institute for the Behavioral and Soc

~

.l Sciences. ATTN PERI l’, 5001 Fusentr ~

n

~

c, Avenue AIexandrua~ Virginia 22333,

FINAL DISPOSITION This report may he destroyed when Iii no longe r needed. Please do not return it to th. LI S. Army Research Institute foi the Behavioral and Social Sciences

~QJj The lindings in this repo’s Ire not to be construed as an official Department cit the Army position , unless so des ignate d by other IUthOrise,j documents . 

-. -- -, --—. .

— ~~~~~~ “ —~-— --—..-—- .

~ ~--~-.-- .- —~~~~.-~~~~~--~~~a - --~..-~~~~~~~~~ --
Unclassified

SECURITY CLASSIFICATION OF THIS PAGE (Wh en Data Entered) __________________________________

~~~ A ~ READ INSTRUCTIONS rc i-ORT DOLUMENTMTIUN ~~~~~~~~ BEFORE COMPLETING FORM

I. REPORT NUMBER - 2. GOVT ACCESSION NO 3. RECIPiENT’S CATALOG NUMBER

Technical Paper 327

4. TITLE (aed Subtftl.) 5. TYP E OF REPORT A PERIOD COVERED

A COMPARATIVE ANALYSIS OF METHODS FOR TACTICAL

DATA INPUTTIN G --

6. PERFORMING ORG. REPORT NUMBER

7. A UTHOR(.) B. CONTRACT OR GRANT NUMBER( .)

Fields , A. F., Maisano , R. F ., and Marshall , C. F. ——

a. PERFORMING ORGANIZATION NAME AND ADDRESS ~

O PROGRAM ELF.MENT. PROJ ECT , TA SK

Army Research Institute for the Behavioral and APEA & WORK UNIT NUMBERS

Social Sciences ~

PERI—OS) 2Q763743A774

5001 Eisenhower Avenue , Alexandria , Va. 22333

1 1 . CONTROLLIN G OFFICE NAME AND ADDRESS 12. REPOR T DATE

Army Combined Arms Combat Development Activity September 1978

Fort Leavenworth , Kans . 13 . NUM BEROF PAGES

_______________________________________________________ 

28

~

‘t4. MONITORING AG ENCY NAME 6 ADORESS(I1 different fro m Controlling Office) IS. SECURITY CLASS. (of thi. report)

—— Unclassified

ISa . DEC LASS IFICATION/DOWN GPA DING SCHEDIJLE

16. DISTRIBUTION STATEMENT (of thii, Report)

Approved for public release ; distribution unlimited .

17. DISTRIBUTION STATEMENT (of the ebetract entered in ffZock 20, II dif ferent from Reo.irt)

IA. SUPPLEMENTARY NOTES

19. KEY WORDS (ContIn.~

e on rev.,.. elite if n.c.A.a,y end identify by block number)

Military Information Systems

Man—compu ter Interaction

Input Procedures

I Computer—assisted Messaqe Input

,j

~,

2O. ABSTRACT (Continu. en ,øv•,e• .id. if n.e.e. ~~~ end identify by block numb.,)

“Nearly all information in tactical operations systems is input manually.

Two problems that arisc’ when manually inputting data are (a) the introduction

of ~rrors in translatinq i nformation into computer format and (b) t.he introduction of a bottleneck in total system response time . Therefore , alternative methods of input tinq data for accuracy and speed should be evaluated . _— ‘T ~~~~~~~~

DO ~~~~~~~~~~~~ 1473 EOITION OF t NOV BB ,s o.SOLETE Unclassified

SECURITY CI.ASSIFICATICS W OF THIS PAGE (Wh.n Pete gnl.r.d)

- . — - - -~~~- - — - - - - - - — —-— -~~-~~- — - - - -~~-~~~~~ —~—-———. ..—~

..—-—.-——-—- ~.——-

~

——.——— — .~~~~~~ - - - - —
____________________ _ - -

Ur~

ciassirieO

SECURITY CLASSIFICATION OF THIS PAOE(Wlu .n Dat. ~ nt.r.4)

20.

Four methods were examined for s1,c~~d and accuracy in iiip Uttiflq

tactical messages concerning enemy activity into an Army computer format . The methods were (a) typing--the user types the appropriate codes

into a message format; (b) typing with an error corrector--the computer

automatically attempts to correct common spelling and/or typing errors ;

(C) menus——the user indicates which of the legal entries is desired from

a list; and (d) typing with autocompletion and an English option--the

user must type only sufficii -

~

ni.

~ characters to uniquely identify the item ,

using either the appropriate code or its English definition .

Thirty—two enlisted persons representative of the class of military

e personnel who might serve as input device operators input one of four sets

of nine messages using each inputting method .

‘The use of menus was tL~ ~ &st accurate inputting method . For users

of limited experience (1 day of inputting) , there were no differences in

speed among the inputting methods .

Consideration should be given to the adoption of menus in tactical

operation systems with a menu override option for experienced users.

The use of an error corrector, autocompletion , or an English option

is probably not warranted unless operational use shows a specific need

for such an aid.

\

~

j .. ~~

.. 

~~~

. 

. .

.

~

Unclassified

SECURITY CLASSIFICATION OF THIS PAGE(W7ien Date Ent.r.d)

~

.. —-—— — — —-- —. —-.-.—-—--.—.- —-.

- ———‘. - ---.- - —— ~— — — - .-.—— ——.——.~~~.- . -- ,-~~

..

~~

--—- —-— - ._ ._ ~

_ .1~~—
-

~ - —-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ — -~ - . —- ____

Technical Paper 327

A COMPARATIVE ANALYSI S OF METHODS FOR

TACTICAL DATA iNPUTTING

Alison F. Fields, Richard E. Malsario, and

Charles F. Marshall

Edgar M. Johnson, Work Unit Leader

BATTLEFIELD INFORMATION SYSTEMS TECHNICAL AREA

Submitted as complete and Approved By: technically accurate, by:

Edgar M. Johnson A.H. Blrnbaum, Acting Director Technical Area Chief ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

Joseph Zeidner

TECHNICAL DIRECTOR

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5007 Eisenho~~r Avenue, Alexandria, Virginia 22333

Office . Deputy Chief of Staff for Personnel

Department of the Army

S ptsmber 1978

Army Project Number Tactical Operations System 20763?43A 774

fti~pr~~~d for public rsls...i; distribution Un limit , I

____ __ 

~~--- .

- - - - - -~~~-~~~~~~~~~— - -~~~~~ ~~~~ “

--— -- -—-- .-—~~~~~~~
ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any f ind ings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

— — - —~ 

~~~—~~- -- - --- - --- --- - —--- - - ---~~—~~~-- --.~~~~ ---- -~ -— - -~~- - .~~~ - —--—~~~~~~ ~~~~~~~~
—- - 

FOREWORD —

The Battlefield Information Systems Technical Area of the Army Research Institute (ARI) is concerned with the demands of increasingly

complex battlefield systems that are used to acquire , transmit, process ,

disseminate , and utilize information . This increased complexity places

greater demands upon the operator interacting with the machine system .

Research in this area is focused on human performance problems related

to interactions within command and control centers as we]l as on issues

of systems development . Such research is concerned with software development , topographic products and procedures , tactical syinbology , user—

oriented systems, information management , staff operations and procedures ,

decision support , and sensor systems integr& ion and utilization.

An issue of special concern within the area of user-oriented systems

is the improvement of manual data input procedures , especially in the

Tactical Operations System (TOS) . The main source of information for

tact ical da ta systems is manual data entry-—a slow , error-prone process.

The capability of tactical data systems such as TOS to support command

staff actions with accurate , complete , and timely information is dependent on the performanc e of the person who must manually enter information

into the system . Previous ARI research on data entry has resulted in

simplified message formats , improved reference codes , and aids for online preparation and verification of message entries . Although progress

has been made , data entry remains a major system bottleneck . The research

reported here compared alternative inputting methods w ith four levels of

system aiding and prompting and has produced specific design recomtnendations for improving the speed of data entry and error rates .

Research in user-oriented systems is conducted as an in-house effort augmented through contracts. This report resulted fron an in-house

research effort responsive to requirements of Army Project 2Q 763743A774

and to special requiremeri~ s of the U.S. Army Combined Arms Combat Developtnent Activity, Fort Leavenworth , Kans . Special requirements are contam ed in Human Resource Need 76— 162 (77-295) , “Processor Aided Retrieval

and Storage . ”

~ 

JOS~EPH Z

~ T.~

D

~~ R

‘~ ep’h nical Director
A COMPARATIVE ANALY SIS OF METHOD S FOR TACT ICAL DATA INP UTTIN G

BRIEF _____________________________________

Requirement :

Nearly all information in tactical operations systems is input

manually. Two problems that arise when manually inputting data are

(a) the introduction of errors in translating information into computer

format and (b) the introduction of a bottleneck in total system response

time . Therefore , alternative methods of inputting data for accuracy and

speed should be evaluated .

Procedure :

Four methods were examined for speed accuracy when inputting tactical messages concerning enemy activity into an Army computer format.

The methods were (a) typing--the user types the appropriate codes into

a message format; (b) typing with an error corrector—-the computer automatically attempts to correct common spelling and/or typing errors; (c)

menus——the user indicates which of the legal entries is desired from a

list; and (d) typing with autocompletion and an English option--the

user must type only sufficient characters to uniquely identify the

item , using either the appropriate code or its English definition .

Thirty-two enlisted persons , representative of the class of military personnel who might serve as input device operators, input one of

four sets of nine messages using each inputting method .

Findings :

The use of menus was the most accurate inputting method . For users

of limited experience (1 day of inputting) , there were no differences in

speed among the inputting methods .

Utilization of Findings:

Consideration should be given to the adoption of menus in tactical

operation systems with a menu override option for experienced users .

The use of an error corrector , autocompletion , or an English option is

probably not warranted unless operational use shows a spec ific need for such an aid .

-

~~~~~—~~~~~~~~~—- - - - —,~~~~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-,- -- ----- -

A COMPARATIVE ANALYSIS OF METHODS FOR TACTICAL DATA INPUTTING

CONTENTS

Page

INTRODUCTION 1

OBJECTIVE 2

METHOD 2

Subjects 2

Task 3

Experimental Station 3

Inputting Methods 3

Dependent Variables 7

PROCEDURE 7

RESULTS 9

Accuracy 9

Time 10

Preferences 11

Use of Computer Aids 11

DISCUSSION 13

Menus 13

Time Data 16

Error Correc tor 16

English Option 17

CONCLUSIONS 17

REFERENCES 19

APPENDIX. STATI~

3TICAL SUMMARY TARLES 21

DISTRIBUTION 27

LIST OF TABLES

Table 1. Experimental design 8

2. Accuracy ranking of inputting methods 9

____ ___ ___ —-~~~- - —~~•--~~~~~~ --~~~~~~ ----- - -—---
- .- -

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -~~~~~~ --~~~~~~~~-~~~~~

- - - 

~~~~

- ----

~~~~~~

-- ---- --

- - - - -~~ -

LIST OF TABLF S (continued)

Table L ‘l i m e ranking of inpu tting methods . 10

4. T ime data for sessions 11

5. Agreement table between preferences and

fewest errors, by inputting method 12

6. Agreement table between preferences and

best time , by inputting method 12

7 . Use of backspacing 14

8. Use of error corrector 15

9. Autocompletion and English option 15

A-l. Analysis of variance table for error

data (32 subjects) 22

A-2. Analysis of variance table for time

data (32 subjects) 23

A—3 . Analysis of variance table for logs

of time data (32 subjects) 24

A-4 . Means and error data for four variables

and for each cell 25

A-5. Means and time data (in seconds) for four

variables and for each cell 26

LIST OF FIGURES

Figure 1. Typical free text message describing

enemy actions 4

2. Format for entering data 5
— —

A COMPARATIVE ANALYSIS OF METHODS FOR

TACTICAL DATA INPtJTTINC~

INTRODUCTION

Despite the long—term possibility of direct-sensor to computer—

data transfer , the main source of information for tactical operations

systems is manual data inputting . Two problems arise when data are entered into the system manu~ 1iy : (a) the introduction of errors in translating information into computer format and (b) the introduction of a

bottleneck in total system re~ rouse time .

The U.S. Army Research Institute for the Behavioral and Social

Sciences (ARI) has been engaged in research to improve performance of

data inputting at the man-machine interface (see Alderman (1976) for

a discussion of previous research) . Earlier research sought to improve human performance by individualized training techniques (Gade,

Fields , & Alderman , 1978), by computer prompting and instruction (Strub ,

1975), and by using on-line inputting with verification (Strub , 1971).

The earlier reseai ch , however, did not focus on actual inputting

procedures . To learn how the method of inputting affects speed and

accuracy, four different methods for entering intelligence data into

a Tactical Operating System (TOS) format were examined . The fastest

method could be used to alleviate the bottleneck at the man-machine

interface . The problem of introduction of error has two parts : (a)

the operator must form a correct concept of the information (from reading a messa ge , hearing a telephone message , viewing a radar scope , etc.)

and (b) the operator must input properly formatted information based on

a correct concept. In other words , an error could arise if the operator misunderstood the information or incorrectly input the information .

Although it is difficult to separate the two sources of error , the

method of inputting affects inputting errors rather than concept formation errors.

The first of the four methods chosen for testing was typing in

which the user types the appropriate codes into a message format. Typing is a common method of inputting and is used in most versions of the

Army ’s developing Tactical Operations System and previous ARI research .

The second inputting method was also typing , except that a computer

automatically attempted to correct common spelling and typing errors .

The computer should be able to make corrections faster than the human

user and thus speed up the rste of inputting .

A third inputting method used menus , in which the user indicates

which of the leqal entries listed is desired . A track ball moving a

cursor was used to indicate the entry . This method is a common alter—

native to typing ; it requires less training and the user cannot make

1

- --a - ~~~~~~
__

spelling errors. Also , because the use of menus places i smaller cognitive load on the user (requiring recognition memory rather than recall

memory),1 there is also the possibility that fewer errors will occur.

The fourth method of ~nputting was aimed especially at speeding up

the rate of inputting . This method , which used typing as its base, required t~~ t the user type only enough characters to identify a member

of the table of legal entries for that item of the format. The computer

would fill in the rest of the entry , thus saving typing time . Because

research has sh~~ x i that typing English is faster than typing codes , the

user of this method was allowed to use either the code or its English

definition. In either case , the participant was to use only the minimum number of characters needed to identify the code or definition .

The different operating characteristics of the four methods provide a basis for predicting relative inputting performance. Thus , corn—

pared with normal typing , typing with an error corrector should result

in fewer errors as well as a time savings. Similarly , the use of menus

should result in fewer errors and possibly a time savings. However , the

fastest and most accurate inputting method should be autocompletion .

OBJECTIVE

The purpose of this research was to evaluate alternative methods of

inputting enemy situation data for speed , accuracy, and ease of use.

METHOD

Subjects

Thirty—two enlisted men and women were randomly assigned to four

groups for the experiment . Participants possessed a GT score of 105 or

above , vision correc table to 20/20, and were representative of the class

of military personnel who might be trained to serve as input device operators. Although 26 of the 32 participants claimed some typing skill , and

several had had limited exposure to computers , none had used the inputting

techniques being evaluated in the experiment .

1

References to this phenomenon are common . For examples, see Loftus &

Loftus, 1976, pp. 56— 91 , or Baddeley, 1976, pp. 285—286 .

2
~~ -~~ --~~~~~~~~~-- ~~~~~~~~~ -

~~~~~~~~ -~~~~~~~~~~~~~~ ~~~~~~~~~~ - - - -

Task

The participants acted as intelligence staff members entering information about the enemy into a computerized data base. They received

free text messages describing enemy actions . An example appears in

Figure 1. Their job was to extract the necessary information and enter

the appropriate codes into a variation of the Enemy Situation Data Add

(ESDA) format of the TOS system (see Figure 2).

Experimental Station

The experiment was conducted in an area containing a table and chair

for the participant. Two cathode ray tube (CRT) displays sat on the

table , one showing the format into which the participant would enter

data and the other containing the menus for the menu-selection inputting

method. A track ball for the menu—selection method was placed in front

of that CRT. A keyboard was placed in front of the format CRT. A dictionary was available that contained the valid codes and their English

definitions cross—indexed to the items in the format. An intercom was

provided for calling the experimenter .

Inputting Methods

Four different inputting methods were investigated in this experiment. Each method used automatic tabbing ; that is, after the completion

of an entry, the computer automatically moved the cursor to the next entry item in the format. The computer did not accept invalid codes2 for

an entry and gave the user an error message to that effect.

I. Typing. In this method , the message format appeared on the

screen with the cursor placed at the first item . The users then filled

in the appropriate codes (inciuding blanks) for each item , referring to

the dictionary of valid codes and their definitions as needed . The computer rejected any inpu .. that was not a legal entry and would not allow

the user to continue until a valid code was entered . Participants could

backspace and type over their answers. The cursor automatically moved

to the next item in the message format when a valid entry had been typed .

2

An invalid code is one that does not appear in the valid code list for

an entry . The terms “valid” and “l egal” are used interchangeably . A

valid code may be incorrect if it appears in the val id code list but does

not, correctly describe the information in the free text message.

3 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SPOT REPORT

PRECEDENCE : FLASH - IMMEDIATE - PRIORITY - ROUTINE

DISPOSITION : GI G3OPS G4 G5 FSE TASE

ENGR 1BDE 2BDE 3BDE

FROM: 1st Brigade, Ground Surveillance Section

REPORT TIME : 1745

EVENT TIME: 1630

CLASSIFICATION : C S TS

DESCRIPTION : Convoy moving SW along Route #15

7 tanks , 22 trucks, 52 APC’s picked up by unattended ground sensors vic. XT437262

MESSAGE NUMBER - 45618

riqure 1 . Typical free text message describing enemy actions .

4

- ~~~~~~~~~~~~~~~~~~~~~~ .
- .

_..- --—

ENEMY SITUATION DATA ADD - ESDA

ORIG/NO :[ / I SCTY [ ] PRES[ ] RESTR [

RECD-FR [ ] REPT-ID [ I REL-TO [ I

AGENCY [ I SOURCE-ID [

• EVENT-TIME[ ] NATION [ I

EN-UNIT [ / / / ]EN-.PARENT [ / / / I

--SUBJ---ACTV---QUANT-DESCRIPTION LOCATIONl( / / / / )

2( / / / /

3( / / / /

4( / / / /

-

‘ Sc / / / /

REMARKS [

DISTR [ , , , , ] INTEREST[

Figure 2. Format for entering data.

5

— -- -. - ,. — ~~~~~~ . .

~~~~ -

~~~~ •~~~~~~ -~~ ~~~~~~~~~~~~~~~~~~~
II. Typing with Error Correction . This method was the same as

Method I except that an aid had beex : adcle1--a typing error corrector .

When the user typed an invalid entry , the system assumed that a typing

error had been made and attempted to correc t the error . The typing

error corrector dealt with four kinds of errors : transposition of two

adjacent letters , insertion of an extra letter , deletion of one letter ,

or substitution of 01! ? incorrect lettor. When the typed item was an

invalid entry , the typing corrector tried to form a hypothesis of ~hat

the entry should be. For each of the four kind s of errors , th~ typinq

corrector found a i.ist o~ all the members of the valid lirt LIOL WIe

either the same length as ~he inval id entry , one character shorter, or

one character longer , ~epe~ di n~

i on which of the four types of typirg

errors was being tested . Them: it went through one of four processes:

1. The typing corrector scanned the target word and the entry from

the left for mismatches. At the first mismatch , it transposed the mismatched letter in the entry with the letter to the right. If the new

word matched the target word , the target word was the hypothesis. If

there was no match, the typing corrector went on to the next target word .

2. The typing corrector scanned the target word and the entry from

the left for mismatches. it deleted the mismatched letter in the entry

and checked for a match . If there was no match , the typing correc tor

went on to the next target word .

3. The typing corrector scanned from the left for misniatches and

deleted the mismatched letter from the target word and checked for a

match. If there was no match , it went on to the next target word .

4. The typing corrector scanned from the left for mismatches. At

the first mismatch, it skipped the mismatched letters in both words and

checked the rest for a match . If there was no match , the typing corrector went on to the next target word .

The first hypothes~ ’ formed when the typing corrector found a match

printed out on the screen with an error message. If it was the entry

the user actually wanted , the user hit a key to signal acceptance; otherwise the user could hit a key to signal a retyping of the entry . If the

typing corrector could not form a hypothesis , an error message appeared

on the screen and the user had to retype the entry .

III. Menus. In this input method , typing was used for entering

three types of items in the message format: map coordinates , dates , and

cardinal numbers . All other data had to be entered by selecting the appropriate item from an alphabetically or logically ordered menu ; data

could not be typed in. The message format appeared onm the right CRT with

a cursor at the first item . The menu of l egal entries for ‘

~~,e first item

appeared on the left CRT in their unccded form (i.e., English definitions

from the dictionary) . If a blank was a valid entry , it was included in

the menu . When an item was chosen , its code was filled into the message

6

- ~~~~~--

~~-~~~
F

~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~

~

-. —— —-.—--—~~---,“—..- -~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~

format by the computer and the cursor moved to the next item . If an item

had to be typed rather than chosen from a menu (e.g., map coordinate

string), instructions to type the item would be displayed where the menu

would normally appear. In these cases, invalid entries (e.g., insufficient characters in the coordinate string) were rejected and the next

item could not be entered until a valid entry had been typed in , as in

Method I.

IV. Typing with Autocompletion and English Option. This method

was the same as Method I, but with two additions . First , the participant could enter the English definition from the dictionary in place of

the code , as desired . Second , when the participant felt that enough

characters had been typed to identify a member of the valid entry list

(either code or English) , he would push the send button , causing the

autocompletion program to take over . If indeed the program could match

the characters with the beginning of one, and onl y one , member of the

valid entry list , it would automatically finish the entry for the participant and move the cursor to the next entry . If a unique match could

not be made, the program asked for more characters . The participant

could invoke the autocompletion program at any time and as often as

wished until a unique match was made .

Dependent Variables

The primary dependent variables were format completion time and

accuracy in terms of number of errors per format. Typing errors as wel l

as errors of interpretation and errors specific to a particular inputting

method were included in evaluating the number of errors per format.

Other data collected included the number of times the typing corrector was used , the number of times the typing corrector correctly identified the target code , the number of times autocompletion was used , the

number of times English definitions were used in place of codes , the

number of times the participant correc ted an entry by backspacing and

typing over before ent

~ i ing the item , and the participant’s stated

preferences among the input methods . All the participants ’ entries

and the entry times were recorded as well.

PROCEDURE

Thirty—six free text messages describing enemy actions were divided

into four sets of nine messages with each message set (M) balanced for

types of subjects , sources , restrictions , unit identifications , and

difficulty during the pilot testing . For each message set, the first

message was always a practice message, and the other eight mes sages

made up the experimental set. In addition to these 36 messages , 2 other

messages were initially presented as practice messages .

7

~

—- ----- - ..-- ~~~. ~~~~~~~~~~~~~ -. - - --~~~~~~ . - - --—-.. ~~~ . - - s
_______

RE SULTS

Analysis of variance summary tables and means for the dependent

variables in this experiment can be found in the appendix . Only major

findings are given here .

Accuracy

Acc uracy of an inputting method may be considered from two perspectives : (a) mean errors per message and (b) the number of participants

who had the fewest errors using a particular method . For both measures,

the rank order of inputting methods was the same (see Table 2).

Table 2

Accuracy Ranking of Inputting Methods

Par ticipants who made

Mean number of the fewest errors

Method errors per message using this method

Menus 2.64 17

Typing with error a corrector 3.36 8.5

a • Typ].ng 3.77 3.5

Typing with

autocompietion 4.39 3

a

lncludes one tie between typing and typing with error corrector.

An analysis of variance of the mean error data indicates that the

differences among the inputting methods are significant (F 13.98, df 3,84, p < .001). There were no statistically significant residual

interactions between inputting methods and message set , sessions, or

groups . The analysis of variance table (Table A-i) and a discussion

of other significant findings is provided in the appendix.

9

____________________________ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ,_ - - ___,

~~_ -
-_____

Time

The mean time per message using each of the four inputting methods

and number of participants who had their fastest scores using a given

inputting method are shown in Table 3.

Table 3

Time Ranking of Inpu tting Methods

Number of participants who

Mean time per averaged the fastest time

Method message (in seconds) using this method

Typing with

autocompletion 413.13 13

Typing with error

corrector 397.01 9

Typing 396.05 7

Menus 396.52 3

Although scoring by number of participants (who averaged their bes t

times over the eight messages using a given inputting method) is consistent with the original predic tions (see the discussion of inputting methods

in the Introduc tion) , the mean time ranking is almost the exact reverse .

An analysis of variance per foimed on both the raw time scores and the

logs of the time scores (used because of skew in the time scores) showed

that the differences among inputting methods in mean times were statistically insignificant. (The analysis of variance summary tables can be

found in Tables A-2 and A-3 of the appendix.) The most significant factor

in the analysis of variance of the time data was Sessions . The more practice a participant had , the shor ter the inputting time became (P < .001).

This can be seen in the mean times across all inputting methods for each

session , shown in Table 4.

I ()
Table 4

Time Data for Sessions

Session Mean time (in seconds)

1. 531.11

2 387.12

3 360.21

4 324.27

Preferences

As the participants were debriefed , they were asked about their

preferences among inputting methods . It was hypothesized that people

would prefer a method with which they had performed well--either in

making the fewest errors or in averaging the fastest time . The prefer— 3 ence results can be seen in Tables 5 and 6. A coefficient of agreement

(K) and an approximation of the standard error (eK) were determined for

each table. (See Cohen , 1960, for a discussion of K.) Contrary to expectations , there is chance agreement or , at best, a slight negative

agreement between preference and best performance. For fewest errors ,

K = -.108 and (1K .149. For fastest time , IC = -.128 and OK = .057.

Use of Computer Aids

Backspacing . Wht ver a participant was typing an entry , the back—

spacing option was available. This option allowed the subject to back up

and type over an answer to change it. The use of the backspace option

allowed the participant to catch arid correct mistakes before the computer

the coefficient of agreement , is a measure of the agreement between

two variables measured by nomin ai scales (in this case, best performance

and preference) . Like the Pearson product-moment correlation coefficient , r , K can take on values oni y between -l and +1, A score of +1 would

indicate total agreement between periormance and preference, and a score of -1 would ind icate that users never preferred the inputting method they

did best on. A score of 0 would indirat , no predictive linear relation—

ship between prcfurc:ct~ and performance.
Table 5

Agreement Table Between Preferences and

Fewest Errors, by Inputting Method

Preference

Fewest errors Typing Error corrector Menus Autocompletion Total

Typing o 0 2 0 2

Error

correc tor 2 0 5 1 8

Menus 1 3 10 3 17

AUtocompletion 1 0 1 0 2

Total 4 3 18 4 29a

a

Three subjects did not express preferences .

Table 6

Agreement Table Between Preferences and

Best Time , by Inputting Method

Preference

Best time Typing Error corrector Menus Autocornpletion Total

Typing o 0 5 0 5

Error

corrector i 1 5 2 9

Menus o 2 0 1 3

Autocontpletion 3 0 8 1 12

Total 4 3 18 4 29

a

a

Three subjects did not express preferences .

12

Ip__ —~ - ----—-— —_-— - - —-.—----- —. —

rn-- — ——.~~~~

-

~~~~~~~~--— - - — -- -. -— - -— ——
would catch them . This option was frequently used by the participant ,

as shown in Table 7. The backspacing option could be used with three

results : (a) a correct entry , (b) a valid (correctly spelled) entry

that was not the correct entry for the message , or (c) an invalid entry .

Error Corrector. Table 8 shows the use of the error corrector .

The error corrector could change an invalid entry to a val id and correct entry , and could change an invalid entry to a valid and correct

entry but the participant could reject that correct answer . In about

a third of the cases the error corrector could not arrive at a valid

entry .

Au tocompletion with English Option. Table 9 shows the use made of

the autocompletion option an-~ the English option . There were six forms

these options could take . The subject could use the full English definition, and that definition could be either correct or incorrect. The

subject could use autocompletion on either the code or the English defi

nition and these autocompleted entries could be either correct or

incorrect.

DISCUSS ION

Menus

Menus appear to be more error-free than the other inputting methods

examined (see Table 2). Indeed , there is a 40% decrease in error with

menus from the wors t case , typing with autocompletion . Menus are popular ; 18 out of 29 participants expressed a preference for them. Finally,

menus do not have higher mean times than the other inputting methods.

Therefore it would seem reasonable to suggest that the use of menus be

seriously cons idered for adoption for use in tactical data inputting.

This study considered menu selection with a track ball. However ,

the track ball method of menu selection does not seem to be a good method

of inputting . It is slow and cumbersome to use. There are several other

methods of a menu selection (e.g., liqhtpen , touch-sensitive panels, a

typed index cod e or letter) that should be considered . Each method has

advantages and disadvantages ; the results of this research do not suggest

which one would be the best for the tactical data inputting task .

None of the menus in this research was particularly long . Most fit

on one screen (i.e., they were less than 40 items long). The longer

menus (e.g., “subject”) were divided into sections (e.g., “personnel,”

“vehicle,” etc.). The first screen shown ~he participant was a list

of sections from which the subject could select a screen—sized menu .

Even with the relatively fast speed of the experimental displays and

the relatively short length of the menus , participants still had to

wait for a menu to be printed out so that they could respond . With

slower equipment, longer menus , or more experienced users, delays due

- —- ---- —— -~~~~~~~~~~~~ -~~ -— —~~~~~- _ I.

~~

_

~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~—- -

~~~—~~ --_.- .~~~~ ~~ --- -

a) 4J

0

~ 0

~~~ 4 a) N ~

n t

~

n ,..

~ ~~~ , C (‘~ U ‘

~~ 0

‘-I -

~ -~ in ‘C

I W I

~~~~~~~~ ‘i-i o ~

~~~~ a)

~

4 C O 0 0 N in (N w W ~~-~ ~~ .0 ~~‘ ‘-4 ~~‘ ‘

.0 .~~~a )o’u~ -4 -4 s 8 ~ ~1 ..4 ~~

~~~~~~~~~~~~

a) 4J

~ U 0’ in ‘.0 l

~~~~~ a) ‘.0 N in 0’ 0’ m -~ U~ u

‘-I ‘C U)

U) U)- 4

~ a)i~ 0 5 E ~ a)

Ii 5

S 5 0 0 4.~ .,.f C) .4.) tt~ (N In 0’ ‘-4 0 00 ~~~U ‘-4 -4 -4 In

N U) II) .

)

~ .

~~ .~~~‘~~ ‘-4 O ~ l () E0WWU .Q l~

0

a) 4) a) O~ U U) ~~~ 4w 0~ ~~ ‘

.0 .-4

~~~~~~~ C’) 0 (N ,-4 0’

‘

.0 ~ C’) in CM ‘C U) r’i

U) U) 4)

a)~~~~~ C)

4)

C O

0) LI) Lfl m

00 o in 0 ‘.0 c’) (N (‘.4 ,..4 ,..4 N

4) U)

~~~~~~~~

~ o Q) 4J

S a)

o

4.) 0 0

a) .4.)

.C~~ 4 .C-4 4 ) 0

~~ U)

14

- - - - ~~~~~~-~~~ -~~~~~~~~~~~~~~~~~ -rn -- —~~ ~~~~~~~~~~~~~~~~ _ _-

~~ - ~~~~~~~~~~~~ 

-

~~~ 

---

~~~

--

~~ ~~~~~~~~~~~~~~~~~~~~~~~~
- 

I

a) 4)C) 1644W 0’ 

I

LI) 

0

4) 4.

4 (N

‘C 

U) (6

C 4 ) 0

C) ._I

(6’

a) 

~4 W ( n ~~ (6’

a )0’

~

I 

0 04 —4 84)44

44 44

tl.4~

~ .4 4J

4J 

N

0005 t 

C 055 0 44 4.)

.00 —4 (6 5.5 8445 ,..4 (fl I 04 .,.4 UI (N Z W046 4)E-~ m 1 O 0 ~~~~ ‘-4

44

4.)

0 59.1 0 ( 6 0

0S

(6 

5 I o I 0 04

6) .,.4 4.) 

0

4.) a) (11 0 04 4J -4 CO a) 44 ‘ 644W O 44q )

~~~~ 

0 0 04 ‘.0 (1)

5 8 6) N • a)

.4.) ~~~~~~~ .5 4i0

’~ (N .5 C) C) U) 4 4 00 I a) ‘

C (14 •‘-4 000 4.4 Q4J a) 4.4 Cl) CO 0 0) 5 H ( 6 0 S

o a) (1444 >, 

~ a) 

0 .4 44 Q ) 0 ’-4 .4 ‘ti C 4)

.f

~ 44 ..4 C) () (43 

.,.4 U) 0 I ‘

~~ 5 

0

44 4)Wa ) a) 4444 5 43) ...4

9-44444 0 0 .-4 4)

000’~ .-‘. .4 04 

0) 

m O 0 0 W 4.) 0 II) 

C 44 54) 6) C)O ’ti a) • S a) a) 44 ., 4 () .-4 f W OO - ‘.0 4.)

U) .00 5 04 4 40 0 In 44 E 4 4t O -r~ 8 0594

4.4 4.) 

Z Q

544 (65 

0

a

~~~~

44 a 0443 0 04

0 0

4)

S .5 

‘

.0

‘C 

0

4) -4 O e~ 

‘

w ~~

~ (6 In N .5

440 —4 (6

I 0 ‘-4 • 4)

(J6J ,-4 

0

II) 43) 4) I 4~4 5 9 ~ •r

l

a.’ 

t~ J 

,Q (6445 LI) S (N 43) ((4

Cl) (‘1 5 .5 ‘C U) C)

(43

0.-l W (6’ 43)

(6’ 449.4,..4 (6’

44 0.’ ,-l 43)

5 0 

O ’44 5 0.’ oor ~ (43

4) 5 4) 43

) 440 I (6 0044’ti (6’ 0 0 446 ) 0 5 43) 4) 43) 44 04) ‘-I a) 0.’ 0 .5 ~~ 440 04 5 (6445 4) O a) Z a )44(6 0 I 44~~ 8 ~~~ (6 (14 (43

- 15 -~~~~ — — ---

-

~~~~ 

-
to either paging through a menu or waiting until the menu was printed

out could be a serious defect. Therefor . either a menu override option should be made available for experienced users , or careful consideration should be given to the design of an operational menu system

taking these point~

s into account. Otherwise a poorly des igned menu

sys tem could negate the benefits found in this research .

Time Data

The time data from this experiment suggest that the participants

never reached a plateau where pract ic€ effects no longer affec ted perrormance . If such a plateau had been reached , perhaps time differences

among the inputting groups ~-

~~1d have emerged .

Would autocomp letion , for example , have been faster if the subjects

had had more experience? It is impossible to say . Autocompletion was

the fastest inputting method for 40% of the participants , or 13 people.

An a priori estimate assuming no differences among inputting methods

would lead us to expect each inputting method to be fastest for 25% of

the participants . The mean time of those 13 people , 308.23 seconds, however , was faster than any other group , and their use of the autocompletion option was higher than the total group average . This mean inputting

time was also faster than the average for any of the four sessions . Re—

call that the analysis of variance on sessions produced the most signif icant results , indicating a training effec t (the fastest session was the

last one , the four th , and the mean was 324.27) .

Many subjects complained that autocompletion was confusing and difficult to understand . This confusion may have led to the high error

score and wide variance seen in the autocompletion data . It would seem

that autocompletion is a useful tool only for a sophisticated user .

Error Correc tor

The computer spelling error corrector was impressive to watch in

action; it came up with correct hypotheses in most cases (i.e., a hypothesis acceptable to the user) and decreased errors by 11% from typing

without the error corrector . Yet the practical value (less than .5

fewer mistakes per message) does not seem valuable. The error corrector should probably be considered only if operational use of a tactical

inputting system shows a disturbingly large number of typing and spelling

errors .

16

__ -L . - ~~~~~~~~ .-- -
— --~

.4-~

— —--- --—— -

~~~~

English Option

The option to use English was ziot as popular as the option to USe

codes (see Table 9) . The English option was used most often where the

code and English definition were most dissimilar. In inputting the

precedence codes (R = Routine , P = Priority , I = Immedia te , and Z -

_

Flash) for example, F with autocompletion was often used rather than Z.

This unpopularity may be because the English required many more characters to be typed , on the average , than the code and also required more

characters to uniquely identify the item . An English ontion is :ro~ - 

y

not very useful, particularly with well-designed , user—oriented i-uuo~~.

CONCLUS IONS

Menus are recommended for use in inputting tactical data because

they appear to cause fewer inputting errors without noticeably affect—

ing input rate than the other methods examined in this research . However , to maximize the benefits of the menus , special attention should

be given to the design of an operational menu—based inputting system--

particularly the length of menus , the speed with which they can be displayed , and the manner in which items are selected from a menu .

Spelling correctors or autocompletion should be considered only

for experienced users and in operational settings that spec ifically

require such aids .

17
REFERENCES

Alderman, I. N. Tactical Data Inputting : Research in Operator Performance and Training. Proceedings of the Second National Symposium

on Management of Data Elements in Information Processing . Washington, D.C.: NBS/ANSI, 1976.

Baddeley , A. D. The Psychology of Memory . New ‘!3r.’

~: nasic Books , 197E .

Cohen , J. A Coefficient of Agreement for Nominal Scale’ . Ecu’ust~ onal

and Psychological Measurement. 1960, 20W, 37--46.

Gade , P. A., Fields, A. F., & Alderman , I. N. Selective Feedback as a

Training Aid to On-Line Tactical Data Inputting . ARI Technical

Paper 349 , 1978.

Loftus, G. R., & Loftus , E. F., Human Memory : The Processing of Information. Hillsdale. N.J .: Lawrence Erlbaum Associates, 1976 .

Strub, M . H. Evaluation of Man-Computer Input Techniques for Military

Information Systems . ARI Technical Research Note 226 , May 1971.

(AD 730315)

Strub, M. H. Automated Aids to On-Line Tactical Data Inputting . ARI

Technical Paper 262, February 1975. (AD 010350)

Winer, B. J• Statistical_Principles i.n Experimental Design , 2nd ed.

New York : McGraw Hill , 1975 , pp . 719-721.

19

_____ 

~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~ -
--- --,—---- -

~~~~

. 

~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~

APPENDIX

STATISTICAL SUMMARY TABLES

Table A-l contains the analysis of variance summary table for

the error data; Table A—4 contains the means and error data for each of

the four variables and for each cell .

Table A—2 contains a summary of the analysis of variance for time

data . Several outliers were found in the data (e.j., some subjects had

never used a typewriter keyboard before and literally had to “hunt and

peck” for each key). Because the presence of outliers can unduly influence an analysis of variance , a second analysis of variance was run

using logs of the time score The results of the second analysis are

summarized in Table A-3. Table A-S contains the means and time data

for each of the four variables and for each cell.

DISCUSSION

Error Data

Message sets (M ) are a significant source of var iance as can be

seen in Tables A—l and A-4. Groups (G) and sessions (S) are also somewhat different . However , since the residual (or interaction) term is

not significant and since the experimental design is a Greco-Latin

square with within-subject measures , the se fac tors do not affec t the

main conclusion concerning inputting methods .

Time Data

Use of the log transform to reduce the skew in the time data reduced the size of the residual term (Tables A— 2 and A—3) . However , the

ordering of the other terms remained approximately the same . In both

analyses , inputting methods (I) are not significant and sessions (S) are

very significant (p < .001) .

A mistake in numbering two messages put a short message from set 4

into set 3 and a long message from set 3 into set 4. Therefore , message — 

set 3 is about eight items too short and message set 4 is eight items

too long . This probably accounts for the difference in means between

them (see Table A—5) .

21

--- - —- —~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~—-4——. -.
ijIj Aj ~

i ~ ~~~~~~~~~~~~~~~~~~~~ :

~~~~ ., r

~

-

~

T

Table A-i

Analysis of Variance Table for Error Data (32 Subjects)

Source

of variation 55 df MS F

Between subjects 31

Groups (G) 114.38 3 38.13 3•79*

Subjects/groups = e

1 281.51 28 10.05

Within subjects 96

Message sets CM) 34.20 3 11.40 9.34***

Inputting methods (I) 51.18 3 17.06 13.98***

Sessions CS) 13.09 3 4.36 3~ 57*

Residual 1.76 3 0.59 0.48

Subjects/groups = e

2 102.09 84 1.22

Total 598.21 127

* p < .05.

< .001.

22
- ____________

Table A-2

Analysis of Variance Table for Time Data (32 Subjects)

Source

of variation ss df MS F

Between subjects 31

Groups (G) 314557.95 3 104852.65 4.86**

Subjects/groups = e

1 

603568.31 28 21556.01

Within subjects 96 -

Message sets CM) 87759.42 3 29253.14 12.23***

Inputting methods (I) 6631.30 3 2210.43 .92

Sessions 789543.52 3 263181.17 l09.92***

Residual 41975.21 3 13991.74 5.85**

Subjects/groups = e

2 200989 .96 84 2392.74

Total 2045025.66 127

** p < .01 .

***p <‘ .001 .

23

_____ _____ _____
Table A-3

Analysis of Variance Table for Logs of Time Data (32 Subjects)

Source

of variation SS df MS F

Between subjects 31

Groups (G) 313915.66 3 104638.55 4.22*

Subjects/groups = e1 694914.81 28 24818.39

Wi thin subjects 96

Message sets (M) 64248.09 3 21416.03 lO.0l***

Inputting methods CI) 1440.03 3 480.01 .22

Sessions CS) 781955.53 - 

3 260651.84 l2l.81***

Residual 23799.95 3 7933.32 3.71*

Subjec ts/groups = e

2 179741.9 84 2139.8

Total 2060015.97 127

* p < .05. -

***p < .001.

24

I -- —----- — ----~~~~~~~~-~~~~~~~~~~~~~~~ ---~~~~~~~~~~ - - —---
0) 

~.D it~ -~ a’ a’

(N 

C’)

3)

a)

a)~~ ‘-4 (N C’)

I U) U)

a)

I;

C N ~~‘ a’ 4)

Id N C’) C)

—. a) . . . . W z r’) C’) (N ~~. Ii—

I 

04) ‘a - o o w 0 114 4.) U)

-4 ~~ o 14 r.4 4) Id O

W a) a) C) C 1 40 ’ C) . c w . C O 14Id 4)14 4J .,.4

.e a’ .,.4 14 

.~.4 4) U)

.

4) ~~~~~ 0’ 0’ 4) C Cli U) C E 14 5 .,4 .,4 ()

o O

~~

14 C 04 C) a’ 4.4 5 a) H .~~~

U)

Id 

4 )14 a) a’ ‘-4 a) U) a’ ‘

.D ‘~

~‘ II U .~I . -~ 4) It) (N (N m IN

C

Id C Ii U ii ii •

Id

14 a) Ix Ix Ix Id .I

~ > Q40

a) 14 4J U)

.Q 0 

t.0 Lfl &fl 11W Id ~~‘ N (0 01 LI)

14 

~~ ‘ 

‘

.0 (N .-i ,-i 04 m m

ii 4 — 

C Id I

~~ H O 4.)

Id 4)

a’ 

~~ ‘ 

0 ,-I

14 ‘.0 0 U) 

‘

.0 o . . . . . 4) 14 — ~~~ ~~ ‘ ‘ m m (N (N .4 m a) 14 U) 

— - - - 

- ii m ii ~~ ‘ ii ‘-4 ii oi ii 

0 0’4J ‘a U) x I x C C ..-4 Id Id o

U) m 

LI) a’ (0 54..) C U) 0 0) N ~

.4 Id . , . . 

. a) (I) (N C’) (N ~~‘ (N .-l (N (N (f)

• — 

a’ ii (N ii ,-4 ii ~~‘ ii (N ii a)C .C .4 U.: >‘ >4 Ix

(fl 4J .,.4

a’ a’ ,-l Ii

(N a’ U) (N C) Ii Id . . . . It) ,-I ~~ 01 ,-

~ m ~~~ 

.9 4 ii .

~~ CId >4 >4 >4 >4

-~ 4 )

-

C, a) 

m U ) — E 115

U) 4) 14.4 0) “4 E o ,~~~H 14 0) Id m

— 

25 

4

___

----—--

_

~- --— ----‘~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

DISTRIBUTION

AR t Distr ibutio n I -y

4 OASD IM&HA I 2 HQUSA CDEC . Ft Ord . ATTN Libr ary

2 HQDA IOAM I-CSZ I 1 HQUSACOEC , Ft Ord . ATTN ATEC- EX -E—Hum Factor )

1 HOD A IDA PE-P BR 2 USAFEC , Ft Benjamin Harrison . ATTN: Library

1 HODA IDAMA-ARI 1 USAPACDC Ft 5p~i

~arn ’ Harrison , ATT N: ATCP—H R

I IIODA IDAPIr lI RE P0) 1 USA Comm E)

~ct Scli . F, Mor,rnoiitb ATTN: ATSN - -EA

1 HODA ‘,c~HD - lDi 1 I)SAEC . Ft Monrno u: ATTN AMS FL -CT HOP

Hi

~~~A ‘A M- (tOT Cl 1 USAFC , Ft Mn,irnr i~

tb , ~ ‘T\ ,rMSt L- PA P 

~

LiA iOAP C 1ii

~

l ! A l 1 USAEC Ft M’ir.moi’- AT ~ . AM SI L SI 1

~fJ

1 lOLA LA L’4 4*’I A) 1 USAEC , Ft Mo mpiitl ATT ’iI C I

- . - I)-’ . L’

1 ‘IODA IDAP~ HEl l 1 USA Mate riah S~

o An al Ary.v Abe ’ “- - At ~N AMXSY -- P

1 oi)l)A DAFt ~~~ ~) I Erfge wood Arse nal Ab.’ril,’i ri A~ T N- ~A R EA- BL H

I I-IODA II’)API -OW) 1 USA 0~ I Ctr & Sr~r

~, Ahirr rl,’en , A T E N AT SL—TEM--C

1 HODA (DA PE- I4RLI 2 USA Hum Engr La), . Abc rdei’r, , ATTN Libr ary/ Dir

1 HODA )DAPE-CPS ) I USA Combat Arm) Tng Sd , Ft Benninij, ATTN: Ad Supervisor

1 HOOA DAF[) MI-A l 1 USA Infantry Hum Rsch Unit . Ft Ben riu ng , ATTN: Chief

1 HODA IDARD-A RS - PI I USA Infantry Sd, Ft Banning, ATTN: STEBC—TE--T

1 HODA IDAPC PA( Al 1 USASMA , Ft Bliss , ATT N ATSE - L RC

1 HODA )DUSA - OR) 1 USA Air Del Sc!,, Ft Bl iss , A1’TN: ATSA - -CTD - ME

1 HODA IDAMO-RO RI 1 USA Air Oaf 5db , Ft Bits. , ATTN: Ted , Lib

1 HODA DASGI 1 USA Air Del Bd , Ft Bliss , AT ’TN: FILES

1 I-4 00A DA1O-Pl) 1 USA Air Del Sd , Ft Bliss , ATTN : STEBO—PO

1 Ch,.) . Ciu’rsu)t Div lDA - OTSGI . Adeiph u , MD 1 USA Cmd & Gener a) Stf College . Ft Leavenworth , ATTN~ Lib

1 Mi) Asst Hun u Res , 000El& E, DAD (E&LS) 1 USA Cmd & General Stf College , Ft Leavenworth , ATTN: ATSW—SE—L

1 HO USARAL , A PO Seattle , ATTN ARAGP 5 1 USA Cmd & General Set CoI!e ~e , Ft Leavenworth , ATTN: Ed Advisor

1 HO F.r ~t Army . ATTN AFKA - Ol It 1 USA Combined Arms Crnht Dee Act , Ft Leave nworth , ATTN: DepCdr

2 HO Fifth A u my Ft Sam Houston USA Comb ined Arms Cmbt Dee Act . Ft Leavenworth , ATTN: CCS

1 Dir , Army Stf St u rt ies Ofc . ATTN OAVCSA (DSP! 1 USA Combined Arms Cmbt Dee Act , Ft Leavenworth , ATTN: ATCASA

1 Of,. Chief of Si ’, Studi es Ofc 1 USA Combined Arms Cmbt Dee Act , Ft Leavenworth , AT’l’N: ATCACO—E

1 DCSPER , ATTN CPS/OCP I USA Combined Arms Crnht Dee Act , Ft Leavenworth , ATTN: ATCACC—CI

I The Army Liti, Pa,rta~~.,, ATTN; A SS Chief I USAECOM . Night Vision Lab , F’ Bel vo ir , ATTN: AMSE L—NV—SD

I The Army Lib , Pentagon , At TN- A NRAL 3 USA Computer Sys Cmd , Ft Belen i, ATTN: Tech Library

1 Ofc . Asst Sect of the Army (R&D) 1 USAJeIERDC, Ft Belvoir , ATTN: STSFB—DQ

I Tech Support Ofc , ojcs I USA Eng Sch , Ft Belvo ur , ATTN: Library

1 USASA , Arlington , ATIN: IARD -T 1 USA Topograph ic Lab , Ft Belvoir , ATTN: ETL- TO—S

1 USA RscP, Otc , Durham . ATTN: Life Sciences Dir I USA Topogr aphic Lab , Pt Belvoir , ATTN: STINFO Center

2 USARIEM , Natick , ATTN. SGRD-UE - CA 1 USA Topographic Lab . Ft Belvoir , ATTN: ETL—GSL

I USATTC , Ft Clayton . ATTN. STETC-MO-A 1 USA Intelligence Ctr & Sch , Ft Huachu r,a , ATTN: CTD— MS

1 USA IMA , Ft Bragg, ATT N ATSU -CT D-OM 1 USA Intelligence Ctr B. Sch, Ft Huachuca , ATTN: ATS—CTD—MS

1 (JSAIMA , Ft Bragg, ATT N: Merguual Lb 1 USA lrs te ll ic ience Ctr & Sch , Ft Huachuca . ATTN: ATSI—TE

1 US WAC Cti & 5th. Ft McClellan , ATTN: Lilt I USA tntelliaeiuce Ctr & Set , Ft Huachuce , ATTN: AT SI—TEX—GS

1 US WAG CIr & Sch , Ft McCle llan , ATTN: Tnq Dir 1 USA Intelligence Ctr & Sch , Ft Huachuca . ATTN: ATS I—CTS --OR

1 USA Ouart ermaste, Sets , Ft Lee , ATTN- ATSM TB I USA Intel ligence Ctr & Set , Ft Huachuc a , ATTN: ATSI CTD DT

1 t nt ellu qenc e Material Dcv Ofc , EWL . Ft Hoi alt,i d 1 USA Int eil iqenos Ctr & Set , Ft Huachuca , ATTN : ATSI—CTD—CS

1 USA SE Signal Sch , Ft Gordon , ATTN ATSO- BA 1 USA Intelli gence Ct, & Set , Ft Huachuca , ATTN~ DAS/SRD

1 USA Chair ta un Ctr & Sch , Ft Hv n rj lto n , ATTN ATSC -TE-RD 1 USA Intellige nce Ctr & Sets , Ft Huachuca , ATTN : ATSI TEM

I USA TSCH . F (into , ATTN: F-due Advisor 1 USA Intelligence Ctr & Sch . Ft Huechuca , ATTN : Library

1 USA War Coll ege , Carli sle Barracks , ATTN: Lib 1 COR , HO Ft Huachuca . ATTN: Tech Ref Div

2 WRAI R . Neii rnpsyc hi atny Ore 2 CDR . USA Electronic Prvg Grd , ATTN : STEEP—MT—S

I DLI , SDA . Monterey 1 COR , Projec t MASSTFR . ATTN: Tech Info Center

1 USA Concept Anal Aq cy, Bet l i,’sd a, ATTN : MOCA-WOC 1 Hg MASSTER , USATRADOC , LNO

I USA Concept Anal Aqcy. h,rth.’sda. ATTN: MOCA-MR 1 Research Institute , HO MASSTER , Ft Hood

1 USA Concept Anal Ait cy. Bethesda , ATTN: MOCA.JF I USA Recruiting Cmd , Ft Sherd ian , AT’TN: USARCPM—P

1 USA Antic Test Ct r , APO Seartle , ATTN : STEAC- M0 - ASL 1 Senior Army Ada., USAFA000/TAC , Elgin AF Aus FId No. 9

1 USA A itu,: Test CIr , APt) Seattl e , ATTN AMSTF.PL-TS 1 HO USARPAC , DCSPER . APO SF 96658. ATTN GPPE—SE

I USA Aimempiut CmiI Ited~uoni

’ Arseital , ATTN: ATSK-TEM 1 Stim lo n Lit, , Academy i,t Health Sciences , Ft Sam Houston

I USA Ar, namo ,,t Cmii , Rock lsl ~nd , ATIN: AMSAR - TDC 1 Marine Corns Inst., ATTN: Oem— MCI

I FAA- NAF EC , Atlnu,t i i - City, AT IN

~ Luh r~ry 1 HOIJSMC , Comma uud ent . ATTN: Code MTMT 51

1 FAA-NAI I-C, A ila rutu, - City , Al T N Hor n (rI~II Bi I IIOUSMC, Consms,t ,lant A ri N (‘ode MPI 20

1 FAA Aar on ’nutieal Ctr , Oklahom a CIty, ATTN: AAC 441) 2 USCG Academy, New Iiiuuilon , Ai TN: Admis sio n

2 USA FId A,ty Sch, Ft Sill , ATTN: Li l irary 2 USCG Academy, New Lie u, on, ATIN: Library

1 USA Armor Su: lu , Ft Knox , ATTN: Library I USCG Training Ct, . NV , AT EN: CO

I USA Armor Set , , Ft Knit s , ATTN: ATSB- Ol- E 1 USCG Tr.Inlng CD, NY . ATTN: E,fi,c Sec Ole

I USA Armor Set , Ft Knox , ATTN: ATS B-OT TP 1 USCG . Psychol Ass Br . DC , ATTN: OP 1162

I USA Ar m or Silt , Ft K ,m,,s , ATTN : ATS8-CO AD 1 HO Mid—Ring. Br , MC Oct . Duantico, ATIN: P&S Div

27

~

—- -- —- . -~~~~~~--

~~~~~~~~~~~ -.---- - ~~~~~~~~~~~~~ -----~~~~~~~~ - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- 

— - - -

~~~ ii - 

~~~~~~~~~~~~~~~~~~~~~~~~~

I US Ma,inr’ Cm,’ps Li

~ lsioiu Ole AMC. Aleev nd ria . ATTN: AMCG$—F 1 0.0 & CivIl Inst of Envfro Msdlcin . Cenada

I USATRADOC , Pt Monroe , AIIM ATRO.-EO i AIR cREss, Kensington. A’rTN~ Info Sys Br

6 USATRADOC , Ft Monroe, AT IN ’ ATPR-AO 1 MIIIta.rpsykolOgiiIi Tisneeti, Co~.Pseg.n

I USATRADOC , Ft Monroe ATTN ATTS—EA 1 MIlitary Attach.. French Ernb~~sy. ATTN: Doc S.c

I USA Forces Cmd , Ft McPherson. ATTN: Library 1 Msd.cln Chef. C.E.M.PA—A,ianal, Touicn/Nev.f Franc.

2 USA Aviation last Bd , Ft Rocker , ATTN : STEBG—PO I Prin Sci.ntiflc Off , AP~ l Hum Engr Rids Div . Ministry

1 USA Agcy I . - Avi ation Safety. c i Ru~ker , ATTN: Library of D.f.nw. New D.lhi

I USA Agcy for Aviation Safety, P . Rucke r , ATTN: Educ Advixoe 1 P.s’s Rich Ofc Library, AKA. veal Defense Farces

I USA Aviau ,,on Sets . Ft Ruckse . ATTN: P0 Otivasc o 1 Mlssigterla van Osfenate, 000PIKL Afd Sociesi

1 HOUSA Aviation Sys Cmd , St Lou is , ATTN : AMSAV—Z DR ~~yth0

~~~ 1~~~ Z.kan, The Hague, N,therlinds

2 USA Av iadon 9yg Tell Act., Edwards AFB , ATTN: SAVT E—T

1 USA Air Del Sets , Ft Bliss, ATTN ATSA IBM

1 USA Air Mobility Rsch & Dee Let , , Motfett FId , ATTN: SAVDL—AS

1 USA Aviat,on Sets, Ret Tog Mit ’ Ft Rue ’

eer . ATTN: ATST—T—RTM

1 USA Aviation Set , CO. Ft Rucker , ATTN: ATST—D—A

1 HO, DAR COM , Alexondri i. ATTN: AMXCO—TL

1 HO. DARCOM , A levandria , ATTN: COR

1 US Military Ac ari emny, West Point , ATTN. Serial s Unit

1 US Military Academy, West Point , ATIN: O4~ cif Milt Ldrshp

1 US Military Academy, West Point , ATTN: MAOR

1 USA Standard ization Op. UK , FPO MY , ATTN: MASE—OC

1 Ole of Naval Reels , Arlingt on . AT TN. Code 462

3 Ole of Naval Rach . Arlington . ATT N: Code 458

1 Ole of Naval R,eh , Ail inq toni , PITH’ Code 450

I Of cof Naval Rich , Ar lin ijt rm n. ATTN: Code 441

1 Nival Avros pc Mid Re. Lab . Pensjcula , AIIM: Acous Sell Div

1 Naval Aeros lic Med Res Lab , Pensacola , ATTN : Code L5I

I Naval Airos irc Med Re. Lit, . Pen sacola , ATTN: Code LB

1 Chief of Nav Pm ’ rs . AITN: Pers ’Ofl

1 MAVAIRSTA , N,,n!iulk ATIN: Sat aly Ct,

1 Mae Oceaiiuq r,&l iic . DC, ATTN: Code 6251 , Charts & Tech

1 Center of New ) A ume l . ATTN Doe Ctr

1 NavAirS ys Cois i , ATTN: Al It -b3t3 C

1 Nay OuMed , AITN: 713

1 NevHelicopt.rS.ib&tua 2, FPO SF 96601

1 AFHRL (FT) William AFB

1 AFHRI.. (Il l Lowry AFB

I AFHRL (AS) WPAFB , OH

2 AFHRL 10031) Brooks AFB

1 AFHRL I DOJNI Lackland AFH

I HOUSAF IINVSDI

I HOUSAF IDPXXAI

1 AFVTG ! Af l i Randolph AFt )

3 AMAL IHE) WPAFB , O~

’4

2 AF Inst of T i -” WPAFB , OIl . ATTN: ENEISL

1 ATC IXPIDI Randolph A Ffl

1 USAF A” m M . ’ ,I L - t - Snoo k s A FR SUL - 41. ATIN: DOC SEC

1 AF OSA ML! Arlin g ton ’

1 4~ Losj Cmi! , MnCIella rr A F S. ATTN : ALC/OPCRB

I Aim C r,,,.,. Acarlmr uy, CO. ATTN: Opt’s u’l “i.) Sen

5 NavPprs & Dee Cr, , San Diego

2 Navy Med Neuropiych iet rrc Rsm ’h Cmii , Sc- Diego

I Niv Elactronme Lilt , San Diego . AT TN R oe Lab

t Mis T i n ~ ‘ri Sen Diego . A’l’TN’ Cod.’ 9(00—f Ib

1 NrwPtn tGr.S,:lm , Mrsntr’rruy . AIIM: Code 5t)Aa

I NevPos tGrsSclu , Monterey, Alt N: Cork ’ 2)24

I Navlrng Er~

r u i 1 r Ct i , Orlsntiia , AT TN: Tech Lb

I US Dept of Labor , ic, An N: M.irr1sowrur Ailmin

1 US Dept ii? Justice , DC. ATTN . Dru g Lnl o rey Adm ln

1 Net Burnt Staurut a nrfs . DC, AITN: Co mputer info Section

I Nat Clearing H.uuisi ’ for MH- lnIo , Rockei lle

1 Derive , Fetter. ) Ct r Lakewood , ATTN . RIM

17 Defense Doe,umnermtetio n Center

4 DIr Psych , Army Hg, Russe ll Ofcs , Canberra

1 Scr.ntitic ArM,, Ml Rd. Ar my Hq, Russ ell Ofcs . Canberra

I Mit .yn it Air Attache , Austrian Euusls aesy

I (“‘ii,’ In’ Recise tchy Dr’s f- act,’urs l-4uinsa ir-n s ’ dv Ii Defense

Natirr nalc’. Brrrssrlt

2 Canadian ,li’ii iit Staff Waohi” rqt ,sn,

I (‘‘Air Staff , Rrryai Caneifit,rr AF , ATIN: Firs Std Anal Br

3 Chief , Cinediert Del Reefs Stiff ATTN: C/CRDS(W)

4 BrItish Del Staff . Briti sh Embassy, Wult ington

28

L -- .

~~~~~~~ -——-— ~~~~~~~~~~

. -