______ - -- — -

- -— - --.

~———--~

- _

~~~~ #

Technical Paper 326 

~ 

AD 1

LEVE .

EASE OF LEARNING ALTERNATIVE TOS

MESSAGE REFERENCE CODES

a Charles 0. Nystrom

George M Gividen

ARI FIELD UNIT AT FORT HOOD,TEXAS

-a.

[

~

1rJ

U. S. Army

Research Institute (or the Behavioral and Social Sciences

S.pt.mb.r 1978

Approved for public release; dèstnbutèon unlimited.

- -
_____ ________ 

- -

~~~

-

~~

-

~~~~~~

__________

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Fteld Operating A gency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WILLIAM L. HAUSER

JOSEPH ZEIDNER Colonel, US Army

Technical Director Commander

NOTICES

DISTRIBUTION: Primary distribution of this report his bien mad. by ARt . Please address correspondence

con cerning dist ribution of reports to: U. S. Army R~ ssrch Instit ute for the Bihsvi~raI and Social Sciences. ATTN. PERt-P . 5001 E,senhoviaer Avenue. Alesrendria, Virgins. 22333. •

FINAL DISPOSITION: This report may be destroyed ,vhen iS is no longer needed, P1e11 do not return it to

ifs. U. S. Army Rsis~~ch Institut e for the Bshevsoral and Social Sciences.

~QIL Th. findings in th is report ire not to bi coflitruid is an official Oep.rtmant of the Army position. unless so designated by other authorized documents. 

~~~~~

• 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~ 

~~~~~~ ~~~~~~~~ ~~~~~~~~~ ~~~~~ :T .:T ~~~~~~~~~~~~~~~~~~~~
_______ ~~~~~~~~~~~~

—- —-

~~~~~

--- -----

~

—: •

-

- - -—.-

~

-.- 

~~~~~~~~~~~~~~~~~~~~~~~~

Unclassified

SECURITY CLASSIFICATION OF THIS PAGE (WP.w DMa &,t.r.d~

DCD SDT 1

~ E~~~~~ ”&’ DAI

~E READ INSTRUCTIONS ,

~~~~~,

- 

~~~~ u i,

~~~~ us~ u~ s ~~ U s~ ’

u

’’ BEFORE COMPLETING FORM

I. REPORT NUMBER 2. GOVT ACCESSION NO 3. RECIPIENt’S CATALOG NUMBER

Technical Paper 326

LTlT~~L~~~d~~ ~~~$— — 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5. TYPE OP REPORT & PERt0O COVERED

I 

~~~ 

OF LEARNING ALTERNATIVE TOS MESSAGE I ( )J~~~FERENC !~~9DES, --

6. PERFORMING ORG. REPORT NUMBER

7. AUTH S. CONTRACT OR GRANT NUNBER(.)

aries 0. Nystrom

orge M.1 Gividen --

S. PERFORMING ORGANIZATION NAM E AND ADDRESS 10. PROGRAM ELEMENT. PROJECT . TA SK

U.S. Army Research Institute for the Behavioral, m AREA & WORK UNIT NUMBERS

and Social Sciences (1 W

Alexandria, Va . 22333 ~

‘ee. 

~ Q~

62l?6A723

II. CONTROLLING OFFICE NAME AND ADDRESS J, RFmIRI np~~~a

Assista

Combined 

nt 

Arms 

chief 

Center 

of Staff 

TRADOC 

for Force Developmen 1 ‘ 

Septi uuus~~ er~~~ 7~~~7 ~nsr r e ~i

Project Manager ARTADS 66 ti MONITORING AGENCY NAME & AODRESS (S1 di lierait from Controlling Ollic.) 55. SECURITY CLASS. (of (ills report)

Unclassified —— IS.. OECLASSIFICATION/DOWNGRADING SCHEDULE

ti DISTRIBUTION STATEMENT (of this Report)

Approved for public release; distribution unlimited .

TTi~53~~~~~7 cI

~

_EIJ 17. DISTRIBUTION STATEMENT (.1 lb. ab.trac .nt.r.d in Block 2Ô, if diff.rwt from R.port)

tS. SUPPLEMENTARY NOTES

IL KEY WORDS (Continua on rovsr•• aide ii n.c..say aid identify by block niaub.r)

Tactical Operations System (TOS) screening

Input procedures learning

User input-output device (UIOD) Input error rates

Message reference codes Message input time

User/computer language (

7 ~~~~~

‘ Automatic data processing (ADP) system

2S~~ASSt RACT (Caiuinu. ai r.vsea. aide it n.ce..aly aid id..niity by block nsisb.r)

‘

~~~~~~‘ To interact with a tactical data processing system such as the DEVTOS

or 7CS~

’, action officers and UZOD operators must correctly enter a message

reference code. The senior auth~r had participated in reconinending that

DEVTOS’ letter-letter-number (LL$

~ áodes be replaced by LLLL message reference codes for the forthcoming ~~~~ ~

ystem, predicting in effect that the

LLLL codes would be easier to learn . To confirm or disprove this prediction, 40 officers and 20 enlisted personnel were employed to learn one or .

~~.—

DO , ~~~~~ 1473 EDITiON OF t NOV~~~~IS~~~ BOLETE Unclassified

~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

OF ThIS PAGE (

~~

,e,s D.te Enisre~~ “

- - ~~~~~ •~~~~~•. . • • ____ _______________________________
--— --. .- —-

~~~-.-

Unclassified

SECURITY CLASSIFICATIoN OF THIS PAGE(Whw Del. &,iises~

20.

~the other of the code sets. Seated individually at a computer terminal,

the learner was shown message category titles and action names. The learner

entered the message reference code , transmitted it, and received feedback

as to the correctness or incorrectness of his input, plus the correct code.

The error rate associated with the learning of the LLLL codes was less

• than half that associated with the learning of the LL# codes (13% versus

29% for EM working on the G2 message codes; 11% versus 21% for officers

on the same G2 message codes; and 7% versus 15% errors for officers working

on G3 message codes). The LLLL codes were learned in about 60% of the total

time required by the LL# codes. Analysis of error rates by character position revealed that the old code’s numeric character and arbitrary second

letter were large sources of error. Several characters in the LLLL code

produced undesirably high error rates; the reasons for this and possible

corrections were usually obvious.

. •— - —•—--—-——.—— S

ACQES~ -

• 

. / ~~~~~~—

DOG -.

ww. JuS

~ 

I

Unclassified

SECURITY CLASSIFICATION OF THIS PAGEf ITh.n D•t. Eni.red)

- 

. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -

, —
- - - - ____ - -

- 

H .

Technical Paper 326

EASE OF LEARNING ALTERNATIVE TOS

MESSAGE REFERENCE CODES

Charles 0. Nystrom George M. Glviden

ARI FIELD UNIT AT FORT HOOD, TEXAS

Submitted as complete and Approved By: technically accurate, by:

George M. Gividen A.H. Blrnbaum, Acting Director

Field Unit Chief ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

Joseph Zaidner

TECHNICAL DIRECTOR

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

-9 Office, Deputy Chief of Staff for Personnel

Department of the Army

• September 1978

Army Project Number Tactical Operations System

200S2106A723

Approssd for public raises.; distribution unllrnit.d.

___________ - ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~ • — •. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
—— -~~~ --~~ 

—

~~~~~

-‘

~~~

-

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the. time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

I

— ——-~ .

- •

~~ -~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —•- 

- :_ .
FOREWORD

Technological advances have led to increased speed, mobility , and

destructive power of military operations. So that coninanders can make

tactical decisions consistent with the rapid change and succession of

events, information on military operations must be processed and used

more effectively than ever before. To meet this need, the Army is developing automated systems for receipt, processing, storage, retrieval,

• and display of vast amounts and different types of military data. There

is a concomitant requirement for research to determine how human abilities can be used to insure that command information processing systems

function with maximum effectiveness.

Manned systems research in this area done by the U.S. Army Research

Institute CARl) is directed toward enhancement of human performance and

improvement of man-machine interaction in relation to total system effectiveness. The research involves experimentation with various con—

figurations of system components, considering interactions and tradeoffs.

The end products--immediate or ultimate--are scientific findings on

human capabilities under varying conditions within the system . These

findings have implications for system design , development, and operational use. This publication describes the evaluation of the relative

ease of learning two message reference code sets used in inputting,

editing, and obtaining displays of information within the context of

a computer—assisted tactical operations system.

The entire research effort is responsive to requirements of Army

Project 2Q062 106A723 , ‘Enhancement of Performance in Military Organiza—

tions,” Fiscal Year D74 Work Program , and to special requirements of

the Assistant Chief of Staff for Force Development, the Combined Arms

Center of the U.S. Army Training and Doctrine Command , and the U.S.

Army Tactical Data Systems Project.

• -

Te nical Director
EASE OF LEARNING ALTERNATIVE TOS MESSAGE REFERENCE CODES

BRIEF

Requirement:

To evaluate the relative ease or difficulty of learning two alternative message reference codes used during inserting , editing , and extracting information from an Army tactical system .

Procedure:

Forty officers representative of potential G2 and G3 action officers

and 20 enlisted personnel representative of potential TOS input-output

device operators were given 2 minutes to examine tabular information

on describing the message titles and action in association with a message

reference code consisting of either two letters and a number (LL#) or

four letters (LLLL). Then, seated at a computer terminal with an input

keyboard and a cathode ray tube (CRT) display device, the soldiers individually typed and transmitted message reference codes in response to

displays of message titles and action names. They had to learn a list

of G2 messages or G3 messages. On the first pass through the list,

titles were displayed in an orderly sequence. On subsequent passes,

titles were in random order. Subjects had to make an error—free pass

before they were considered to have met the criterion of learning and

were considered finished.

Findings:

The error rate associated with learning the LLLL codes was less

than half that associated with learning the LL# codes (13% versus 29%

errors for enlisted personnel working on the G2 list; 11% versus 21%

for officers working on the G2 list; and 7% versus 15% for officers

working on the G3 list). Time required to meet the learning criterion

for those learning the LLLL codes was about 60% of that required to

learn the LL# codes. Analysis of errors by character position revealed

that the old code’s numeric character and arbitrary second letter were

important sources of error.

• F

____
Utilization of Findings :

A version of the new LLLL code has now been specified/incorporated

into the Tactical Operations System , Operable Segment (TOS2) computer

system as the result of having been jointly recommended for such incorporation by members of a TOS2 working group that included the senior

author as a participant. The findings support the previous hypothesis

that the LLLL code would be easier to learn than the LL# code used by

Development Tactical Operating System (DEVTOS) personnel. 

__________ —j

•

~~~~-

=~—~~~- —--•‘- •.--‘—-~ 

-,

- -.•- ~~~~~~~~~~~

_

~~~~~~~~~~~~~~~~ — . _ _ _ . ~~~~ 

— 

~~

.

• 

• 
EASE OF LEARNING ALTERNATIVE TOS MESSAGE REFERENCE CODES

CONTENTS

Page

INTRODUCTION 1

METHOD 2

Subjects 2

Equipment and Workspace 2

Procedure 3

Experimental Variables 4

RESULTS AND DISCUSSION 7

Effects of Type of Code 7

Officer—Enlisted Personnel Differences 9

G2 Versus G3 Differences 11

Errors by Character Position 11

Alternative Message Reference Code Schemes 11

CONCLUSIONS AND RECOMMENDATION S 16

APPENDIX A. INFORMATION AND INSTRUCTIONS FOR SUBJECTS 19

B . STATISTICAL ANALYSIS TABLES 33

C . ANALYSIS OF ERRORS BY CHARACTER POSITION 41

D . INDIVIDUAL PERFORMANCE DATA 55

DISTRIBUTION 65

LIST OF TABLES

Table 1. Message titles with old (LL#) and new (LLLL) codes . 6

2. Action names with old and new codes 7

3. Error rate, passes, and code input time 9

4. A partial revision of the LLLL code and titles 13

-

~~

—---- ~~ —— -

~~~~~~~~~~ .

---

~ -~-- •~~~~~ . • -
— ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

LIST OF TABLES (continued)

Table 5. Three—word message titles with three—letter

acronymic codes 14

6. Message titles, action name s , and code letters

required by a variable length acronymic code 15

7. Message titles , action names , and code letters

required by a three-letter partially acronymic

cone 17

A—la. Message titles and their abbreviations ,

for G2 functions 22

A—lb . Types of actions and the letter code for each 22

A—lc . Message code list to be learned in response to

message title and type of action in the area

of G2 functions 23

A— 2a. Message titles and their abbreviations , for

G2 functions 24

A-2b. Types of actions and the numerical code for each . . . . 24

A— 2c. Message code list to be learned in response to

message title and type of action in the area

of G2 functions 25

A-3a. Message titles and their abbreviations , for

G3 functions 26

A—3b . Types of action and the letter code for each 26

A— 3c. Message code list to be learned in response

to message titlc and type of action in the

area of G3 func tions 27

A-4a . Message titles and their abbreviations , for

G3 functions 29

A—4b . Types of actions and the numerical code for each . . . . 29

A-4c. Message code list to be learned in response to

message title and type of action in the area

of G3 functions 30

B-l. ANOVA of number of passes on G2 list for LL#

versus LLLL codes and officers versus EM 34
- • - - •••- - •-•-——--~~~••-•--•—• .•- •- • •- --- -

~~~~

-- -• - • • -- --- -

~~~~~~~~

- •

LIST OF TABLES (continued)

Table B-2. ANOVA of number of passes by officers

for LL# versus LLLL codes and G2 versus G3

lists 34

B-3 . ANOVA of percent error on G2 list for LL#

versus LLLL codes and officers versus EM 35

B-4. ANOVA of percent error by officers for LL#

versus LLLL codes and G2 versus G3 lists 35

B—S . ANOVA of time to input a message reference

code for G2 list for LL# versus LLLL code

and officers versus EM 36

8-6. ANOVA of time to input a message reference

code by officers for LL# versus LLLL codes

and G2 versus G3 lists 36

B—7 . t-tests of difference between means of correlated measures of percent errors for 1st

and 2nd character of the LL# code 37

B—8 . t—tests of differences between means of correlated measures of percent errors for 1st,

2nd and 3rd charac ters of the LLLL code 38

B-9. t—tests of differences between means of correlated measures of percen t errors for message

title codes and action name codes 39

8—10. t—tests of differences in mean percent errors

for numeric versus alphabetic ac tion codes 40

C—i . Summary of mean error percentages by character

position 42

C— 2. Mean percent errors by character position 44

C—3 . Mean percent errors by character position 45

C—4 . Mean percent errors by character position 47

C—5 . Mean percent errors by character position 54

D-l. Errors by pass: EM, G2 list, LL# code 55 

- ----- -~~~~~ 
—

LIST OF TABLES (con tinued )

Table D— 2. Errors by pass: EM , G2 list, LLLL code 56

D-3. Errors by pass : officers , G2 list, LL# code 57

D-4. Errors by pass: officers , G2 list, LLLL code • 58

D-5. Errors by pass : officers, G3 list, LL# code 59

D— 6. Errors by pass : officers , G3 list , LLLL code 60

D—7 . Errors by character position and by officer ,

G2 list, LL4~ code 61

D-8. Errors by character position and by officer ,

G2 list, LLLL code 62

D-9. Errors by character position and by officer ,

G3 list, LL# code 63

D-l0. Errors by character position and by officer ,

G3 list, LLLL code 64

LIST OF FIGURES

Figure 1. Mean percentage of code responses containing

at least one error as a function of code type,

list , and rank 10

L - - • 

i.._~~1
r

~~~~~~ L.T 

~~~~~~~~~~~~~~~~~~~~~~~~ _

~~~~~~~~~ ~~~~~~~~~

EASE OF LEARNING ALTERNATIVE TOS MESSAGE REFERENCE CODES

INTRODUCTION

Designers, developers, analysts , and researchers of Army tactical

data systems continue to seek software, display , and procedural means

of facilitating the task of translating free-text message information

into computer-acceptable input. This report is closely related to previous Army Research Institute CARl) publications1’2 and is part of a

long—term concern with problems of information presentation , processing ,

and utilization in Command and Control Systems .

A specific requirement for the research reported here was first

stated in early 1972. Anticipating the replacement of the Development

Tactical Operating System (DEVTOS) by the TOS—operable Segment (TOS2),

the Combat Developments Command (cDC) directed the formation of an ad

hoc committee to (a) recommend revisions of TOS2 formats, incorporating

DEVTOS and (MASSTER) experience where applicable; and (b) evaluate such

revisions using subjective analysis and DEVTOS testing .

Representatives of CDC , Modern Army Selected System Test, Evaluation ,

and Review (MASSTER) , Computer Systems Command (CSC) , Project Manager ,

Army Tactical Data Systems (PM ARTADS) , and ARI (then BESRL) formed the

committee. Their work produced four recommendations :

1. Consolidation of message formats ,

2. Use of variable field data entry ,

3. Map reference coordinate “packing , ” and

4. Use of a new message reference code system .

A very limited try-out of all these revisions on the DEVTOS facility

produced encouraging results. However , it was clear that more testing

was needed to assess separately the influence of each change on TOS

operators’ performance , to determine if each change facilitated the

• interaction of the operator with the semiautomated information system .

• :LBaker J. D., Mace , D. J., and McKendry , J. M . The Transform Operation

in TOS: Assessment of the Human Component. API Technical Research

Note 212 , August 1969.

2

Strub , M. H. Evaluation of Man-computer Input Techniques for Military

Information Systems. ARI Technical Research Note 226 , May 1971. (NTIS

No. AD 730 315)

1

-- ~~~~~~~~~~~~~~~~ ~ ie.i~~ — -~~~
-

The recommended message reference code set uses four alphabetic

characters for each member of the code set. In most cases, these characters comprise “acronyms” of the message titles and action names . The

DEVTOS code set , by contrast , uses two alphabetic characters to represent the message title and one numeric character to represent the action

name. The TOS~ system managers have replaced their DEVTOS-like letterletter-number (LL#) code set with the four—letter (LLLL) code set recommended by the TOS2 Format Revision Committee. This report compares the

“learnability ” of the two code sets.

METHOD

Subjects

The test subjects were 20 enlisted personnel and 40 officers from

MASSTER who successfully learned one of four message reference code

sets.3 A requirement for the enlisted men was that they have an Armed

Forces Qualification Test General Technical (GT) score of 90 or higher;

such enlisted men presumably would be more representative of those who

• might function as TOS2 MIO1~ (message input-output device) operators than

would a random sample of Fort Hood enlisted men . A requirement for all

subjects was that they must not have had prior experience in using the

DEVTOS message reference codes. Officer grades ranged from 01 to 05;

enlisted grades ranged from E2 to E7.

The enlisted personnel were randomly assigned to learning one of

the two G2 message reference code sets. Officers were randomly assigned

to learning one of these two G2 code sets or to one of the two G3 message reference code sets.

Equipment and Workspace

All subjects accomplished their learning task , one at a time , at

the same computer terminal . The terminal equipment includes a flazeltine

2000 cathode ray tube (CRT) and a separate keyboard input device. A

third component, a hard-copy output device , was not used . The equipment

was situated on a table in a large, well-lighted room . The keyboard was

located immediately in front of the CRT, and either item could be moved

to suit the comfort of the subject seated in front of them .

3

The learning performance data of an additional six officers and nine

enlisted personnel was incomplete and , hence , not usable. In nine of

these cases malfunctions of some component of the terminal/computer disrupted and/or terminated operations. In three cases the subject did not

cooperate properly, and in another three cases the subject was called

away on higher priority business.

2
— ~ -—-- -- ---

The computer terminal was linked to an IBM 370 computer system4 3t

Central Texas College, Killeen, Tex. A computer program presented the

message format titles and appropriate feedback displays, and recorded/

compiled data regarding response times and errors. Listings of these

recorded performance measures were subsequently used in performing statistical analyses of the performance data.

Procedure

Instructions to Subjects. Each participant, on arrival , was greeted ,

offered coffee , and seated at a chair facing the CRT. The experimenter

or an assistant sat with the participant to demonstrate keyboard usage

and control the duration of exposure. The subject of tabular information regarding the message titles was displayed , along with action names

and their associated title codes and action codes, which jointly cornprise the message reference code. For uniformity of presentation , instructions were read to each subject. Some procedural variation was

introduced when subjects asked questions regarding procedures or purposes; they received answers that others did not. Performance variation

due to differences among the presenters of instructions, if any occurred,

is likewise co—mingled with the effects of the main independent variables: type of code, G2 versus G3 list, and rank of learner .

The instructions and the four sets of three tables describing each

of the four code sets are presented in Appendix A.

In addition to describing keyboard usage , CRT display content, and

timing relationships, the instructions indicated that subjects were to

continue at the task until they made a “perfect pass ,” tha t is , the subject must have gone through the entire list of either 26 G2 message format titles or 57 G3 titles with no errors . The orderly arrangemen t of

titles during the first pass through the list and the random ordering

of the items on subsequent passes were described . Each subject was told

the purpose of the experiment.

As the subject began working , the experimenter stayed beside him to

detect any misunderstanding the subject might have concerning how to respond . When the experimenter judged that these procedural difficulties ,

if any , were resolved , he moved across the room out of the subject’s

vision.

Display Characteristics. A given message title, a hyphen , and the

name of an action were initially displayed in capital letters on the

• bottom line (of 15 lines total) of the CR’I This stimulus display then

appeared rolled upward to the 11th line . The subject’s answer (the

4

Consnercial designations are given only in the in terest of precision

of reporting . Their use does not constitute endorsement by the U.S.

Army Research Institute or the U.S. Army .

3

t

L • - 

_

i•

• • _ _ ~~~~_______~•

__ ______ ___•_______ ____ •
attempt at providing the message reference code) appeared on the 12th

line as the subject pressed the keys to enter an answer. When the subject pressed the “ESC” key to transmit a code—answer to the computer,

the display characters shifted up two lines, and a feedback display appeared on the empty 11th line. If the answer (e.g., UL3) was correct,

the feedback display would read as follows:

UL3 IS CORRECT

If UL3 was wrong and the subject should have answered UL2, the feedback

display would read thus :

*ERROR*

UL2 IS CORRECT

The *EP~

ROR* appeared first on line 11; all displayed characters were

shifted up a line; then UL2 IS CORRECT was displayed on the now-vacant

11th line.

This technique of shifting or rolling displayed lines upward was

used so that the subject could anticipate coming events. Inserted five

lines before each title display was this instruction :

ENTER ANSWER, THEN PRESS “ESC” TO CONTINUE

The separations among displayed lines were arranged so that when a new

title rolled up to line 11, the feedback display for the previous title

disappeared at the top of the CRT. The following two display lines appeared at the end of a pass through the list of titles wherein one or

more incorrect answers had been given:

YOU HAVE COMPLETED PASS

YOU ARE REQUIRED TO CONTINUE

When the subject met the criterion of learning by making a perfect pass,

these two display lines appeared :

YOU HAVE COMPLETED PASS

YOU HAVE FINISHED

Experimental Variables

Independent Variables. Three independent variables were used in

this experiment; the one called ‘code type” or “type of code” is of primary interest. One of the two types of codes used consists of two alphabetic characters followed by a digit, i.e., the old DEVTOS LL# code.

The two letters represent the title of a message category , and the digit

1 

•

.-.. -- -~~
7

represents the type of action the user desires to take with respect to

that category . Note, however, that in this experiment, message formats

were not displayed--only the names/titles of the message categories and

associated permissible actions.

The second type of code used is a four—letter code. The first three

letters represent the title of a message category , and the fourth letter

represents the type of action desired .

The “ rules” or schemes by which the two code sets are generated are

different. It is judged that the rules regarding the new four-letter

code are simpler to state (see Tables 1 and 2), partly because there are

fewer exceptions . An individual learning the new LLLL codes is typically

forming acronyms by using the first letter of each word . Most of the

difficulty in learning the LLLL codes should arise when the message category title contains other than three words. In the old LL4$ code, the

first letter sorts between friendly , enemy , and “common” message categories, with an exception included (“F” for aircraft-related message

categories). The second letter is arbitrarily assigned to the messages

within each of the friendly or enemy categories, but it is always an “A”

for the common messages. For the action code, the numbers 1 through 6

are used for the six permitted actions, but exceptions arise with the

common messages where 0, 8, and 9 replace the 1, 3, and 4 to enable the

computer program to discriminate among actions . The preceding description of the rules by which the two code Sets are generated is incomplete ,

but it helps to show why the four-letter code was recommended for incorporation in the TOS2 system and why it was predicted that learning the

four—letter code would be easier than learning the LL# code.

A second independent variable sorts learners into two categories :

officers and enlisted men . Both may be employed in roles wherein , among

component tasks , they must use the message reference code .

Subjects learned either the codes for the G2 message , plus action

terms, or the codes for the G3 terms ; thus G2 versus G 3 becomes the

third independent variable whose effects on performance can be assessed.

This assessment can be made only for the officers , however , as the available number of enlisted men did not allow assigning them to the G3 code

learning task .

Dependent Variables. The percentage of message reference codes

input with one or more characters in error (or not supplied) is judged

to be the most pertinent measure of learning performance (dependent variable) in this experiment. Another dependent variable is the number of

passes through the code list that each learner made in reaching the

success criterion of a perfect (error-free) pass. This measure counts

a pass with one error the same as a pass with many errors and therefore

• is a less sensitive measure of performance than the error rate measure

mentioned above; however , the two measures should be positively

correlated.

5

• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - • _ _ ___ _ _ -._ - - —— — - _ - — - - _ - - - —
- 

~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~

Table 1

Message Titles with Old (LL#) and New (LLLL) Codes

Message title code

G2 message titles Old New

Enemy Uni t Status EA EUS

Enemy Situation Da ta EC ESD

(Enemy ) Intelligence Summary ED EIN

(Enemy) In telligence Working File EE EWE

Enemy Situation Data Base Index EG EDX

Relaya REL

Named Area of Interesta a NAI

Standing Request for Information File AA SRI

03 message titles

Task Organization UA UTO

Task Force UB UTF

Tactical Dispositions UD UTD

Unit Disposition , One Coordina te UE UDC

Unit Disposition, One Unit UP UDU

Unit Disposition , General UG UDO

Air Control Measure UH ACM

Operations Journal UL UOJ

Situation Report UN USI

Pending Change UN UPC • Spot Report US USR

Aircraft Available FA AAV

aThese messages are also used for G3 functions and were displayed as

03 message titles as well.

6

-

~~~•-— 

__~~~~—~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - : 

—

~~~~~

-

~~ ~~~~~~~~~~~~~~~~~~ 

• - 

-
~~~~~~ ----~~~~~~~~ -~~~~- -~~~ ~~—---~~~~ - ---~~ -

Table 2

Action Names with Old and New Codes

Action code

Action name olda New

Add 1 A

Change 2 C

Delete 3 D

Query 4 Q

Special processing 5 P

Establish standing request for information 6 S

these special cases :

1. Use an action code of 0 to add a Relay message .

2. Use an action code of 8 to delete an “SRI File” message .

3. Use an action code of 9 to query the “SRI File . ’

Instructions to the subjects did not suggest that they emphasize

speed of response. Explicit emphasis such as “Be accurate, not fast”

was not given. However , the weight of the instructions is thought to

favor accuracy and to ignore speed . At any rate, the dependent variable

of time to input a message reference code was computed for each learner,

averaged over the 26 or 57 code inputs for each pass , and over all passes .

Supplementary analyses of error rates were made for the action code

alone (last character) , for the message title code (first two or first

three characters) , and for each character position . This was done to

determine if the arbitrariness in the second letter ’s assignment and

the exceptions in the action code (previously pointed out regarding the

LL# code) inflated the error rates in comparison to corresponding error

rate scores for the four-letter code.

RESULTS AND DISCUSSION

Effects of Type of Code

LLLL codes were clearly superior to LL# codes in that subjects

learned the LLLL codes in fewer passes and with a lower rate of errors .

7

— -

~~~~~~~~ -~~~~~~~:~~~: TJ..T i. - -
___________________________________________ - 

Number of Passes to Criterion. Subjects learning the LLLL code

achieved the goal of successful code learning in fewer passes , on the

average , than did subjects learning the LL# code (see Table 3). This

result was obtained for enlisted personnel (5.2 versus 9.2 passes) and

for officers (3.8 versus 6.8 passes) working on the G2 message reference

codes and again for the officers (2.9 versus 4.7 passes) working on the

G3 message reference codes. Among these three comparisons of LLLL versus LL#, there is a strong consistency of relationship; i.e., the mean

number of passes needed to learn the LLLL code is, respectively , 57%,

56%, and 62% of the number of passes needed to learn the LL# code.

Number of passes is, of course, higher correlated with total time to

learn.

Statistical analysis of these results indicates that the differences

favoring learning of the four-letter code are statistically significant

for each of the three comparisons (p < .01; p < .01; p < .05 respectively)

noted above. Analysis of variance tables relating to these and other

dependent variables are presented in Appendix B.

Percentage of Errors. The mean percentage of errors (responses

containing at least one error) made by each of the six groups of subjects is shown in Table 3 and Figure 1. Those learning the four—letter

message reference codes had smaller percentages of errors than did those

learning the LL# codes. As with number of passes, this result was obtained for the two groups of enlisted personnel (13.2% versus 29.1%) and

the two groups of officers (10.6% versus 21.4%) working on the G2 message

reference codes, and also for the two groups of officers (6.5% versus

15.1%) who learned the G3 message reference codes. All three differences

are statistically significant (p < .01 in each case). The error rates

associated with the LLLL codes are 45%, 50%, and 43% of the size of the

error rates associated with the learning of the LL# codes for the three

comparisons , respectively . The consistency of these error rate relationships (45% and 50%) for the enlisted men and officers working with the

G2 message titles is reflected in the analysis finding of a nonsignificant and very small interaction of learner ’s rank with type of code .

Time To Input a Message Reference Code. Results for this dependent

variable are shown in Table 3. The mean time to input a message reference code (averaged across inputs for a given learner , and then averaged

across learners wi thin a group) was not sign ificantly nor practically

influenced by code type : i.e., LLLL code versus LL$ code . This result

appears to be consistent with the lack of emphasis on inputting code responses quickly. Recall also that subjects had 20 seconds to input each

code response; this was 13 to 14 seconds more than the average time

taken . No one complained that this length of time was a source of pressure , irritation , or other concern ; neither did the monitors judge it to

be such.

8

S.
r

~ ~~~~~~~~

--

~~~

-— - -

~~~

-

.. I

Table 3

Error Rate, Passes, and Code Input Time

No. of Code No. of Mean % Mean no. of Mean input

Rank List items type subjects errors passes time (sec.)

EM G2 26 LL# 10 29.1 9.2 6.50

EM G2 26 LLLL 13 13.2 5.2 6.44

OFF G2 26 LL# 10 21.4 6.8 6.28

OFF G2 26 LLLL 10 10.5 3.8 6.53

0FF 03 57 LL# 9 15.1 4.7 5.64

OFF G3 57 LLLL 8 6.5 2.9 5.54

Note. Errors refers to codes input with at least one character incorrectly supplied or not supplied .

Officer-Enlisted Personnel Differences

The median GT score for the enlisted personnel participating in

this experiment was 124. Despite their high GT score , the enlisted

personnel took more passes in learning the LL# codes (9.2 versus 6.8

passes) and in learning the LLLL codes (5.2 versus 3.8 passes) than

did the officers. Analysis of variance of the 2 x 2 factorial design

(2 ranks x 2 code types on G2 messages) indicated that the overall difference between the two groups ’ performances was statistically significant (.01 < p < .05).

In terms of mean percentage of errors made while learning the G2

message reference codes , EM had a higher error rate in learning the

LL# codes (29.1% versus 21.4%) and in learning the LLLL codes (.~

.3.2%

versus 10.6%) than did the officers . The EM—officer overall error rate

comparison (21.14% versus 16.0%) was associated with a probability of

.053 , which just barely fails to meet the null hypothesis rejection

level of .05.

Given either a larger sample size and/or a lower distribution of

EM ’s GT scores , these error rate differences would probably have been

significant at the .05 level. Also, the gap between officer and EM

error rates is greater for the more difficult LL# codes than for the

LLLL codes. If LLLL message reference codes do close the gap between

• officer and EM performance, this might be exploited beneficially in

future TOS operations.

9
-

~~~~~~~

3j .

2Z

(I)

0

~~ 

G 2 -

____ ________ ___

LL

~~ 

LLLL. LL# LLL LL# LLL *~ Et~

’1

Lis t , Co de Type , cno’ 9’‘ cnk

Figure 1. Mean percentage of code responses containing

at least one error as a function of code type ,

• list, and rank.

L

10
~

1

/

Observed differences between officers and enlisted personnel for

mean time to input a message reference code averaged only .1 and .2

seconds for the two comparisons . These differences were far from being

statistically or practically significant .

G2 Versus G3 Differences

Subjects learned message reference codes related to G3 operations

in fewer passes than they learned the 02 codes . For the LL# type of

code , the comparison is 4.7 passes versus 6.8 passes. For the LLLL

codes, those applying to G3 were learned in 2.9 passes , on the average ,

versus 3.8 passes for the 02 list.

Error rate data also favored the learning of codes related to G3

operations over codes related to G2 operations . The overall comparison

is 10.8% errors on the G3 lists versus 16.0% errors on the G2 lists.

This difference is statistically significant (p < .01) . For the LL#

codes , officers working on the 03 list made 15.1% errors compared to

21.4% errors for the officers working on the G2 list. For the LLLL

codes , there were 10.8% G3 errors versus 16.0% G2 errors.

Overall , the mean time to input a code while learning was 0.87

seconds faster for the G3 list of codes (5.60 seconds) than for the G2

list (6.47 seconds) . The probability of obtaining a time difference

this large or larger by chance , under the hypothesis of no difference,

falls short (p = .08 ) of the rejection level of .05. If reliable, the

smaller mean time to input a G3 message reference code may be attributed

to the fact that the G3 list has a larger number and proportion of message titles that could be responded to correctly ; hence, the overal l

mean time to input a message reference code is shorter.

Errors by Charac ter Pos ition

To gain a better understanding of the sources of error and difficul ty in learning the LL# and LLLL codes , an analysis of errors by character position was made for each officer within each of the four combinations of list type (02 and 03) and code type . The results of this

analysis are presented in Appendix C. (The actual individual performance

data are shown in Appendix D.)

Al ternative Message Reference Code Schemes

As noted , subjects learned the LLLL code with abou t half the error

percentage associated with learning the LL # code . However , the analysis

of errors by character position (Appendix C) revealed that seven of the

new code’s characters were associated wi th excessively h igh error rates

in th~ learning situation used in the experiment . Thus , it seems

11

• ~~~~~~~~~~~~~~~ :

~~~~~~ _~~~~ _ • - - ----__•--—-__-_ ---j-—-— ~~~ - -- -~~~~ -- —- —-- -•
possible to further reduce error rates in learning and using message

reference codes. Four alternative approaches to achieving this additional error reduction are described and discussed in the paragraphs

that follow .

Approach A: Partial Revision. If only those message titles, action names , and code characters associated with excessively high error

rates (10% or more) were revised , a substantial further reduction in

error rates in learning the message reference codes could be expected .

Table 4 shows the message reference titles ( split into message titles

and action names) and code characters for both G2 and G3 operations .

An asterisk indicates that a revision has been made . Note that the

message titles do not all contain exactly three words . Consequently

the three—character code associated with the message titles cannot be

completely acronymic. This means that the procedure by which the code

has been generated cannot be explained to a potential learner with a

fairly simple, easy—to—remember statement . However , approaches B , C ,

and D all have the advantage of being easy to explain to a learner.

Approach B: Three-Word Message Titles, Three-Letter Acronyms.

In this approach , all message titles are forced to have exactly three

words so tha t the first letter of each word (in the message title) can

be used to form an acronym . This, in turn , permits using a sentence

such as the following to describe to a learner how to determine the

code for any message title plus action name combination : “The message

reference code consists of the first letter of each word in the message

title followed by the first letter of the first word in the action name .’

Table 5 displays the message reference titles and codes in the same

format as Table 4. The use of this approach raises this question--

Does the availability of the exact three—letter acronym “rul e” compensate

for the many chan ges in the message titles tha t are involved ?

Approach C: Variable-Length Acronymic Code. This approach requires presumably minor computer program revision so that the action

officer may input message reference codes of 2, 3, or 4 characters according to the presence in the message titles of 1, 2, or 3 words .

(The action code adds the addi tional charac ter , of course.) The explanation given a learner of this message reference code could be as follows :

“The code consists of the first letter of each word in the message title ,

then the first letter of the first word of the action name . ”

Implementation of this code requires message t~.tles to consist of

three words or less with no ambiguities ; e.g., two titles generating •

the same code . Table 6 display s the message titles , action names , and

code letters involved in this variable—length , acronymic code .

This approach involves changes to 8 message titles , compared to

16 title changes for approach B .

12 

-• - - - - - - _ ___ • .

• • _ .4
Table 4

A Par tial Revision of the LLLL Code and Ti tles

• Message title Code letters

Task Organization UTO

Task Force UTF

Tactical Dispositions UTD

Unit Disposition, One Coordinate UDC

Unit Disposition , One Unit UDU

Unit Disposition , General UDG

Air Control Measure ACM

Operations Journal UOJ

SITREP* SIT*

Pending Change UPC

Spot Report USR

Aircraft Available U1~

A~’

Common Relay Mes sage* cr&~ Named Area of Interest NAI

Standing Request for Inforrnation * SRI

Enemy Unit Status EUS

Enemy Situation Data ESD

(Enemy) Intelligence Summary EIS*

Intelligence Work File* IWF*

Situation Data Index* SDI*

Action name Code letter

Add A

• Change C

Delete D

Query Q

Special Processing S*

Establish Standing Request for Information E*

*Changed from current version .

13

__ 

:i

.

_

— 

~
i

_ --

_ • •

~~~~~

-— 

~~

-•

Table 5

Three-Word Message Titles with Three—Letter

Acronymic Codes

Message title Code letters

Unit Task Organiza tion * UTO

Unit Task Force* UTF

Un it Tac tical Dispos itions* UTD

Unit Disposition , Coordinate* UDC

Unit Disposition, Unit* UDU

Unit Disposition , General UDG

Air Control Measure ACM

Unit Operations JOurnal* UOJ

Tactical Situation Report* TSR*

Unit Pending Change * UPC

Unit Spot Report* USR*

Fr iendly Aircraf t Available * FAA *

Common Relay Message * CRM *

Named Interest Area * NIA*

Standing Information Request* SIR*

Enemy Unit Status EUS

Enemy Situation Data ESD

Enemy Intelligenc e Summary * EIS*

Intelligence Work File* IWF*

Situation Data Index * SDi~’

Action name Code letter

Add A

Change C

Delete D

Query Q

Special Processing S~ Establish Standing Information Request* E*

*Changed from current version .

14

- ~~~~~~~~~~~~~~~~~~~~
- -

Table 6

Message Titles, Action Names, and Code Letters

Required by a Variable Length Acronymic- Ôode

Message title Code letter(s)

Task Organization TO*

Task Force TF*

Tactical Dispositions TD*

Unit Disposition , Coordinate* UDC

Unit Disposition, Unit* UDU

Uni t Disposi tion , General UDG

Air Control Measure ACM • Operations Journal OJ~ SITREP* S~ Pending Change PC~ Spot Report SR*

Aircraf t Ava ilable AA*

Relay R*

Named Interest Area* NIA*

Standing Information Request* SIR*

Enemy Unit Status EUS

Enemy Si tua tion Data ESD

In telligence Summary * IS*

or INTSLJM* 1*

Intelligence Work File* IWF*

Situation Data Index * SDI*

Action name Code letter

Add A

Change C

Delete D

Query Q

Special Processing S~ Establish Standing Information Request* E*

*Changed from current version .

15

-I
r - •

Approach D: A Three-Letter Code Based on First and Last Words of

Message Title. Approach D could be explained with a statement such as

“The code consists of the first two letter3 of the first word of the

message title , plus the first letter of the last word , followed by the

first letter of the first word of the action name.” This approach does

not employ an exac t acronym , although the code letters form a partial

or incomplete acronym . (Webster ’s Seventh New Collegiate Dic tionary

defines acronym as “a word (as radar , snafu) formed from the initial

letter or letters of each of the successive parts or major parts of a

compound term.”) This approach requires the change of only one message

title. Table 7 displays the message titles , action names , and code

letters involved in this three-letter , incomplete acronymic code.

Which Approach Is Best? It is expected that all four approaches

to revising or designing a message reference code would result in further reduction of error rates ; however , approaches B , C, and D are

believed to be capable of bringing about more error reduction than the

partial revision described in approach A. A further experiment is

needed to determine which of the three approaches is best at reducing

errors; such an experiment is being prepared .

CONCLUSIONS AND RECOMMENDATIONS

It may be reliably concluded that the four—letter message reference code set recommended for use wi th the TOS2 can be learned more

quickly and with less error than the alternative two—letter , one—number

code set that had been used by DEVTOS personnel.

It is recommended that consideration be given to revising the current four-letter code version in accordance with Table 4 or changing it

to one of the three versions shown in Tables 5, 6, and 7. Adoption of

any of these four versions should further reduce the number of errors

m~ide by code learners.

16

___________________
• ~~~~~~~~~~~~~~

Table 7

Message Ti tles , Action Names , and Code Letters

Required by a Three—Letter Partially Acronymic Code

Message title Code letters

Task Organization TAO*

Task Force TAF *

Tactical Dispositions TAD*

Unit Disposition , One Coordina te UNC

Unit Disposition, One Unit UNU

Unit Disposition, General UNG

Air Control Measure AIM*

Opera tions Journal opJ~ Situation Report SIR*

Pending Change PEC*

Spot Report SPR*

Aircraft Available Am*

Relay Message* REM*

Named Area of In terest NAI

Standing Request. for Information File STF *

Enemy Uni t Status EN S*

Enemy Situation Data END*

Intelligence Summary INS*

In tell igence Work File INF*

Enemy Situation Data Base Index ENI*

Action name Code letter

Add A

Change C

Del ete D

Query Q

Special Processing

Establish Standing Request for Information E*

aChanged from current version .

17

_________________________ ___________________________________________________ _____________________________________________________________________________

L.. ~~~~... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~ _:~~~ • _L :-~~~

• •~
___________________ 

•

‘l

APPENDIX A

INFO RMATIO N AND INSTR UCTIONS FOR SUBJECTS

Information and Instructions Read to Subjects

MASSTER , the Compu ter Systems Command , the Army Tactical Data Systems

Project, and the Army Research Ins titu te have been experimen tal ly using

computer systems to assist in message processing . For this test we have

taken one of the G2 or G3 officer ’s sub-tasks , and will have you learn

it.

The G3 (or G2) must use a message reference code to tell the

computer which of 6 acti ons he wants to take on any of the 15 (or 8)

categories of messages that he handles. (Show Tables 1 and 2)

In table a you ’ll see the names of message categories and the code

that stand for each.

in table b you ’ll see the names of the six actions that can be taken ,

along wi th the sing le character action code.

The message reference code which you will be learning during this

experiment is formed by adding these two codes , as shown in table C.

Would you please read aloud the names and codes from table a and b

so that I ’ll know you have paid some attention to each one .

(After he ’s read a l ou d , take away the tables.)

• After we get the program going , you will see a message and action

name appear here at the bottom of the CR1. It will roll up (move your

finger up 4 times) one , two, three four lines and stop . After it

stops rolling , you will have 20 seconds to type in the message reference

code. What you type will appear on the CR1. After you ’ve typed in what

19

A
___ ---~~~~. .- - - • • ~~~~~~

- 

• • ~~~~~~~~~-••• - ---~~~~-

~~~~~~~~~~~~~~~~ --~~~~~ -

you think the code is, then hit this “ESC” button to tell the program that

you are finished making your response.

The program will then give you feedback. If your answe r i s correc t,

it will repeat your answer and say it is correct. If your answer is

wrong , the word “ERROR” will be displayed and then the correct answer

will be displayed. This feedback will last for 5 seconds and then the

program will go on to the next item .

If you see tha t you ’ve typed in something wrong and haven ’t yet hit

• the “ESC” button , then here’s how to correct it. You must replace the

wrong character and all characters to the rignt. Tell the program how

many characters you ’ re going to replace by pressing the shift key and

hitting this arrow (upper case 0). Then type in the correction and

send it.

There are 26 (57) titles in the list you will be learning . On the

fi rst pass through the list the titles will be displayed in this order

(point ~t table c). On the second and later passes the titles will appear

in a random order. You will be finished when you make a perfect pass;

i.e., go through the whole list wi thout making an error.

The purpose of this experiment is to get performance data on how easy

or difficult various message reference codes are to learn . When the

exper imen t i s over , we ’ll i nform the ARTADS people of our findings and

they will make use of them in a new computer system that will be

coming here in about a year.

Tables Shown to Subjects

On the foll owing pages are the tables that were shown to subjects

20

- - _____ _______ •
TI~~

as part of their information and instructions. A given subject saw only

three tables a , b , and c. Which set of three he was shown - set 1 , 2,

3 or 4 - depended on which code (LU’ or LLLL) and which area of operations

(G2 or G3) he was assigned to learn .

21 

--

• .~~~~- - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TJT =—~~~~:

-

t ~~~~~~~~~~ •~~~~~ •~~~ ~~ --—-.- — • --~~ ----
• - - ••--- • 

•

_ __

-• 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

-- ----- --- •

Table A-la

Message Titles And Their Abbreviations , For G2 Functions

Message Title

No. Message Title Abbreviat ir~

ns

Enemy Uni t Status I~

US

2 Enemy Situation Data ESD

3 (Enemy) Intelligence Surnary EIN

4 (Enemy) Intelligence Working File EWF

5 Enemy Situation Data Base Index EDX

6* Rela y REL

7* Name d Area of Int erest MI

8* Standini Request for Information File SRI

*These types of messages are also used for G-3 functions.

Table A-lb

Types of Actions and the Letter Code for Each

Action Desired Regarding Message Action Code

ADO (to data base)

CHANGE (the data base) C

DELETE (from data base) 0

QUERY (the data base) Q

SPECIAL PROCESSING (of some message types) P

ESTABLISH STANDING R[O~

(JEST FOR INFORMATIOfl S

22

p 

~-- -

~~~~~~-~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • • -

~~~~~~~~~ :: ~~ 

- -•
________________ ______________________________________ •

Table,A- lc

Message Code List To Be learned In Response To Messaqe Title

And Type Of Action In The Area Of ~

.2 Functions

No. Message Title And Action Message Code

1 Enemy Unit Status-ADD EUSA

2 Enemy Unit Status-DELETE EUSD

3 Enemy Unit Status-QUERY EIJSD

4 Enemy Situation Data-ADD ESDA

5 Enemy Situation Data-CUANGE ESOC

6 Enemy Situation Data-DELETE ESDD

7 Enemy Situation Data-DIJERY ESDO

8 Enemy Situation Data-ESTABLISH SRI ESOS

9 Enemy) Intelligence Suniiiary-ADD ETNA

10 Enemy) Intell igence Sumary-CHA~GE EIMC

11 Enemy) Intelligence Surnary-PELETE EIND

12 (Enemy) Intelligence Sumarv-SPECIAL PROCESSING EINP

13 (Enemy) Intelligence Work File-ADD EWFA

14 (Enemy) Intelligence Work File-CHANGE EWFC

15 (Enemy) Intelliqence Work File-DELETE EWFD

16 (Enemy) Intelligence Work File-QUERY EWFQ

17 Enemy Situation Data Base Index-ADD EDXA

18 Enemy Situaticn Data Base Index-CHAtIGE EUXC

19 Enemy Situation Data Base Index-OUERY EDXO

20 Relay-ADD RELA

21 Named Area of Interest-ADD NA IA

22 Named Area of Interest-CHANGE NAIC

23 Named Area of Interest-DELETE NATO

24 Named Area of Interest-QUERY NA TO

25 Standing Request For Information File-DELETE SRIO

26 Standing Request For Information File-QUERY SRIO

23

- —-——~-=~~ r--~~~ -~~~~~~~-~~~~~~~~- - : -- - — 

~~~~~~~~~~~~~~
~

Table A-2a

Message Titles And Their Abbreviations , For G2 Func ti ons

Message Title

No. Message Title Abbrevi ation

1 Enemy Unit Status EA

2 Enemy S itua tion Da ta EC

3 (Enem y) In telli g ence Sumary ED

4 (Enemy) Intelligence Workina File EE

5 Enemy Situa ti on Da ta Base In dex EG

6* Relay AA

7* Named Ar ea of In terest AA

8* Standing Request For Information File AA

*These types of messages are also used for G3 functions .

Table A-2b

Types Of Actions And The Numerical Code For Each

Action Desired Regard i ng t’~

essage Action Code

AD1~ (to data base) 1

CHANGE (the data base) 2

DELETE (from data base) 3

QUERY (the data base) 4

SPECIAL PROCESSING (of som e message types ) 5

ESTABLISH STANDING REOI!EST FOR INFQRMAT ION 6

Special Cases: (1) Only an Action Code of 0 may be used wi th the relay

message.

(2) Use an Action Code of 8 to delete an SRI File message.

(3) Use an Action Code of 9 to query the SRI File.

24

~ 

_

-~~~~~~~~~~~~~ -— -- -~~ ----~~~ —-~~~~~~~~-- -- 

•
Table A-2c

Message Code List To Be Learned In Response To Message Title

And Type Of Action In The Area Of G2 Functions

No. Message Title And Action Message Code

1 Enemy Unit Status-ADD EA 1

• 2 Enemy Unit Status-DELETE EA 3

3 Enemy Unit Status-QUERY EA 4

4 Enemy Situation Data-ADD EC 1

5 Enemy Situation Data-CHANGE EC 2

6 Enemy Situation Data-DELETE EC 3

7 Enemy Situation Data-QUERY EC d

8 Enemy Situation Data-ESTABLISH SRI EC 6

9 (Enemy) Intelli gence Summary-ADD ED 1

10 (Enemy) Intelligence Summ ary-CHANGE ED 2

11 (Enemy) Intelligence Summ ary— DELETE ED 3

12 (Enemy) Intelli gence Summary-SPECIAL PROCESSING ED 5

13 (Enemy) Intelli gence Work File-ADD EE 1

14 (Enemy) Intelli gence 1~

!ork File-Cf’P !~

1GF EE 2

15 (Enemy) Intelligence Work File-PELETE EE 3

16 (Enemy) Intelligence Work File_QUERy EE 4

17 Enemy Situation Data Base Index-ADD EG 1

18 Enemy Situation Data Base Index-CHANG E EG 2

19 Enemy Situation Data Base Index-OUERY EG 4

20 Relay-ADD AA 0

21 Named Area of Interest-ADD AA 1 ‘22 Named Area of Intere st-CHANGE M 2

23 Named Area of Interest-DELETE AA 3

24 Named Area of Interest-QUERY AA 4

25 Standing Request For Information File-DELETE AA 8

• 26 Standing Request For Information File-QUERY AA 9

25
___________ - - 

-

Table A-3a

Message Titles And Their Abbreviations , For G3 Functions

Message Titl e

No. Message Title Abbrev i ation

Task Organization UTO

2 Task For ce UTF

3 Tactical Dispositions UTD

4 Unit Disposition , One Coordinate UDC

5 Unit Disposition , One Un it UDU

6 Unit Disposition , General UDG

7 Air Control Measure ACM

8 Operations Journal UOJ

9 Situation Report LJSI

10 Pendi ng Change UPC

11 Spot Re por t USR

12 Aircraft Available AAV

13 Re l ay REL

14 Named Area of Interest NAT

15 Standing Request for Information SRI

Table A-3b

Types of Action and the Letter Code for Each

Action Desired Penardi na Message Action Code

ADO (to data base) A

CHANGE (the data base) C

DELETE (from data base) 1)

QUERY ( the da ta base) Q

SPECIAL PROCESSING (of some message types) P

ESTABLISH STANDING REQUEST FOR INFORMATION S

26

L 

~~~~ _ 

~~~~~~~~ • _.
Table A-3c

• Message Code List To Be Learned In Response To Message Title

And Type Of Action In the Area Of G3 Functions

No. Messaoe Title And Action Message Code

1 Task Organization-ADD UTOA

2 Task Organization-CHANGE UTOC

3 Task Organization-DELETE UTO~

)

4 Task Organization-QUERY LJTOO

5 Task Organization -ESTABLISH SRI UTOS

6 Task Force-ADD UTFA

• 7 Task Force-CHANCE UTFC

8 Task Force-DELETE UTID

9 Task Force-QUERY UTFQ

10 Task Force-ESTABLISH SRI I’TFS

• 11 Tactical Dispositions -QUERY UTDO

• 12 Tacti cal Dispositions-SPECIAL PROCESSI NG UTDP

13 Tactical Dispositions -ESTABLISH SRI UTDS

14 Unit Disposition , One Coordinate -ADD UDCA

15 Unit Disposition , One Coord i nate -CFAN GE 110CC

16 Unit Disposition , One Coordinate -DELETE UDCD

17 Unit Disposition , One Coordinate-C1’ERY UDCO

18 Unit Disposition , One Coordinate-ESTABLISH SRI UDCS

19 Uni t Disposition , One Unit -ADD UDUA

20 Unit Disposition , One Unit -CHANGE UDUC

21 Unit Disposition , One Unit-DELETE UDUD

22 Unit Disposition , One Unit -QUERY UDUO

23 Unit Disposition , One Unit-ESTABLISH SRI UDUS

24 Unit Disposition , General -POD UDGA

25 Unit disposition , General -CHANGE tJflGC

26 Uni t Disposition , General-DELETE UDGI)

27 Uni t Disposition , Gener al-flI ERY UDGO

28 Unit Disposition , General -ESTA BLISH SRI HOGS

29 Air Control ~1easure-ADD ACMA

30 Air Control Measure -CHANGE ACF1C

31 Air Control Measure-DELETE ACMD

32 Air Control Measure -QUERY ACMO

33 Air Control Measure -ESTABLISH SRI ACMS

27

- -.- . - —-—- --—--- - 
Table A-3c , Continued

No. Message Title and Action Message Code

34 Operations Journal -ADD IJOJA

35 Operations Journal-DELETE UOJI)

36 Operations Journal -QUERY UOJQ

37 Operations Journal -SPECIAL PROCESSING UOJP

38 Situation Report-CHANGE USIC

39 Situation Report-SPECIAL PROCESSING USIP

40 Pending Change -ADD UPCA

41 Pending Change-CHA NGE UPCC

42 Pending Change-DELETE UPCD

43 Pending Change-QUERY UPCO

44 Spot Report-ADD USRA

45 Spot Report-QUERY USRO

46 Aircraft Available -ADD AAVA

47 Aircraft Available -CHANGE AAVC

48 Aircraft Available -DELETE AAVD

49 Aircraft Available -QUERY AAVO

50 Aircraft Available -ESTABLISH SRI AAVS

51 Relay-ADD RELA

52 Named Area of Interest-ADD NA IA

53 Named Area of Interest-CHANGE NAIC

54 Named Area of Interest-DELETE NAID

55 Named Area of Interest -OUERY PIAIO

56 Standing Request for Information File-DELETE SRID

57 Standing Request for I nformation File-QUERY SRIO

28

-

~ 

~~~~~~~~~~~~~ ~~~~~~~~~~-

~~~~• ••~~~~~~~~~~~~~~~~ • ~~~~ -•~~~~~~~~~~ - -
Tah1~ A-4a

Message Titles And Thei r Abbreviat i~~s , For G3 Functions

Message Title

No. Message Abbreviation

1 Task Organization UA

• 2 Task Force UB

3 Tactical Dispositions UI)

4 Unit Disposition , One Coordinate HE

5 Unit Disposition , One Uni t UF

6 Unit Disposition , General HG

• 7 Air Control Measure I’ll

8 Operations Journal IJL

9 Situation Report ur~

i

10 Pending Change UN

11 Spot Report US

12 Ai rcraft Available FA

13 Relay AA

14 Named Area of Interest AA

15 Standing Request for Inform ation File AA

Table A-4b

Types of Actions And The Numerical Code For Each

Action Desired Reri~rdinq Messane Action Code

ADD (to data base)

CHANCE (the data base) 2

DELETE (from data base ) 3

QUERY (the data base)

SPECIAL PROCESSING (of some messa qe tvnes) 5

ESTABLISH STANDIIIF- REQUEST FOR IHFOPMPTJON 6

Special Cases : (1) Only an Action Code ci’ 0 may he used w ith the

Relay Messacie .

(2) Use an Action Code of 8 to delete an SRI File

message. (3) Use an Action Code of 9 to query the SRI File.

29
Table A-4c

Message Code List To Be Learned In Response To Message Title

And Type Of Action In The Area Of G3 Functions

No. Message Title And Action Message Co~

1e

1 Task Organization-ADD hA 1

2 Task Organization-CHANGE hA 2

3 Task Organization -DELETE HA 3

4 Task Organ ization-QUERY !JA 4

5 Task Organization -ESTABLISH SRI HA 6

6 Task Force-ADO UP 1

7 Task Force-CHANGE h R 2

8 Task Force-DELETE UB 3

9 Task Force-QUERY UB 4

1 0 Task Force—ESTABLISH SRI HR 6

11 Tactical Disposition s-QUERY liD 4

12 Tactical Dispositions-SPECIAL PROCESSING HO 5

13 Tactical flisposition s-ESTABL ISH SRI UD E

14 Uni t Disposition , One Coordinate -ADD liE 1

15 Unit Disposition , One Coordinate -CHAN GE tJE 2

16 Unit Disposition , One Coordinate -Q EL ETE lIE 3

17 Unit Disposi tion , Ore Coordinate -QUERY tiE 4

18 Unit Disposition , One Coordinate -ESTARLISH SRI liE 6

19 Unit Disposition , One Unit -ADD liE 1

20 Unit Disposition , One Unit -CHAN GE liE 2

21 Unit Dispo sition , One Unit-DELETE HF 3

22 Unit Disposition , One Unit- OUERY HF 4

23 Unit Disposi tion , One Unit-ESTABLISH SRI UF 6

24 Unit Disposition , Genera~-ADD HG 1

25 Unit Disposi tion , General -CHANGE HG 2

26 Unit Dispo sition , General-DELETE UG 3

27 Unit Disposition , C,enera l-flL’E~Y UG 4

28 Unit Disposition , General -ESTABLISH SRI tiC 6

29 Air Control Measure-ADD IJIl 1

30 Air Control Measure-CH ANGE li 2

31 Air Control Measure-DELETE Uli 3

32 Air Control Measure-OHERY UH 4

33 Air Control Measure-ESTABLISH SRI UH 6

34 Operations Journal-ADD UI 1

• 35 Operations Journal -DELETE UL 3

36 Operations JOurnal -QUERY UI 4

37 Operations Journal -SPECIAL PROCESSING UI 5

30

~ 

___ ~~~~~~~~~~~~~~~~~~~~ ~~~~ - •.• • •~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-

Table A-4c, Continued

No. Message Title And Action Message Code

38 Situation Report-CHANGE UM 2

• 39 Situation Report-SPECIAL PROCESSING UM 5

• 

. 40 Pending Change-ADD UN 1

41 Pending Change-CHANGE UN 2

42 Pending Change-DELETE UN 3

43 Pending Change- QUERY UN 4

44 Spot Report-ADD US 1

45 Spot Report-QUERY US 4

46 Aircraft Available-ADD FA 1

47 Aircraft Available -CHAN GE FA 2

48 Ai rcraft Available -DELETE FA 3

49 Aircraft Available - DIJERY FA 4

50 Aircraft Available -ESTABLISH SRI FA 6

51 Relay-ADD AA 0

52 Named Area of Interest-ADD AA 1

53 Named Area of Interest-CHANGE AA 2

54 Named Area of Interest-DELETE AA 3

55 Named Area of Interest_QUERy AA 4

56 Standing Request for Information File-DELETE AA 8

57 Standing Request for Information File-QUERY AA 9

31

- - ~

—;. 

~~~r ; : ~~~~~~~ _ j ~~~~~_ . . 1

~L_
APPENDIX B

STATISTICAL ANALYSIS TABLES

This appendix contains tabular presentations of the results of the

several analyses of variance performed on the data. The ordering of the

tables follows the order in which the dependent vari ables are presented

for a given i ndependent variable; i.e., numbe r of passes to reach the

learning criterion , percent errors made , and time to input a message

reference code followi ng presentation of a message title and action name .

Because the number of subjects in the six groups was not equal

(n = 10, 10, 10 , 13 , 9, 8) calcu l ation of sums of squares by usual

procedures produced some negative interacti on sums of squares and mean

squares . The method recommended by Wa l ker and Lev* on pp. 381-382 to

handle unequal frequencies was followed in all of the 2x2 ANOVAS performed .

• *Wa lker , H.M. & Lev , J. Statistical Inference, Henry Holt and Company New York : 1953.

33

• 

i 

UIDD

~ L~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~T71 --- ~~~--—.~~~=--

~~

----

~~

- —~~~-- -

_ -_-— ~~~~~~~~~~~~~~~~ — _j : - _ i

~~- — -—_
Table B-i

ANOVA OF NUMB ER OF PASSES ON G2 LI ST

FOR IL# vs. IILI CODES AND OFFICERS vs. EM

Source S.S. df M .S. F Prob .

Code 12.0895 1 15.81 p~. .001

Rank 3.5231 1 4.61 .Ol~ -p & .O 5

C x R 0.2736 1 0.36 p, .50

— 

Error a 39 0 7645a 

- - - - -

aError Mean Square 3l6.49234 39 x 0.0942 = 0.7645

where 0.0942 = mean reciprocal of sample sizes and

316.4923 = Error S.S. from usual ANOVA.

Table B-2

ANOVA OF NUMBER OF PASSES BY OFFICE RS

FOR LU vs. LLLL CODES AND G2 vs. G3 LISTS

Source S.S. df M.S. F Prob.

Code 5.8153 1 5.8153 18.63 p~ .001

List 2.2907 1 2.2907 7.34 p = .01

C x I .3838 1 0.3838 1.23 .25~ p~ .30

Error a 36 O.312la - - - - - -

aError Mean Square = 108.7827+ 36 x 0.1033 = 0.3121

where 0.1033 = mean reciprocal of sample sizes , and

108.7827 = Error S.S. from usual ANOVA.

34 

—-— -

~~~. - _..: ~~~~~~~~~~~ 

—._•—-. — _ - — -
- - -~~ - ~~~~~ - - -~~~~~~~~~~~~~~~~~~~ -- - - ‘

Table B-3

ANOVA OF PERCENT ERROR ON C-~ LIST FOR LL# vs. LLLL

CODES AND 0FFIU~

RS vs. EM

Source S.S. df M .S. F Prob.

Code 179.1583 1 179.1583 26.94 p< .001

Rank 26.6773 1 26.6773 4.012 p= .053

C x R 6.0269 1 6.0269 0.91 .35 (p~~.4O

Error p 39 6 6495a

aError Mean Square = 70.5891 .

~

. 39 x 0.0942 = 6.6495,

where 0.0942 = mean reciprocal of sample sizes , and

70.5891 = Error S.S. from usual AHOVA .

Table B—4

ANOVA OF PERCENT ERROR BY OFFIC ERS FOR LL# vs. LLLL

CODES AND G2 vs. G3 LISTS

Source S.S. df M .S. F Prob .

Code 95.6484 95.6484 29.60 p~ .001

List 26.5226 1 26.5226 8.21 p~ .01

CxL 1.3224 1 1.3224 0.41 p~~.5O

Error a 36 3 2309a

• aError Mean Square = 31 .2768 ~

- 36 x 0.1033 3.2309 ,

• where 0.1033 = Mean rec4 procal of sample sizes ,

and 31.2768 Error S.S. from usua l ANOVA.

35
Tabl e 8-5

ANOVA OF TIME TO INPUT A MESSAGE REFERENCE CODE

FOR G2 LIST FOR LL# vs. LLLL CODE AND OFFICERS vs. EM

Source S.S. df M.S. F Prob.

Code .0092 .0092 0.04 p>.50

Rank .0037 1 .0037 0.02 p>.5O

C x R .0234 1 .0234 0.11 p>.50

Error a 39 20ç2a - - _ - _ -

aError Mean Square = 84.9721

~~ 

39 x 0.0942 = 0.2052 ,

where 0.0942 = mean reciprocal of sample sizes,

and 84 .9721 = Error S.S. form usual ANOVA.

Table B-6

ANOVA OF TIME TO INPUT A MESSAGE REFERENCE CODE

BY OFFICERS FOR LU vs. LLLL CODES AND G2 vs. G3 LISTS

Source S.S. df M.S. F Prob.

Code .0018 1 .0018 0.01 ~~~~~~~ List .6942 1 .6942 3.26 p = .08

C x I .0629 1 .0629 0.30 p~~.5O

Error a 36 2130a _ - - - - -

aError Mean Square = 74.2774+ 36 x 0.1033 = 0.2130,

where 0.1033 = mean reciprocal of sample sizes ,

and 74.2774 = Error S.S. from usual ANOVA.

36
____ - 

— 

_ • - - --—--

~~~~

- — - ~~

—•

~~~~~

--- ——-

~

--.- -• - -

~~~

• _ 

~

—‘--—- - ——-_ _.•—----_.•_ - •

Table B-7

t-TESTS OF DIFFERENCE BETWEE N MEANS OF CORRELATED MEASURES

OF PERCE NT ERRORS FOR 1 st an d 2nd CHARACTER OF THE LL# CODE

(Officer Data Onl y)

G2 List G3 List Overall

1st Letter 4.55% 3.49% 4.05%

2nd Letter 13.34% 10.99% 12.23%

Difference 8.79% 7.50% 8.18%

S

~ 1.3924% 1.1404% O.8983~

t 6.31 6.576 9.10

Probability * c .001 ~ .001 ~ .001

*two tai led

37

• • _ . 0

~ -~~ -

______________________ • 

~~~~~~~~~~~~:T~~ iT ____ :r::~~

—-

~=~

:: : .

~~~~~~~~~~~ 

- A
___ 

~~~ - ~

-

~

-‘

~I 

~ ~~~~~~~~~

Table 8-8

t-TESTS OF DIFFERENCES BETWEEN MEANS OF CORRELATED MEASURES OF

PERCENT ERRORS FOR 1st, 2nd and 3rd CHARACTERS OF TUE LLLL CODE

(Officer Data Onl y)

G2 List G3 Lis t

1st Letter 3.25% 3.81%

2nd Letter 5.35% 2.10%

Difference 2.092% -1.7125%

0.6180% 0.4991%

t 3.39 3.43

d.f. 12 7

Probability * ~ .01 4.02

2nd Letter 5.35% 2.10%

3rd Letter 7.13% 3.65%

Difference 1.7846% 1.557

S

~ 

0.6299% 0.244 9%

t 2.83 6.33

d.f. 12 7

Probabili -ty * < .02 ‘ .001

1st Letter 3.25% 3.81%

3rd Letter 7.13% 3.65%

Difference 3.877% -0.1625%

0.9809% 0.5318%

t 3.95 -0.31

Probability* 

12 

>.5O

*two.tailed

38

________________ - 

- —~~~~~~~

--

~~~~~~~~~~~~~~
Table B-9

t-TESTS OF DIFFERENCES BETWEEN MEANS OF CORRELATED MEASURES

OF PERCENT ERRORS FOR MESSAGE TITLE CODES AND ACTION NAME CODES

(Officer Data Only)

LL # CODE

G2 List G3 List Overall

Title Code 8.96% 7.23% 8.14%

Action Code (#) 13.11% 6.24% 9.86%

Difference -4.15% 0.9888% -1.7158%

S

~ 

1.0772% 0.7873% 0.8953%

d.f. 9 8 18

t -3.85 1.25 -1.92

Probability* ~ .Ol .26 .074

*two_taj led

LILL CODE

G2 List G3 List Over all

Title Code 5.25% 3.20% 4.47%

Action Code (L) 4.22% 1.60% 3.22%

Difference 1.0385% 1.60% 1.2524%

S- O.5763’~ 0.7300% 0.4451%

d.f. 12 7 20

t 1.80 2.19 2.81

Probabil ity * .098 .068 = .011

*two_ taj led

39
Table B-b

t-TESTS OF DIFFERENCES IN MEAN PER CENT ERROR S FOR NUMERIC VERSUS

ALPHABETIC ACTION CODES.

(Officer Data Only)

G2 List G3 List Overall

Mean (#) 13.11% 6.24% 9.86%

Mean (1) 4.22% 1.60% 3.22%

Variance (//) 25.2277 6.4903 27.9026

n(#) 10 9 19

Variance CL) 6.3664 1.7114 6.1126

n (L) 13 8 21

S.D. (diff. of means) 1.7356% 0.9670% 1.3265%

d.f. 13* 13* 26*

t 5.12 4.80 5.01

Probability ~ 

.001 <.001 (.001

*Computed by Walker and Lev ’ s formula (7.26) on page 158 .

40

L ~ -

~~~~• -

~~~~~~~~~~~~~~

-

~

- 

~~~~~~~~~~

--=-

~~~~~~~~~~~~~~~~~~~~~~~~~ -. • ~~~~~~~~~~~~~~ —-~~~~~~- - ~~~~~ - ~~~~~~~~~~~~~~~~~~~ —- - •~~~

-

~~~~

—- ---
-

_ 

•

APPENDIX C

ANALYSIS OF ERRORS BY CHARACTER POSITION

Errors By Character Position

To gain a better understanding of the sources of error and difficulty

In l earn i ng the LU and LLLL codes , a count of errors by character position

was made for each officer wi thin each of the four combinations of list type

(62 and 63) and code type . These error counts were then divided by

each officer ’ s number of chances to err. Summing across officers by character position and , for other comparisons , across the message ti tle

character positions as well , a measure of the mean percent errcr by character position (or by set of character positions) was calculated . The

results reveal some differences not capable of being shown by the previously discussed dependent variable , percent of resnonses containing

at least one error .

Effect of the Second Letter of the LL-~ Code

Error data regarding the second letter of the LL~ codes give strong

support to the prediction that its arbitrari ness and the gaps in its

alphabetic sequence would be a major source of error in learning the

LL# codes. Com puted over the 62 and 63 lists (Table C-i), the mean

percent error for the second letter (12.23% ) was three tines that for the

first better (4.05°). The ratio is slightl y less than 3 to 1 for the G~

list and slightl y more than 3 to 1 for the 63 list. Three t-tests of

these differences between means (13.34% vs. 4.557 , 10.99% vs. 3.49%, and

12.23% vs. 4.05%) each indicated statistic al significance (see Table B-7)

41
Tabl e C-i

SUMMARY OF MEAN ER ROR PERCE NTAGES BY CHARACTER POSITION

(Officer Data Only)

Message Title Title Action Message Code Characters Code Code Reference

List Code 1st 2nd 3rd Overall Character Code

~Y LL# 4.55 13.34 NA 8.96 13 .11 10.34

62 LLLL 3.25 5.35 7.13 5.25 4.22 5.00

63 LL# 3.49 10.99 NA 7.23 6.24 6.91

63 LLLL 3.81 2.10 3.65 3.20 1.60 2.80

LL# 4.05 12.23 NA 8.14 9.86 8.72

LLLL 3.47 4.11 5.80 4.47 3.22 4.16

42

I

r
with the probability of obtaining at least the observed differences being

less than .001 under an hypothesis of no difference. Examination of

Tables C-i and C-2 shows that the mean percent errors made on the second

letter exceed those for the first letter for 16 of the 17 message title

codes other than the “AA ” codes. The “AA” codes are especially complicated , the result being that similar error rates were associated wi th

each of the two letters. The remaining exception is the ‘FA ” message

code where 6.2% errors were associated with the “F” and 4.3 errors wi th

the “A” (Table C-3).

It seems clear tha t the secon d letter of the LL~ co de was a major

source of error because i ts charac ters had very lit tle mnemon i c value

and w as even d if f i cul t to lea rn as a se quence of th e al pha bet because

of gaps . It also appears that other factors , su ch as rela ti ve frequency and recency of occurr ence , i nfluence th e learn i ng of the co de

characters so that in a certain minority of cases the first character

is as hard or harder to learn than the second character.

Error Rela tions in the lessage Portion of the LLLL Code

Neither the second nor third character of the LLLL code has three

times the error percentage of the first character , as was noted for the

LU code. Yet , in the case of the G2 list, hi gher error percentages

occur for the second character than for the first character , and still

higher error percentages occur for the third than for the second character (3.2 5’, then 5•351 , then 7.137; see Table C-i). From data in Table

43

____ ~~~~~~~~-
r ~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~

_ _

~~~~~~~~~~~~~~~~~~~

—

~~~~~~~~~~~~~

__

:-

~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

•

Table C-2

MEAN PERCENT ERRORS BY CHARACTER POSITION

G2 List , LL# Code , 10 Officers

Messane

Title Code Action Code

E A 1 * 3 4

4.7 6.6 10.4 14.4 4.3

E C 1 2 3 4 6

4.3 20.7 3.3 9.9 8.6 4.0 49.5

E D 1 2 3 5

4.0 16.9 4.8 6. 4 6. 1 45.4

E E 1 2 3 4

1.4 11.0 3.1 3.7 3.2 2.8

E 6 1 2 4

5.4 22.6 4.0 2.8 13.0

A A 0

13.6 9.9 20.1

A A 1 2 3 4

2.4 1.8 9.6 13.6 8.9 5.0

A A 8 9

10.8 11.0 45.1 38~~

1st 2nd 3rd 1 2 3 4 5 6 0 8 9

4.6 13.3 1TJ 5.9 7.3 8.2 5.8 45.4 49.5 20.1 45.1 38.6

*Gaps such as this occur because not all action codes are applicabl e to each

message category.

44

-— .- _~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~—-~~---- _ -~~~~~~~~~—~ --—---• - 

- 

• • • .

•- -
Table C-3

MEAN PE.R~

CENT E RROR S BY CHA RACTE R POSITIO N

(G3 List , LL# Code , 9 Officers)

Messa ge

Title Code Action Code

U A 1 2 3 4 6

0.4 3.9 1.9 3.7 0 0 20.5

U B 1 2 3 4 6

3.9 8.9 7.1 1.6 1.9 4.4 5.3

U D 4 5 6

4.3 20.4 12.5 21.4 7.5

U E 1 2 3 4 6

1.5 19.2 6.5 0 2.8 11.2 3.4

U F 1 2 3 4 6

0.6 19.0 1.6 5.6 0 0 3.4

U 6 1 2 3 4 6

0.6 4.0 0 9.1 9.3 3.2 0

U H 1 2 3 4 6

6.4 9.1 0 0.4 5.9 6.9 0

U L 1 3 4 5

0.7 b 6 .1 0 17.7 0 12 .1

U M 2 5

4.2 21.3 3.4 14.4

U N 1 2 3 4

6.2 17.5 0 7.5 0 12.4

U S 1 4

2.8 15.7 3.7

F A 1 2 3 4 6

6.2 4.3 0 0 2.8 3.8 2.8

A A o

20.9 20.9 35.2

A A 1 2 3 4

2.9 2.5 TT

~ 

5.3 5.6 4.4

A A 8 9

6.7 8.r 38.0 26.5

1st 2nd 3rd 1 2 3 4 5 6 0 8 9

3.5 1.7 3.7 4.6 5.2 16.0 5.4 35.2 38.0 26 .5~~~

1

_________ _

~~~~~E~ ~~~~~
A—4 , one can i nfer that the higher error rates for the second and third

characters are mainly the result of troubles in learning EIN for (Enemy)

Intelli gence Summary , EWF for (Enemy ) Intelligence Work File , and EDX

for Enemy Situation Data Base Index . The third character incurred higher

error rates than the second character primarily because learners persisted

in responding “EIS” rather than “ElM ” to the title (Enemy) Intelligence

Sumary . The difference in mean percentage errors between the first and

second characters of the 62 list , LLLL code , was statistically significant

(pc .0b ) and so was the difference between the second and third characters

(p~~.02; see Table B-8).

The pattern of errors among the fi rst three characters on the 63

list (Table C-i) was different from that of the G2 list. For the 63

list , the mean percent errors for the second character (2.10% ) was less

than that associated with either the first character (3.81%) or the third

character (3.65%). While these differences were statistically significant

(p< .02 and p< .001), the difference between the firs t and third character ’ s

error rates was not statistically significant. A likely reason for the

l ower error rates for the second character is that many of the G3 titles

contained just two words. In these cases subjects appeared to learn

rather quickly that they should enter a “U” for the first character and

then enter the acronym of the first title word as the second character

of the code.

Messa ge T itl e Errors Versus Ac ti on Name Error s

For just one of the four combinations of list and code type was the

46

I
Tabl e C-4

MEAN PERCENT ERRORS BY CHARACTER POSITION

(62 List , LLLL Code , 13 Officers)

Message Title Code Action Code

E U S A D Q

4.1 4.1 2.9 1.5 2.0 4.1

E S D A C D S

2.5’ 2.7 2.4’ 4.5 2.6 0 17.9

E I N A C D P

1~8 3.8 13.p 5.1 4.1 1.9 2’3.8

E W F A C D

4.5 10.0 9.7 9.0 1.9 0 3.8

E D X A C Q 2~1 12.7 14.2 5.1 1.3

R E L A

12.4’ 12.2 12.2 6.4

N A I A C D

1.3 1.7 1.7 1.5 2.6 1.3

S R I D 0 1~3 1.9 5.1 1.3 f.3

1st 2nd 3rd 4th A C D 0 P S

~~ 

5.4 7.1 4.2 3.9 2.7 1.7 L7 23.8 17.9

47

• —-—---- --—-—--- ---- ~

- 

-- 

- - - - ~•.
mean error percentage for the action code greater than that for t•he message

title code. The numeric action codes of the 62 list incurred 13.11%

errors while the message title characters of this list i ncurred 8.96%

errors. The difference between these means was statistically si gnificant

(p<.Ol ; see Table B-9). For the G3 list , LL# code , the action portion

had a l ower error rate (6.24%) than the title portion (7.23~

’) but the

difference was not significant (p = .26).

For the LLLL Code , a statistically significant difference was

obtained onl y when error percentages were compu ted over both 62 and 63

lists . For this comparison the title portion incurred 4.47~

’ errors while

the alphabetic action code portion incurred 3.22~ errors (p = .011).

Overall , the action code letter was inpu t with 3.22% errors while

the action code as a number was input wi th 9.86% errors (a statistically

significant difference ; p c .OO1 from Table B-b ). Since the error rate

for the message title portion dropped fron 8.14% for the LL# codes to

4.47% for the LLLL codes , we fin d the new LLLL code has been more helpful

in reducing action code error rates than in reducing message title ccde

error rates. Evidently the numeric action codes with their exceptions were

somewhat more difficult than expected . On the other hand , the new LLLL

title codes were somewhat less of an improvement than had been hoped for.

Action Code Problems

• The LL# code contains rinc different action codes , all numbers ,

compared to only six different letters serv i ng as the action codes in the

48
LLLL code system. The extra three action codes (0, 8, 9) apply to action

names that most often require a response of 1 , 3 , and 4. Apparently a

system designer felt that the fact that the three message categories

involved (Relay , Named Area of Interest, and SRI File) were common to both

62 and 63 , warranted giving them all the same letter-letter (AA ) designation.

Then , to provide unique action code numbers , for the computer program , he was

forced to use a “0” to distinguish Relay-Add from Named Area of Interest-Add;

to use an “8 ” to distinguish Standing Request For Information File-Delete ,

from Named Area of Interest-Delete ; and to use a “9” to distinguish SRI FileQuery from Named Area of Interest-Query . The high error rates for the

0, 8, 9 action code numbers show clearly that the use of the comon AA

designation was an unnecessary and undesirable complication (See Tables C-2

and C-3).

For the LLLL code and the G2 list (Table C-4) the worst action code

letter is the “P” standing for “Special Processing. ” This should be considered in conjunction with the “S’ that stands for Establish Standing

Request for Information . These two had error rates of 23.8% and 17.9%

for the 62 list , where they occurred only once per pass. On the G3

list , (Table C-5) P’ was required three time s per pass and ‘S” was required

six times per pass. Since their error rates dropped to 2.9% and 1.3%

overall on the G3 list , their rarity or low frequency would appear to be

the major factor producing the high error rates on the 62 list. 62

learners need help; 63 learners apparently need little hel p if any , wi th

the “P” and “S.” Review of the 62 learners ’ wrong responses indicated

that many were responding wi th an “S” to Special Processing and with an

49

______ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
___ ________ -

“E” to Establish SRI . They were using the acronym rule that most often

did apply. Unfortunately for their learning ’s success , these two codes

were requiri ng use of the first letter of the second word of the action

name Instead of the first letter of the first word . The reason for the

original assi gnment of “S” to Establish SRI was that it seemed appropri ate

or naturally associated with SRI . But this assignment ignored the need to

use the verb “establish” to describe /explain the action involved , and it

took insufficient account of the error-producing consequences of letting

“P” stand for Special Processing now that the “S” was pre-empted . It is

thought that chang ing the code letter for Special Processing to “S” and

the code letter for Establish SRI to “E” would aid the learning of the

62 list.

“Establish SRI” is quite clear in its meaning of the action. “Initiate

SRI ” seems a bit more precise , but to change the name of the action to this

(with a code letter of “I”) seems unlikely to reduce error rates below that

which the use of “E” would produce. The action name of “Start SRI ,” using

a code of “S ,’ would allow retention of the “5” code with perhaps not too

significant a loss in conveying the meaning of the action to the learner.

Individual Message Title Code Problems

The letters EIN were the code response required to the message category

titles displayed as (Enemy) Intelligence Summary . The “N” has a 13.2%

error ra te , and the common error was to give “5” as the third letter. It

• seems obvious that , given this display of the message title , EIS should have

been the required code. The situati on here has been complicated by the

50

___ -~~~~~ -

~~~~~~ • - — --~~~~~~ ----_- ---~~~——••~~~ • • • -•- ---- • -~~
addition of (Enemy ) in the display of the message title; this word does

not appear in the title printed on message format sheets . It was displayed

parenthetically to justify the use of “E” as the firs t letter of the code

and to create three words in the title thus allowi ng straightforward

application of the three letter acronym rule for this title. If all message

reference codes mus t have four l etters , “EIS” coupled wi th the use of

“(Enemy ) Intelligence Summary” as a title, is judged to be a combinati on

that would produce a lower error rate than “EIN” did.

In the next case , however , the addition of “(Enemy)” to “Intelligence

Work File ” disallowed simple application of the three letter acronym rule.

The “W” and “F” each had a 10% error rate. Subjects were charged with an

error for inserti ng the letter “I” and were unfortunately recuired to

leave this letter out of the code . The title displ ayed should have beer’

simply “Intelligence Work File ” and the code for it should be “IWF” plus

the appropriate action code.

The next 62 problem is the five-word title “Enemy Situation Data Base

I ndex ” (code is EDX) where use of a three letter title code requires the

subject to learn which three of the five words contain the code letters .

The “E” is the best first letter code; while the “0” was assi gned without

expectation that it would be more readily learned than “S” or “B ,” the

two remaining words “first letters . “X” stands out as a cue for index ,

but creates an unnecessary exceptional case , for it’ s not the first letter

of a word . The error responses were usually “ESD. ” Again , this shows

that on these occasions the learners were trying to follow a simple

1 

51
acronym rule. Unfortunately, the ex i stence of ano ther message ca tegory

ti tle “Enemy Situation Data ” pre-empts the use of “ESD” as a code. A

change in title seems much needed in this case in order to permit simple

application of the three letter acronym procedure for generating or determining the title code. “Situation Data File ” (Code to be “SDF~) and

“S itua tion Da ta Index ” (Code to be “SD!”) are candidate title reductions.

It is thought that either (the latter is somewhat preferred) would reduce

error rates below what are produced by the present combination of “EDX”

and a five word title.

“Relay-Add” is used by both 62 and G3 operations . The error rate

for the three code letters “REL ” was virtually the same (12.4%, 1 2.2%, and

12.2%) for each l etter for the G2 list and was the same (15%) for each

letter when being learned by subjects assigned to the 63 list. Most of

the errors regard i ng the “R” were of a prematu re comm ission nature -

subjects i nput the R before the display had stopped “rolling up. ” It is

apparently coincidental that the error rates for the R match those for

the other two letters . Given a one-word message title and a three-letter

code input requirement , “REL” seems fairl y obvious. Combined with “A ,”

the only action code permitted , produces “RELA ” which seems inspired or

at least clever. To adapt it to a three word and three letter acronym

scheme , the cumbersome title “Common Rela y ~essa ge” and associated code

“Cl

~

l” is a possibility .

Only one other character on the G3 list stands out wi th a high

error rate. Situation Report is encoded as “US!,” an d the error ra te

52
- --

for the “I” (Table C-5) is 27.2%. The bulk of the “I” errors come abou t

because the su bjects are i npu tti ng “USR .” “USR” is the more appropri ate

code, but it is already the code for “Spot Report. ” This duplication of

first letters (“5” and “R” ) for two different titles is not simple resolvable while staying within the acronym procedure. It seems necessary

to change one of the two ti tles to enable formulation of a different

acron ym i c code. The shor t form “SITREP” could be used , and as wi th “Relay ”

the first three l etters “SIT” employed as a (nonacronvmic) code. However ,

the dropping of “t.~~

’ from the code shoul d cause error ra tes to stay fai rl y

high, for “U” (standing for “unit” ) is frequently used in the G3 list as

a filler letter when the message titl e contains only two words.

53

- -.-- “ -.• ----
______ -- -

~~

• •

-

‘

~

-:—

•

~

-• --- 

I

Tabl e C-5

MEAN PERCENT ERRORS BY CHARACTER POSITION

(63 List , LLLL Code , 8 Officers)

Message Titl e Code Action Code

U T 0 A C 0 Q S

10.8 ~.6 6.3 T5.6 7.3 6.3 4.2 6.3

U T F A C D S

1.8 1.3 3.8 2.5 6.3 0 0 4.2

U T 0 P

0 8.8 0

U D C A C D 0 S

1.5’ 2.7 2.7 4.2 0

U D U A C 0 S

2.0 0.6 1.8 0 0

U D 6 A C D S

3.( 0.6 0.6 0 0 0 3.1

A C M A C D S

7.2 1.9 0.6 0 3.1 0 0 0

U 0 J A 0 P

4.0 3.0 2.3 3.1 0 0 0

U S I C P

7.8 4.7 27.2 0

U P C A C 0

2.3 0 2.6 0 0 2.5 0

U S R A

0 0 3.6

A A V A C U 0 S

5.3 2.6 4.5 0 2.5 0 0

R E L A

15.0 15.0 15.11 6.3

N A I A C U

0.6 0 0 0

S R I D

3.1 5.2 2.1

1st 2nd 3rd 4th A C D Q P S

3.8 2.1 3.7 T

~~ 

2.6 1.7 1.2 0.6 2.9 1.13

54

______ 

•
_____ - - __ ___

LU N- LL . r ~~ 1.0 , U) C~ U) ,- N- •.C) .- 1.0 C 1.0 CC

fl r r

1.0 0 0

U) C’)

c’

.J

(V) C’)

LU

0

C\J C’) C C’)

-J

-J

0 ~~~. ,

II.~

o ~ C C C’) 1.0 U)

>c C

— La.. •

~~ 1.,)

w LU

w a 0..

Q~ -~~ ~

D 1/) I— v~

0 0.

— CD C C’) C N- C’) N- C

>-

— cc 0

— N- . ~~ C’.J N- C’) U) C C’.J C

LU (.0 C’) U) U) a~ 0 C’~

J (‘J ~~ IU) .- cc cc C~J CC 0 1.0 0)

r —

CC CD .- C~’J C”J t

~~ cc I- r C ~ ~J i1’) C’) C’) U) CC C’) C C’) C\i C N-

— — I- (‘4 ~~ _- —

CC N. U) 1.0 ,— C’J C’4 14) u—

‘- .- (‘4 (‘4 I- (‘4 C~4

c~ ~0 0) 0 ~~~ (‘4 r- C) C

1

~~ •- I- I

~~ (‘4

L)

w •.- a, w 0 IV 0 W 0 -C ~~

~~~ .j _i v~ ~~ C c~ ~- C~~ cc C’) U) NV) LU LU LU LU LU LU LU LU LU LU

55

• — • —-.--.-• —--:• — - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - ~~~~~~~~~~~ T:~~~~~~__
- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Table D-2

ERRORS BY PASS: EM, 62 LIST , LLLL CODE

PASS

SUBJECT 1 2 3 4 5 6 7 8 9 10 ZE

E2 Ta 4 4 2 1 0 11

E4 RC 14 11 7 7 6 5 1 2 5 0 58

E4 CR 5 7 3 2 3 1 0 21

E 4 F~

a 4 1 0 5

E4 Ma 1 0 1

E6 WT 6 4 0 10

E5 WB 10 13 4 2 5 3 2 0 39

E5 Pa 8 7 3 2 0 20

E7 Ch 7 7 2 0 16

E8 Fi 15 6 2 2 0 25

56

— ~~TT ~~~~~- - ~---r ~~~~~~~~
Table 0-3

ERRORS BY PASS : OFFICER S, 62 LIST , LL# CODE

PASS

SUBJECT 1 2 3 4 5 6 7 8 9 10 11 12 ~~E

O4 Go 10 20 16 24 15 11 12 8 3 2 3 0 124

O4 Sc 5 13 6 6 1 4 2 1 0 38

O4 Gi 7 8 3 1 1 1 0 21

O3 Wa 6 11 4 3 0 24

03 0t 10 16 9 5 2 0 42

O3 Sp 14 11 5 0 30

O3 Ze 4 7 3 3 0 17

O3 Qu 6 5 0 11

O5 St 12 12 9 7 3 2 1 0 46

O2 8r 10 21 7 4 4 4 4 1 0 55

- — -

~~~~ — ~-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .- -

-

~~~~~~~~~~. - - •~~~~~~ • •

Table D-4

ERRORS BY PASS : OFFICERS , 62 LIST , LLLL CODE

PASS

SUBJECT 1 2 3 4 5 6

O4 DS 4 7 0 11

O4 Pr 5 12 6 5 2 0 30

O4 Ke 1 0 1

O4 WB 5 3 1 0 9

04 R6 4 2 0 6

04 1H 7 7 0 14

O4 Mu 4 4 0 8

04 J6 6 5 3 1 3 0 18

O2 Bj 3 1 0 4

O3 Lo 4 3 1 0 8

O3 Re 3 1 2 0 6

O3 Mi 9 5 1 0 15

O4 NP 6 3 6 1 0 16

58 

_----~~-- _ _ • •• •.- --_ ~~ 

••

~~~

•

~~ ~~~~~~~~~~~~~~
Table 0-5

ERRORS BY PA SS: OFFICERS , G3 LIST , LL# CODE

SUBJECT 1 2 3 4 5 6 7

O4 La 10 23 5 2 0 40

O4 St 17 16 3 0 36

O4 Pa 12 9 2 0 23

O4 Su 5 19 5 0 29

O4 Sl 11 15 0 26

O4 Be 13 6 0 19

O3 Mu 20 25 5 4 1 0 55

04 Br 18 35 13 11 11 2 0 90

O5 Ch 13 33 14 3 1 0 64

59

-- • . ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ---~ --— —~~~~
—~~ ~~ -_•- • • —~~ --—~~~•---- ---— ~~~~ .• _ -~~~~~ - -•-~~~~ - - -_ • ~~~~ • ---• - •

~~~~

Table 0-6

ERRORS BY PASS : OFFICERS , 63 LIST , LLLL CODE

PASS

SUBJECT 1 2 3 4 5 _____

• O4 Al 6 4 0 10

O4 Ma 7 0 7

O3 Cu 7 1 0 8

O3 La 8 0 8

O 3 L e 5 0 5

O5 Su 8 6 4 3 0 21

O3 Yo 15 3 4 0 22

O3 Me 8 0 8

60

• -—-_ - —-• _ ~~~~~~~~~~~~~~~~~~~~~~ • -——---- —_- •

~~~-• - -—• -•- - _________
I..

a,

0. 0

0 Cfl -.- C’J~~~ (‘4 0 ‘0 ~~~ C) CCC)

a,.4~

) .— C’) CD (’) U )C C’)N- C) C’)

o C’) (‘4 .— — .— — — (‘.

4 (‘.4

0’n 4 0 0

LU

o

C

L)

t0

~J D N. CO U) CD .— — C CC U) C’) 0)

0 LU

• C’J • ~~~ (0 U) U) C) C’) C U) U) N- — C)

t .——

_J 4’ ~~ N- CD C’) .— — C’) 0) .— C’)

(fl LU CD (‘.4 (‘4 (‘4 ~~~ (‘.4 .— C’) U)

(

C,

‘4 — —

LU

C.)

1-4 C ~~~~N- ’0 C~~~~~~~~ U )C’)0)~~~~ —

La.. 0 W

La •.- ~~ 00(0 CU) (‘.4 C )C ~~ (‘.4 C’) 0 4-’ CsJ .- — .- (‘.4 .- — e-

.‘- S.-

>- 11) 4)

cc 0.0

0- N. C ~~~

~~ ~~ ~~ S.- C’) U) (‘4 C’) ‘

~~~ C’) .— CC —0 )

C’) 1.0 C’4~~~~~~~~~ C’4 (’.J. C’) C’J

a. CD £ i-i

‘C

I- —

U)

C C 1.0 CC (‘4 .- (‘4 ~~ C’J N. C (‘4 C’)

0. 0 LU

~~ U) 0) CD C’) 0) U) 1.0 N. (‘4 1.0 C’) 4) C)J — — — — r- — LU ~~~~~~

C) 0 4-’

~~~ 0. 4-)

a’ .

~~ C LU CD C’) U) N- CU) CC ’0 U) C)

C.) (‘4 CC (‘4— — C’) — (‘4 C’)

>-

cc

U)

C C It)

~~ 0 N- N- N- 1.0 ,— CC CC 0) 0) ~~ U)

LU LU 4) ~~ N- ‘ C’4 ‘

~~‘ N- ~~~ C’) C’) (‘.4 ‘0

(“ a ,

0 4-)

a) 4-I --i

LU ~~~ ~~ ~‘) ‘0 U)

— (‘.4 —

4-)

U • a, o u~.-

~~~~~~~~~~~ 4-’ S..

‘)

.0 4)40

~~ U) C C C C 0 0 00 00 X

61

______________________ 

~~
4) .- 0)1.0 (‘.4 ~~ 0) 0) 0) 1.0 CC ~~ 40 40 C o 4..) N- U) U) CD N- N- N- U) N. C C C C’) C -.- — ~

C 

‘O Cfl

0

.— — — — — — U,

~~ 

0 C C’) IC) 1.0 N- . C’) ’- 0) 0) C’) 40 (“4

b-~ U)C~~~- C ’) U )N . U) a~

a U)a~~U)

LU — C +

C ~~~

C) +

0) (‘4 — C’) CC (‘4 C) 14) (“J O) 0) U) LU — — — — (‘.4 — C’.J C’.J

-J

-j

5-. (‘.4

(1) 1.D~~

-0 0)0) N- N. 40 C’) 0) 0) N- CC (‘4

— LU (‘4 (1) 0 (‘.4 C’) N- N- (.0.— (‘4 (‘4 N- C’) 40.

(‘4 4)

• 4)

~~~ LU c ’4 C ) 0 C’m 1 . 0 1.0 0.- c ’)C ’)~~~~ In LU — C)

1-4

CD C’)

>- C’) Li

o ~~ 0)’0 0 4 0-.0 0l.0U)U) 1.O (’-) .— U) Na) CD 4-’

S... 

4) — — — — .

~~ 4) .0 a’

~~~ _) LU N- 1.0 0L C) U ) 0 ) U )C ) 4 0 N-4 0( ’J O) CD (‘.4 — 1-4

I-

‘-4

U)

0

0. 14)

CCCC.- C~~~~~U ) ’) C ) C C’ 0(’J C’) Ui

LU (‘4 b

~ C’) CD,— C’) 11) 0) 1.0 r C’) C’) 0) 1.0 14)

5- — C) I

~

< a,

4-,

~~~ 4)

C.) _J LU C’)U)~~~~~~~~~~~ N . U) N.,—~~ J 400C)

>-

cc

U) U)

— 0) O~~0 ) — 0) U ) C’J CC 0 N- C ”(“J 4 O LU

•.~ U)C’), — .- U)C’) (’JC’) 0 , —.— U)U) C’)

LU S...

a’ 4-)

4-’ a) LU 40 1.0 .- (‘.4 40’ C’) (‘.4 U) CD .- — N- C)

4-)

4) U) 5.. 4) CC C, .,

~ ~~ CD -r, 04 ) -.- 0.

“-

_0 4)10 40 40’ 40’ 40’ 40~ 40.40 (‘4 C’) C’) C’) 40 0 4 ) 41) O C) 0 0 0 DQ CCCC) D C ~~

62

• •

~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

--

~~ -~~~~~~~~~~~~~ ::~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ T • ~~~~~~: -
-- _ -_ . - _ •__—- -__-•—-_-.-—__ --- — _~~••

~____—~

I’

S..

4)

0. C 11) 0 4).,~ U)C)CCC)—.-C”.J 0) C%J~~~ 0 4-’ 0) (.4 C’4 C’4 N- N- 4 0 0) 40 ~~ C •.- (%J C’.4 (‘4 (‘.4 — — C’) C’) C’) ‘) 10 11) ‘.1 -C 0

L) 0..

LU

o C’)

C ~~ C ~ 0)40- 40 0)1.0 1.0 C’) 1.0 - (‘.4 C) C Li

(‘4 ., U) N.. 40 1.0 N- If) 0) 0) 0) flN0

4J (fl

• 0 —‘0 00) N. 0) 0) 40’ 1.0 -

I— — C’) C’) C’) (‘.4 (‘4 ’—’ U) N-U) 0 U) ‘)

-5 40’ C N- N- C’) ‘0 0 ) .— C’) 0 N- ‘4 (”J C’) 0 LU C, .._ 

~ 1.0 IC) 41) (‘.4 CC 40’ N. — 40’ 0 1.0

— C) - •r S..

0.~ (1) 4)

Ui 0.0 C.) 0.E —

Ia.. 

U.. ~~~~~

~~ Li 0) C) C’) —— — — C’.J~~~ — 4)

C — ‘

~

t ‘0—— C N- C) — ~ ) 0)

— 0 0 LU

.0 ~~ -

~~~ 0) ‘—‘00’- N- — U) 1.0 00 ,

~~ 4) r— r. 

~ r — — —

5- ~~~ CD 0 4-’ — 0.41 I— a) .-4 -

~~~ __3

U) C

CD (‘.4 (‘.4 C’j ,- (‘.4 — — 40’ 1.0 41) .

0. ‘.4

0) LI C CC40 - C . JU ) N . . - - U) C C C\ J ~~~40- I— 0 La.)

“ ~~ ‘4 0 (“4 C’440~ 40’ U) (“4 C’) — C’)

~z ~~~ 4)

c~

D 04- C..) 0.41

a’ >- 4-I __i

CC 41) U)C) L 4 )1 .0 C C N- 0 ) .—r ‘.4

— -_ — — — C C’, lx

C)

lx

lx 41 LU U

a)

.-,

-0 10 4-’ .0~~~~ r-W ~~~~~I

~~.C

~ .-i v )0.U) v )CC~~~CCC) 41) ~~~~~~~40~~4 0404 0’ C’) 40’ U)

63

_______ ____________ • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- - - • - ~~~~~~~~~~~~~~~~~~~~~~~ 

- 

~ =-==r-~~ -~ .

~~~~ -
C0..-..-0

0..

C. .-40’.-’

4040~~~~~C)40- . a) 

0. i— 

N-. N- CC(’.J r. 

— — — .- — C’) (‘Jr C)’

In

a, UC

‘a

C) LUC0

C)

1.0C ’) QC O(

0N- 1.O

C) 1.O C.J Li

— (‘4 

(‘

4 C’) (‘.4 (“.4 11) It) U) 

C’)

C’) C’)

• + 5-. (‘.4 (I) +

— — LU 0)0) CD C’) 0) C’) CO 

.N- 1.

0

— — (‘4 C ’) r

~

- 

(‘.-4 C’) — CD

- 

C lx CD 40 ( ‘4 0) 0 4 0 ’ (‘4 0) CC1.0 LU LU C) ~~ ~~~~~~~~~~ C C) — C-~ 0 — U. 414) —

C U . 5._4

~)

— C 

~~~

4-)

0 4) o >- u~ _i CO LU C’) U) (‘4 — C) 40~ 44 U) 

—C’)

40 .

~~ U) 5- 0)I.OC’) C’) ( 0U)N -C ’) (.4 1.0

LU . - 0 ~~ C’J (’J C’.JU)C’4 C’.J Lfl U) O’i C’) S.. (“4 5--

1-4

U) •.- 4-

’ 0 .C 4) 0. I- .J Li U ) Cfl401 .OC ’) Ne’) 1.0 N- lx ,- 40 LU

I—.

C)

0 lx C CD (‘4 U) CC .-O) U) C) — LU

= 

-v S.-. ~~~ CD — — C’) — — C’) C”) 

(.0 

(‘4 C) Ca, — 0 4.1

cc

v)

-J U)

lx LU C (“4 (‘

.140 (‘

.4 C’) 0)40 (.0

0 C)) lx

lx

LU

0) ‘0 C’) 1.0 U) 

‘00,— U) CO S.- LI - 414) — (‘.4 C\J C’J C’) 40 N-U) CD C’) ( 1)41 C’) 5.. 4.1 ..- a)

LI (‘)C’) 40 C’) 40 C’

) 1.0 N. 

C’) I-— U)

4-, U

a’ 

—

.0 — 10 IDa) ~~ 0 4) 4-’ 40 • ~~ .

~~ ~~ C) _i ..J II

) >- ~~ 

0 41) ~~ 40. C’) C’) C’) U) C’) C’) I64

• • - ~~~~~~~~~~~~~~~~~~~~~~~~ 

- - 

:~~~~~~~~~~~~~~~

_-

~~~~J:~ ~~~~~~~~~~~ • .
DISTRIBUTION

AR I Distribution List

4 0480 (M&AA I 2 HOUSACDEC. Ft Ord , AIIM: Library

2 HODA (OAMI-cSZ) 1 HQUSACDEC, Ft Ord. ATTN: ATEC—EX-E—Hum Factors

1 HOOA (DAPE-PBR 2 USAEEC , Ft Benjamin Harrison , ATTN: Library

1 HODA IDAMA-AR) 1 USAPACDC, Ft Benjamin Harrison , AIIM: AICP—HR

1 HQDA (DAP E-HRE-POI 1 USA Comm—Elect Sch. Ft Monmoutts , ATTM:ATSN--EA

% HQE5A,

~SGRQ-lfl~ t USAEC ,Ftteonmouth,AT’TN: AMSEL-CT-HDP

1 HOQA DAMI-DOT-CI 1 USAEC , Ft Monrr south . AIIM: AMSEL—PA—P

1 HODA IDAPC-PMZ-A) 1 USAEC , Ft Monmouth , AIIM: AMSEL--Sl--CB

I HODA IDACH PPZ-A) 1 USAEC , Fr Monmouth , AIIM: C, Fact Dcv Br

1 HODA (DAPE HRE) 1 USA Materials Sys Anal Agcy , Aberdeen, AIIM: AMXSY—P

1 HQDA IDAPF MPO-C) 1 Edgewood Ars enal . Aberdeen, ATTN: SAREA—BL—H

1 41004 (DARE-OWl 1 USA Ord Ctr & Sch, Aberdeen , AIIM: ATSL-TEM—C

1 HOPA IDAPE•HRLI 2 USA Hum Engr Lab . Aberdeen . A T’TN: Library/Dir

1 HODA IDAPE.CPS ) 1 USA Combat Arms Ing Bd , Ft Benning, ATTN: Ad Supervisor

1 14004 lOAF 0 MFA) 1 USA Infantry Hum Rsch Unit , Ft Ben riin g, AIIM: Chief

1 HODA IDARD-ARS .P) 1 USA Infantry Bd . Ft Benning , ATTN STEBC—TE -T

1 HODA IDAPC-PAS-A) 1 USASMA . Ft Bliss , ATTN: ATSS- LRC

1 HODA (DUSA-ORI I USA Air Def Sch , Ft Bliss . ATTN: ATSA d O ME

1 14004 IOAMO -ROR) 1 USA Air Del Sch, Ft BIns , ATTN: Tech Lib

1 41004 (DASG) 1 USA Air Del Sd. Ft Bliss , ATTN: FILES

1 41004 (DM0-RI) I USA Air Del Sd, Ft Bliss , ATTN: STEBO—PO

1 Ch ,,. Consult Div )DA-OTSG), Adelphi, MD 1 USA Cmd & General StI College , Ft Leavenworth , AIIM: Lib

I Md Ant. Hum Re,, 000R&E . 040 IE&LS) 1 USA Cmd & General Stf College , Ft Leavenworth, AIIM: ATSW—SE—L

1 410 USARAL . APO Seattle , AIIM: ARAGP-R 1 USA Crnd & General Sti College , Ft Leavenworth, ATTN: Ed Advisor

I HO F ist Army, ATTN : AFKA-0I Tl I USA Corntiineil Arms Cmlrt Dcv Act , Ft Leavenworth , AIIM: DepCdf

2 HO Filth Army, Ft Sam Houston 1 USA Combined Arms Cmbt Dcv Act . Ft Leavenw orth , ATIN: CCS

1 Dir , Army Sri Stud ies Ofc , AIIM: OAVCSA (DSP ) I USA Comb ined Arms Cmbt Dcv Act , Ft Leaven wor th, ATTN: ATCASA

I Ok Chief of Sri . Studies Ofc 1 USA Combined Arms Cmbt Dec Act . Ft Leavenworth, AIIM: ATCACO—E

I DCSPER . ATTN CPS/OCP 1 USA Combined Arm s Cmht Des Act , Ft Leavenwor th , AIIM: ATCACC-Cl

I The Army Lib , Pentag on , ATTN: RSB Chief 1 USAECOM . Nig ht Vision Lab . Ft Bc’lvo i r . AIIM: AMSEL— NV—SD

1 The Army Lib . Pentagon . ATTN: ANRAL 3 USA Computer Sys Cmd , Ft Belvoi r , AIIM: Tech Library

1 Ofc . Ast t Sect of the Army (R&D) 1 USAMERDC . Ft Belvoir . ATTN STSFB-DO

1 Tech Support Ofc . OJCS 1 USA Eng Sch . Ft Belvoir . ATTN: Lib rary

I USASA . Arlington , AIIM: lARD-I 1 USA Topographic Lab , Ft Belvo ir , AI1’N: ETL -TO—S

I USA Rsc h 01⁄4. Durham , ATTN Life Sciences Dir 1 USA Topog raphic Lab . Ft Belvoir . ATTN: STINFO Center

2 USARIEM , Natick , AIIM: SGRD-UE-CA I USA Topographic Lab , Ft Belvorr . ATTM : ETL—GSL.

1 USATIC . Ft Clayton , ATTN STEIC-M0.A 1 USA Intelligence Ctr & Sch , El Huachuca , AIIM: CTD— MS

I USAIMA , Ft Bragg, AIIM: ATSU-CTD-OM 1 USA Intelligence Ctr & Sch. Ft Huachuca , AIIN: AIS—CTD—MS

1 USAIMA , Ft Bragg , ATTN Marquat Lib 1 USA Intellig ence Ctr & Sch , Ft Huachuce. AITN: ATS I—TE

1 US WAC Ct. & Sch, Ft McClellan , AIIM: Lib 1 USA Intelligence Ctr & Sch , Ft Huach uca , ATTM: ATSI—TEX—GS

1 US WAC Cr, & Sch, Ft McClellan , AIIM: Tng Dir 1 USA Intelligence Ctr & Sch . Ft Huachuc a , AIIM: AISI-.CTS—OR

1 USA Quartermaster Sch . Ft Lee . ATTN: ATSM•TE 1 USA Intelligence Ctr & Sch, Ft Huachuca , AIIN: ATSI—CID—DI

1 Intellig ence Material 0ev Ofc , EWL . Ft 1-lolabird 1 USA Intel lige nce Ctr & Sch , Ft Huachuca . AIIM: AIS$—CTD —CS

1 USA SE Signal Sch, Ft Gordon , AIIM: AISO-EA I USA Intelligence Ctr & Sch , Ft Huachuca , AIIM: DASISRD

1 USA Chaplain Ct r & Sch , Ft Hamilton , AIIN: AISC-TE.RD 1 USA Intelligence Ctr & Sch , Ft Huachuca , AIIM: AISI—IEM

1 USATSCH Fr Eusti s , ATTN: Educ Advisor 1 USA Intelligence Ctr & Sch . Ft Huachuca , AIIM: Library

1 USA War College . Carlisle Barr acks. AIIN: Lb 1 CDR . HO Ft Huachuca , AIIM Tech Ref Div

2 WRAIR . Neurop sychuatr y Div 2 CDR . USA Electronic Pry9 Grd , AIIM: STEEP—MT—S

1 DLI . SDA . Monterey 1 CDR . Project MASSTER , AIIM: Tech Info Center

1 USA Concept Anal Agcy . Bethesda , AIIM: MOCA-WGC 1 Hg MASSTER , USATRADOC , LMO

1 USA Concept An&I 4gev . Bethesda, AITh: MOCAYR RetPltt )’i lIflti%iit~, 540 MASSTER . El Hood

1 USA Concept Anal 4gev , Bethesd a , AIIM: MOCA-JF I USA Recruiting Cmd , Ft Sherdian , ATTM: USARC RM—P

I USA Artic Test Ctr , APO Seattle , AIIM: STEAC-MO-ASL Senior Army Adv ., USAFAGOD/TAC . Elgin AF Au . FId Mo. 9

1 LISA A ’ric Tes t dIr , APO Seattle , AIIM: AMSTE-PL.IS 1 HO USARPAC . DCSPER , APO SF 96558, AIIM: GPPE— SE

1 USA Armament Cmd, Redstone Arsenal , AIIM: AISK-TEM 1 Stimson Lib , Academy of Health Sciences . Ft Sam Houston

L 1 USA Armament Cmd, Rock Island. AIIM: AMSAR-TDC ¶ Marine Corps Inst ., AIIM: D,a,i—MCI

I FAA MAFEC . Atlantic City , AIIM: Library I HOUSMC. Commandant , AIIN: Code MIMI 51

1 FAA NAEEC , Atlantic City , ATIM: Hum Engr Br 1 HQUSMC, Ci’mmandant , ATT~ I . Coda MPI—20

I FAA Aeronautical Ct’, Oklahoma City , AIIM: AAC 440 2 USCG Academy, Mew Loiid oir , AIIM: Admi ss ion

2 USA FId Arty Sch, Fl Sill , ATIN: Libra ry 2 USCG Academy, Mew London , AIIM: Library

F 1 USA Armor Sth . Ft Knos , AIIM: Library 1 USCG Traini ng Ctr . NY , AIIM: CO

1 USA Armor Sch . Ft Knox , AIIM: AISB-Dl.E 1 USCG Irainiri g Ctr , MY . AIIN: Educ Svc Ofc

I USA Armor Sch , Fr Knox , AIIM: AISB-DT-IP 1 USCG. Psychol Res Br , DC, AIIM: GP 1/62

I USA Armo r Sch . Fr Knox . AIIM: AISB.CD-AD I HO Mid-Range Br . MC Dct. Ouantico , AIIM: P&S Div

65

• ‘ L .l . •• •
1 US Marine Corps Licisson (Sic , AMC , Alex6 ndria , AIIM: AMCGS—F 1 0.1 & Civil Inst of Enviro Medicine. Canada

I USATRADOC . Ft Monroe . ATTN AIRO -ED I AIR CRESS, Kensington, AIIM: Info Sys Br

6 USATRADOC . Ft Monroe , AT’TN: ATPR -AD I M,IitIS!P$yI5 DIO9I$k Tiasseste , Copetsagen

I USAIRAOOC. Ft Monro e , AIIM: AIIS-EA I Milit ary Attac he. French Embassy, AIIM: Doc Sec

1 USA Forces Cmd. Ft McPherson, AIIN: Library I Medecin Chef , C.E.R.P.A.—Arsenal, Ioulon/Naval France

2 USA Aviation lest Sd, Ft Rucker . AIIN: STEBG—P0 I Prin Scientific Off , App ) Hum Engr Rsch Div . Ministry

1 USA Agcy for Aviation Safety . Ft Rucker , AIIM: Library of Defense. New Delhi

1 USA Agcy for Aviation Safety. Ft Rucker . AIIN: Educ Advisor I Pert Rach Ofc Library. AKA , Israel Defense Forces

1 USA Aviation Sch. Ft Rucker. AIIM’ Pt) Drawer o I Mirristeris van Defens i., 000P(KL Aid Soci ul

1 HOUSA Aviation Syc Crnd . St Louis. AIIM: AMSAV—ZDR Psycholoquscfre Zaken, The Hague, Netherlands

2 USA Av iadon Sya Test Act.. Edwards AFB, AIIN: SAVIE—T

¶ USA Ar Del Sch. Ft Bliss. AIIM: ATSA 1CM

1 USA Air Mob ility Rsch & Dcv Lab , MoEfett RId , AIIM: SAVDL—AS

1 USA Aviat ion Sds . Fe~ Tog Myt. Ft Rucker , AIIM: AISI—T—RTM

1 USA Aviation Scit . CO. Fr Rucker . AIIM: 4151—0—A

1 HO, DARCOM , Alexandr ia . AIIM: AMXCO—TL

1 HO, DARCOM , Alexan dr ia . AIIM: CDR

1 US Military Academy, West Point, AIIM: Serials Unit

1 US Military Academy, West Point . AIIM: Ofc of Milt Ldrshp

1 US Mil-tary Academy. West Point , AIIM: MAOR

1 USA Sta ndardi r stin ri Gp, UK. FPO NY . AIIM: MAS E—GC

1 Ofc of Naxal Rsch, Arlington. AIIM: Code 452

3 Ofc of Naval Rseh. Arl in gton . AIIM: Code 458

1 Ofc oI Navel Bsch , Arlington, AIIM: Code 450

1 Ofc of N ac~ Rsch , Arlington . AIIM: Code 441

1 Naval Aerovpc Med Re, Lab , Pensacola . AIIM: Acous Sch Div

1 Naval Aerospc Med Re, Lab, Pensacola , AIIM: Code LSI

1 Naval Aerorpc Med Res Lab. Pensacola , AIIM: Code L5

1 Chipl of Mav Pers . AIIM: Pers -OR

1 NAVAIRSTA . Norfo k , ATIN: Safety Crr

1 Mae Oceanographic . DC. AIIM: Code 6251 , Charts & Tech

1 Center oI Naval An al , AIIM . Doe Crr

1 Nav AirSys dom , AIIM: A IR—53 1 3C

1 Nay BuMed , AIIN: 7 1 1

I NavHelicopterSubSoua 2. FPO SF 96601

I AFHFI L (Fl) Will iam AFB

I AFI-4RL )I1’l Lowry AFB

I AFHRL (AS) WPAFB , OH

2 AFHP.L (DOJZ) Brooks AFB

I A FHRL IDOJN ( Lackland AFB

I HOUSAF (INYSDI

¶ HOUSAF )DPXXAI

I AFVTG IROl Randolph AFB

3 AMRL (HE) WPAFB , OH

2 AF Inst of Tech , WPAFB , OH. AIIM: ENEIS L

¶ AIC (XPIDI Randolph AFB

I USAF ArrroMcd Lb Brooks AFO (StJL—4), AITN: DOC SEC

I AFOSR (Nil . Arlington

I AF Log Cmii, McClellan AF8. AIIM: ALC/DPCRB

1 Air Force Academy, CO. AIIM: Dept o Bel Scn

B NavPers & Dcv Ctr , San Diego

2 Navy Med Neuropsychiatric Rtch Unit , San Diego

1 Nay Electronic Lab , San Diego , AIIM: Ret Lab

1 Mae Trni)Cr.t . San Diego , A TTN: Code 9000—LiD

I NavPostGraSch , Monterv’i , AIIM: Code SSAa

1 NavPostGr aSch , Monterey . AIIM: Code 2124

1 NavlrogEqi ipdtr . Oiianoo , AIIM: Tech Lib

1 US Dept nI Labor , CC. AIIM: Mar poe.rr 46mm

1 US Dept of Justic e , DC, ATT~4. Drug Enfor ce Admin

1 Mat Bur oI Standardv . ()C, AT TN’ Computer Info Section

1 Mat Clearing House for MEl—Info , Rock eille

1 Denver Federal Ctr, Lake,vood, AIIM, BLM

12 Defense I)ocurrsentation Center

4 Dir Psych. Army Hq, Russell Ofcs. Canberra

1 Scientific Ady,r, Mil Bd, Army Hq Russell Ofcs . Canbeera

I Mil and Air Attache, Austrian Embassy

1 Centre dir Recherche Des Facteurs , Hu mane de (a Defense

Natronale , Brussels

2 Canadian Joi nt Stall Washi ngton

I C/Air Staff . Royal Canadian AF , AIIM: Per. Std Anal Br

3 Chief , Canadian Del Rsch Staff , AIIM: CICRDS(W)

4 Britith Del Staff . Briti sh Embassy, Washington

66

L.. . . - . . . . . . • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~