V ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

-

~

V 

. ~~~~~~~~~

@1EVEL Technical Paper 304 ~_____________ ___

~~ 

THE VA LUE OF SPECIAL TRAINING AND JOB

AIDS FOR IMPROVING UNATTENDED GROUND

SENSOR OPERATOR PERFORMANCE

p

~~~ Sterling Pilette and Bill Biggs HRB-Singer, Inc.

and

LJ.J Harold Martinek

Army Research Institute for the Behavioral and Social Sciences

i

~

i

•

~

• ~~Lk

—~~~~~ BATTLEFIELD INFORMATION SYSTEMS TECHNICAL AREA

DDO~

NOV 9 1978

_______ 

Ijh2iT~

5UU 15

Ii~

.: i

~

irJJ 

Research Institute for the 

U. S. 

Behavioral 

Army

and 

~~

Social Sciences

August 1978

App,ovs d lo, public r•l.u.. disi,,but ion unh,mIh.d.

~

;1 025
-

U. S. ARMY RESEARC H INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCE S

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WILLIAM 1. HAIJSER

JOSEPH ZEIDNER Colonel, US Army

Technical (

~rector (Designate) Commander

Research accomplished under contract

to the Department of the Army

HRB—Stnger , Incorporated

NOTICES

DISTRIIUT~ON Primar y distribu tion of th is report his b u n m*

~~ by ARI. Pleas. .dd’sss cOrrespondIflCe

~~

ncsrnutI distribut’sn of ‘sports to. U. S. Army RIes rch Init,t ute for thu IShivior.l snd Social Sciencas . ATTN PERI.P 5001 Esw nhoss., Avsnu. . Ales.nd,i.. Virgi ni. 22333

FINAL DI~~OSiTi~~~ This ‘sport ‘n.y be dsstroy,d whsn it is no longer ns dsd. Pl.~is do not return Il tO

s it. U S. Anity ~sis.,ch I nstitute for tiss Siitiwsor& md Social $ci.ncss

~~IL lit. findi ngs in this ‘sport .rc not t

~ be ceasirusd as en of fid el Department of thu Army position , unless so designated by other suffierised docusepats,
9 :

I

~~nt pmmIfimti SECURITY CLASSIFICATION OF THIS PAGE (lThan Data Ent.t.d) ___________________________________

i

~~~~~~ini.l ~~ kI’~~ READ INSTRUCTION S

~~~~~~ I Ft I 

~~ I IUI~ !~ BEFORE COMPLETING FORM

I. REPORT NUMBER 2. GOVT ACCESSION NO. 3 ENT ’5 CATALOG NUMBER

Technical Paper 4 J

f_i b

~ r

~~~~~~~&.i.:_~ .. .- . J ?

~~Rf~~

’

~~

ERIOcf COV ERED

(

~~ ) / ~~E VALUE O~~,

~ PECIAL ~‘RAINING AND JOB AIDS FOR! Interim $a~~~ t. I

\. ~~~~I

~)VING ~ 1NATTENDED 3N)UND SENSO~~ OPERATOR j Nov 72-—May ~~

)

~~~~~~~~RMANCE . I ~~~~

_. 1*nu1.t I

~~~..

3si’

~~R! NUMBER

I

~~~~~~ TRAC T OR GRANT NUMBER(a)

ster1in9/~1ilette , Bill,

~ ig~ s~

PuIRB Singer, Inc.;

‘

—

~

/ Haroldftartinekfr ~~ 9-73—C-,~d24

9. PERFORMING ORGAN IZAT ION NAM E AND ADDRESS ¶0. PROGRAM EL EMENT. PROJECT . TASK

MRS singer, mci ~

4 ).

... AREA~~~WORK_

UNIT NuMSERS

P.O. Box 60 / lx763723Dl~ l

Science Park 206 1 State College, Pennsylvania 16801 ___________________________

II. CONTROLLIN G OFFICE NAME AND ADDRESS , _ J7. 

U.S ~~~~~~~~~ . Army Assistant Chief of Staff for (/1 Aug~~~ a78 /

Intelligence \....-~ .1*. ~~~~~~~~~~~~~~~~~~

Washington , DC 20310 _________________________

14. MONITORING AGENCY NAME C AODRESS(If dlSf,r.n I from Confr oHI ng OWc.) IS. SECURITY CLASS. (of fbi. r,porl)

U.S. Army Research Institute for th Behavioral

and Social Sciences (PERI— OS) 

~ 

/ Unclassified

5001 Eisenhower Avenue // I5(-~

Q I is.. OECLASSIFICATION/ DOW NGRAO ING

Alexandria, VA 22333 1T_~

-I 

SCHEDULE

IS. DISTRIBUTION STATEM ENT (of fbi. R.porf)

Approved for public release; distribution unlimited.

¶ 7 . DISTRIBUTtON STATEMENT (of IA. ab.tracf .nt.r.d in Block 20. If dIff.r. ,,I from R.p ort)

IL SUPPLEMENTARY NOTES

IS. KEY WORDS (Conf ine. on ,.v.r.. .id. if n.c..wy ned id.r,tify by block numb.,)

Unattended Ground Sensors Target Acquisition

• Seismic Sensors Error analysis

String size

N, Training

Surveillance

20. S RAC I (Conf ine. on r•v•r•• .id. If n.c.a..ry aid id.nUfy by block numb.r)

Requirement: To determine the value of unattended ground sensor operator

training and job aids derived from an analysis of errors and to determine the

effect on operator performance of target-activity level and number of sensors

used in a string . ~

Procedure : The training program and job aids were evaluated using the

pretest and posttest design with 4 hours of scenarios containing typical target patterns at two levels of target activity generated by 2— , 3— , and 4— —

DO ~ J473 EDITION OF I NOV 65 1$ OBSOLETE Unclassified /

~ SECURI1Y CLA SSIFIC ATION OF THIS PAGE (W~i.n Date Enf...d)
__________ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - •

,-~~~ -—- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unclassified

sIcM~~~TV CI.MUIF$CA1ION OP Tst $ PA I4~~~ Die. 

~~—.q

\

~:sor strings . ~~.ratora detected and identified targets and estimated

their speed and number. ~

‘

~~kesu1tsi The individualized training program resulted in signif i-

• cantly improved operator interpretation performance in target detection

right., identification rights , target speed , ajId target quantity estimation. Use of the lob aid (nomograph) significantly improved performance

in identification right., target speed estimation , target quantity estimat ion , and reporting time. Student acceptance of the individualized

training approach and both job aids was high . Operator performance on

3- and 4—sensor strings was 77% detection completeness with virtually

no false alarms. Detection of targets was better during low-target

activity than durin~j high-target activity. The use of three sensors in

a string resulted in the same operator performance as the use of four

sensors.

it. ’

5. ~~• .I

~B U ‘1 ~~~~~~~~~ ci i

. .’ , ,; 

~ -

‘ 

“

~ .•~~~

; • , .

Unclassified

SICUNI tY Cb AS$IFICATION OP THIS PAQ((Vitne Oat. 

~~~~~~~ I , I

~ 

~~~~~~~~~~~~ - ~~~~~~~~~~~~~ .

~~ • -

~~ - ._- - -

~~~~~- r ‘ - ‘. 4
____ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . - - .,,

- 

.~ Technical Paper 304

THE VALUE OF SPECIAL TRAINING AND JOB

AIDS FOR IMPROVING UNATTENDED GROUND

SENSOR OPERATOR PERFORMANCE

Sterling Pilette and Bill Biggs HRB.Slnger, Inc.

and

Harold Martinek

Army Research Institute for the Behavioral and Social Sciences

BATTLEFIELD INFORMATION SYSTEMS TECHNICAL AREA

Submitted as complete and Approved By:

techn ically accurate , by:

Edgar N. Johnson AH. Blrnbaum, Acting DIrector

Technical Area Chief ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

Joseph Zoidner

TECHNICAL DIRECTOR (DESIGNATE)

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office . Deputy Chief of Staff for Personnel

Department ol the Army

August1978

Army Project Number Intelligence Systems

20062101A754

Approved for pubis rsleuse; distri bu tion unlimited .

- -

~~~~-•- . ,..~~ -~~~~~~~ - -.* -- -—
--

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form. 

--- - - 

. - - -

~ 

j
~ ‘ 

~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ P.

FOREWORD ____________

The Battlefield Information Systems Technical Area is concerned

with the demands of the future battlefield for increased man-mach ine

complexity to acquire , transmit, process, disseminate, and use information . The research focuses on the interface problems and interactions

within cosetand and control centers and deals with such areas as topographic products and procedures tactical syutboloqy , information manaqement , user—oriented systems, staff operations and procedur es, and sensor

systems integration and utilization .

One area of special interest is that of huma n factors problems of

the presentation and interpretation of surveillance and tarqet acquisition information. One relatively new source of intelliqence information

is remote monitoring of the battlefield usinq seismic, acoustic , and

maqnetic unattended qround sensors (Tics) . When these remote sensors

are activated by enemy personnel or vehicle movement, a monitor display

• located behind our lines indicates the activity . The operator can derive from this display not only the presence of the enemy but such information as the direction and speed of convoys and personnel , the

number of vehicles in a convoy , and the composition of the convoy , e.g .,

armored versus wheeled vehicles . The present publication discusses the

development and evaluation of a training test and job aids which sig—

nificantly improve operator performance by correctinq known deficiencies .

In addition, the number of sensors in a strinq and level of target

activity was varied in the research to determine their effects on operator performance .

Research in the area of sensor systems integration and utilization

is conducted as an in-house effort auqmented through contracts with organizations selected for their unique capabilities and facilities for

research on sensor systems . The present study was conducted with personnel from HRB Singer , Inc . under contract DAHC l9-73—C-0024 with pro—

gram direction from Mr. Cecil 0. Johnson . The effort is responsive to

requirements of Army Project 2Q662704A721 and to special requirements

of the Assistant Chief of Staff for Intelligence .

The cooperation of the participatinq personnel at Fort Bragg , N.C.,

is appreciated, especially that of Maj. Gronich . A special note of

thanks is due to Sqt. 0. Chiodini , chief of the Read-out Section of the

UGS Division, U.S. Army Intelligence Center and School (USAICS), Fort

Huachuca, Ariz ., for his expert technical assistance .

E ZEIDN~W’

chn cal flirector (Designate )
- • •- —-- -_--••~~ -,-- —-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~ —:-- 

~~~~~~~~~~~~~~~~~~~~~ — _

~

•

~

--_ “ _

~

-

_ -- .- _ -• _ _ - _ _ •---• 

_ • • P

THE VALUE OF SPECIAL TRAINING AND JOB AIDS FOR IMPROVING UNATTENDED

GROUND SENSOR OPE RATOR PERFORMANCE

BRIEF

Requirement :

To determine the value of unattended ground sensor operator training and job aids derived from an analysis of error sources and the effect on opera tor performance of target activity level and numbers of

sensors used in a string.

Procedure :

Based on an analysis of operator errors made in a previous study,

a training program and two job aids were developed . To test the value

of the training and one of the job aids (measuring device) , two 2—hour

scenarios were constructed for pre- and posttraining evaluation . Typical target patterns at two levels of target activity and three levels

of sensor string size were systematically varied within 30—minute segments. Authentic fixed—wing and helicopter activity, artillery shell

bursts, and random noise were included to simulate operational, nontarget activations . The second job aid, a nomograph , was evaluated using

the pretest and posttest design . Two special tests requiring only the

measurement and computations necessary for estimates of speed and enemy

number(s) we re developed for this purpose .

Twenty school-trained Army enlisted men (TiCS (unattended ground

sensor) operators), were given test procedure training and a short refresher in UGS interpretation . Two 10—man groups were formed and each

was given different scenarios for the pretest; these we re then switched

for the posttest. The training program—-given between the pretest and

poettest——stressed individualized instruction including self-pacing ,

immediate feedback, expert assistance when needed, and guaranteed

student—mastery using criterion testing. One job aid, an UGS ruler

for accurately measuring the length of activation patterns, was part

of the training exercise. The other job aid (a nomograph), which had

been developed to simplify arithmetic calculations and decimal point

placement, was tested separately after the above traininq and posttest.

Findings:

The individualized training program resulted in significantly improved operator interpretation performance in target detection rights-,
- ‘.~~~ -_•.~~~ ‘

~ ——-~--~— ~~~- _ _---•-~ .-~ ,. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~- ‘ _—_-,

~

-— —--

~

—--

identification rights, target speed, and target quantity estimation.

Use of the nomoqraph significantly improved performance in identification rights , target speed estimation , target quantity estimation , and

reporting time . Student acceptance of the individualized training approach and of both job aids was high . Operator performance on 3- and

4—sensor strings was 77% detection completeness with virtually no false

alarms . Detection of targets was better during low target activity

than during high target activity . Use of three sensors in a string

resulted in the same operator performance as use of four sensor’ .

Utilization of Findings:

The lesson materials together with the individualized trainina approach should be used to provide review and on— the-job training to operational field personnel and should also be integrated into the TiCS

course at USAICS , Fort Huachuca , Ariz .

The nomograph should be included as standard issue with operational event recorders and should be tauctht at TiCS course USAICS and

given to each graduate to take with him to his assigned unit. A creditcard size TiCS ruler should be issued to everyone involved in monitoring

TiCS target activation .

To increase timeliness, the tiCS operator should send forward a

detection report based on two sensors (or more, if doubt exists) with

a followup report giving target type , speed , and number.

A research study controlling on target difficulty and comparing

2 versus 3 versus 4—sensor strings should be done to provide data for

determining the most efficient string size .

During high—target—activity conditions, operator reports should

be considered as greater underestimates than during low—target—activity

periods.
- • -— -——.--fl- ~

_ — -fl- ~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~

f

THE VALUE OF SPECIAL TRAININ G AND JOB AIDS FOR IMPROVING UNATTENDE D

GROUND SENSOR OPERATO R PERFORMANCE

CONTENTS

Paqe

BACKGROUND 1

OBJECTIVES 2

METHOD 2

Population and Sample 2

Apparatus 2

Training Requirements 2

Development of the Training Materials and Job Aids 3

Design of the Training System 5

Development of the Test Scenarios 8

Training Validation Design . 

• 8

Test Procedure 11

RESULTS AND DISCUSSION 13

Overall Performance Measures 13

Measurement and Computational Procedures 26

Within—Course Performance 28

Student Questionnaire Results 30

Job Aid Performance Measures 35

CONCLUSIONS AND RECOMMENDATIONS 39

Conclusions 39

Recommendations 41

REFERENCES 43

APPENDIX A. LESSON--TWO-SENSOR STRINGS AND FORMULA REVIEW . . 45

B. SUBJECT INSTRUCTIONS 49

C. ASSESSMENT OF PRACTICE EFFECTS IN

H INDIVIDUALIZED TRAINING 57

DISTRIBUTION 59

_____ - -~~~~ -

• •~~~~~ ~~~~~i••~~~~ • _____________
- - • - ___

LIST OF TABLES

Tab le 1. Analysis of variance table for de tection

riqhtc 14

2. Mean number of detection rights by session and

sensor string size 15

3. Mean number of detection rights and comp1eteness

~~ session and target activity 16

4. Mean number of detection rights and completeness

by strinq size and target activity 17

5. Mean number of detection rights by scenario and

target activity 18

6. Mean number of detection rights by string size ,

scenario , and target activity 18

7. Analysis of variance table for identification

rights 20

8. Mean number of identification righ ts by session

and strinq size 21

9. Percentage identification completeness by session

and string size 21

10. Mean number of identification riqhts by session

and taraet activity 22

11. Percentage identi fication rights completeness by

session and target activity 23

12. Mean number cf identi fication rights by string

size and target activity 23

13. Mean number of identification rights by scenario

and target activity 24

14. Mean number of identification rights by string

size , scenario , and target activity 24

15. Analysis of variance table for confidence values 25

16. Analysis of variance table for reporting time 26

17. Analysis of variance table for TT1 values 27

~

------•----• - - - • - - -- - - ••--~~~~~ -~~ -----•--—- 

--i
— - - - H ’ -

~~~~~ - - ~~~

_

~~~

_

~~~~~ ‘ ~~~~~~~~~~~~~~~~~~~~~~~~~~

LIST OF TABLES (continued)

1.8. Analysis of variance table for TM values 28

19. Analysis of variance table for target speed

values 29

20. Analysis of variance table for quantity values 29

21. Data management sheet of criterion exercise

recycling 31

22. Operator ratings of clarity and difficult’.’ on

lesson materials 34

23. Effect of UCS ruler and nomograph upon target

4 variables 36

24. Number right of first half versus second half

of test on speed and quantity estimations 38

25. Average time for first half of test versus

second half of test 38

c—i . Detection and Identification Rights 57

LIST OF FIGURES

Figure 1. Job aids--UGS ruler combined with nomograph 6

2. Test administration schedule 12

3. Reporting time by item 39

_________ 

______ 

I • • ar—--- -

•

~

• • -. . - - - -- - -~~~ -•

~~~ - - -

—~~ - :

~ r~~ _ - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ -
r

THE VALVE OF SPECIAL TRAINING AND JOB AIDS FOR

IMPROVING UNATTENDED GROUND SENSOR

OPERA TOR PERFORMANCE

BACKGROUND

As discussed in the ST 30-2C-1 Army manual (U.S. ArmY , 1971) , in

F September 1966 the Secretary of Defense directed the armed forces to

develop an infiltration interdiction capability to assess vehicle and

personnel flow in Southeast Asia. The anti-infiltration capabili ty that

was developed centered on a group of remotely monitored electronic sensor devices called unattended ground sensors (tics) .

UGS provides the Army ’s battlefield surveillance and tarqvt acquisition system with another versatile capability , that of obtaining rv.~1-

time information about enemy movement . Most TJCS in use today are seismic, others are magnetic , acoustic, electromagnetic , and passive

— infrared . They can be hand implanted , air implaced , or mortar and

artillery delivered . tJGS systems are recognized as effective battlefield assets, and they are expected to become an important source of

combat intelligence for many situations .

Even though LJCSS have already demonstrated their value in operational and test situations (U.S. Army , 1971 and 1972), their informa —

tion potential is far greater than has been realized heretofore . Con—

trolied tests show that varying deqrees of misleading and inaccurate

information are being supplied by both the sensor equipment and human

operators (Martinek , Pilette, and Biqqs , 1978; U.S. Army , 1970; and

West, 1973). Appropriate research and development can minimi ze such

deficiencies .

Systematic field experimentation shows that TiCS can be unreliable

at times . One reason is lack of understanding and anticipation of environinental influences upon sensor response characteristics. For example, in a field test conducted under dry—season terrain conditions ,

considerable site-to-site variation occurred in seismic response characteristics due to soil surface conditions, vegetation ground covers,

and background noises (West, 1973).

Variable sensor response characteristics make the job more difficult for the sensor opera tor , and human error has definitely contributed

to inaccurate and tnisleadinq information output. The problem is to

isolate techniques that will reduce human error in tIG S target reportinq .

Although a number of diverse factors may make tiCS operator performance

less than optimal , the more prominent problems can be attacked directly

through specialized training, operator techniques , and job aids .

— _t.~~_ ~~~~WM~~~~ r — ~~~~ -_ _
fl__ S fl~ f lfl~f l _•

~~• ~~f l_V~~~fl ~~~~~~~~~~~~~~~~~~ ~~~~~~ _

~~~~~ •~~ f lfl ~

•fl~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~ ~~~~~ ~~~~~flflflfl•fl ~~~~~~~~~~~ •flfl~~~~~~ ~~~~~~~~~~~~~~~~~~ - ~~~•

ORJFC’PIVES

• 1. To identify roblem area~ experienced by ttCS operators , develop ~~ tali:e~

i tralnin ; materials to control these difficulties , and empirtt’afly demonstrate the effectiveness of

such training .

~~~

. To determine if two j ob aids (t~

omoqraph and IICS ruler) can

- 

- improve the measurements and computations of tW’iS monitors .

1. To decrease the tim e needed for ttGS reporting .

• 4. To determine parameters of erformance under several operat tonal ioi ~dt t ions

5. To determine the changes in performance due to the number of

sensors used In a strinq .

Mr’r

~

ion

1

~~

tion and~~~~~~le

The population of concern is the tICS operator ( MOS 17M20) who has

been schoo l trained at the U.S. Aru~y Intolli~ enc e Center and School

(USArcS) , Fort Ifuachuca, An t. Twenty en1isted personnel MOS 17M20

of the 82nd Airborne Division stationed at Fort Bragg , N .C., served

as operators . Although they had been in the field for several months,

they had not i nterpreted tiCS activations using the operational dieplay (thy R0376 event recorder) .

$ 

Ap~~ratus

Ten modified Esterline Angus Chart Inspectors were ue~d to display

th. pretest anti poettest scenarios (R0376 event recorder plots) . Fach

was mechanically adjusted to drive the plots at the same speed (12

inches per hour) as the RO37~ event recorder (U.S. Army, IQ6Q). A

display housi ng was placed on each mechanism to reduce Its vI.win~ area to that of the R~~

l7(

~. Thus, the test scenarios were presented

H at the same speed and format as in a field situation .

TEa tnt ntj R*~~ iremonts

R~vit~

w. The first step In the identification of problem areas in

(‘CS monitor performance involved .i review of’ tralnin o beinn conducted

at Fort Huachuc’s . Training manua l documentatIon was reviewed (U.S.

Army , 1Q71, l

~

17.’) ,

~n4 interviews were held with senior Ins tructors to

identify specific areas of substandard performance . Student performance

— ~~~~~~~~~~~~~~~~~~~~~~ — - - -- - 4l~~
•~~~ —~~~~~~~- - ~~~~ --~~~~~~~-—

data , collected during an earlier experiment (Marttn”k , 1’i l e ttt’ , and

Riggs , 1978) , were analyzed in an effort to document and categor i i-v

factors leading to human error in th S monitor performance . Based at:

these efforts, three major problem areas were identified .

Target Detection and Id~ ntificat ion . This category deals wi th the

operator’s ability to detect and identify targets . The operators htected fewer targets when these targets activated .-sensor strings

as compared to 1- and 4-sensor strings . Also , lower detection completeness (detection rights divided by total number of targets) occurred during high target activity as compared to low target activi ty .

Also, approximately 20% of the targets detect cci were not- identi fied

corr ectly as to vehicle or personnel. Following the pattern establ i shed

by the detection results , fewer targets were identified when they were

acti vated with 2—sensor st-rings and during high target. activity .

Column—Length Formula Calculations (see Appendix A ) . This cate-

-

~ gory deals with the ability of the operator to use the column-length

formula with target patterns that he has detected . A sensor activation

pattern is composed of short horizontal lines occurring within a specified time interval (determined by a sensor ’s inhibit time) . They arc

displayed on the appropriate channel of the plot paper which serves as

the readout sheet for the operator. The formula provides an estimated

target speed and number of targets in a convoy . Some specific factors

noted were (1) incorrect placement of the endpoints of sensor activation patterns for measuring TT1 and TM (see paqe 10 for definition) ;

(2) inaccurate estimations of TT1 and TM measurements, even if the

activation end—points are not ambiguous ; (1) arithmetic errors in

multiplication, division , and decimal point placement ; and (4) confusion in computing the combined detection ranae (CDR) value and in

selecting the arithmetic function to use with it.

Nonseismic and Confirmatory Sensors. This category deals with the

inability of the operator to optimally use information presented by the

followinq sensors when these sensors were part of a deployed sensor

string : electromagnetic intrusion detector (5Mm ), magnetic intrusion

detector (MAGID) , and directional infrared intrusion detector (DIRID) .

Development of Training Materials and Job Aids

Although a number of training requirements could be satisfied by

the development of content in specific subject matter areas , analysis

of training requirements revealed a need for -job aids that would assist

the UGS monitor in making measurements and calculations relating to the

column— length formula.

3

_ -

- -- - - 

_

~~- - ~~~fltSSj. ~~~~~~~~~ .
- ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following content lessons were outlined and developed .

a. Lesson 1——UGS Ruler (training in the use of this job aid) .

Objective--To improve accuracy in meaauninq ‘Pr1 and TM values

by using a calibrated ruler.

b . Lesson 2— -2—Senaor Strinqe and Formula Review.

Objective--To improve detection capability using 2-sensor

strings and to provide recall or basic training in computing the column—length formula.

c. Lesson 3-—Fnd-Points .

Objective——To standardize and improve estimations of endpoint locations in the case of irregular activation patterns .

d. Lesson 4—-.Colutsn-Lenqth Quantity .

Objective--To review and to provide more extensive training

in interpreting variations in quantity values.

e. Lesson 5--Irregular Activation Patterns .

Objective——To improve the detection and information extraction from target patterns that do not have the classical ,

clear—cut shape (stair step).

f. Lesson 6--EMIt).

Objective——To increase information extraction potential of

the EMIt) sensor through familiarization with its activation patterns.

ci. Lesson 7--MACID.

Objective ——To illustrate the MACID capability and limitations in providing information concerning the number of

targets in a column.

h. Lesson 8--DIRID.

Objective——To increase the information potential of the

DIRID sensor through familiarization with its activation

patterns.

The error analysis indicated a need to (a) accurately measure the

elapsed time of target activation patterns that are formed from tics and

(b) reliably perform arithmetic functions and decimal point placement .

To satisfy the former requirement, an UGS ruler was developed that

could be laid down on the R0376 plot paper to permit activation patterns to be measured consistently and accurately to the nearest 1/10

mi nute. Al though use of the I1C.S ruler was taught in Lesson 1, it was

also Included as a part of the noisograph developed to satisfy the second need above . Thus, the operator would need only one instrument for

measurement, arithmetic calculations , and decimal point placement as

required by the column-lenqth formula.

- i ___ 

4

-—-

~ 

-

- -
-

~~~~ - ~~~~~~~~~~~~~ —~~~~~~~~ - ~~~~~~~~~~~~~

The nomograph is shown in Figure 1. The abscissa is the distance

between the seismic sensors (in meters) . The ordinate is target activation time (in minutes). A slide indicator or tongue , consisting of

a clear plastic rectangle 3/4 in. x 10-1/2 in. with a black line running

through the long axis , is anchored by a qroznmet in the upper left-hand

corner from where it pivots. For computing target speed , ~ of the

• column-length formula, the slide indicator is pivoted untiTMthe black

line is over the intersection of the activation time and the sensor

distance. The target speed can then be read from the meters-per-

-

~~~ minute scale at the bottom and right-hand side of the nomoqraph .

With the slide indicator remaininq in the same position , the nomograph can then be used to compute ~~ (‘PF~) of the formula. This is done

simply by locating the intersection of the black line on the slide m dicator with the TT1 time measurement, and then reading the distance value

from the meters scale . The difference between this value and the CDR

is the target culumn-lenqth. The nomoqraph , therefore, can be used for

division and multiplication. The Port Huachuca training personnel have

developed additional uses for the nomograph in various applications .

Design of the Training System

-

~ With the traininci content determined , the next step was to design

the training system with which to implement the content. The training

system was designed to allow each individual to proceed at his/her own

pace, test knowledge after short segments (lessons) of instruction,

obtain immediate feedback on mastery of the content, and receive expert assistance when needed . The training system is described below

in terms of its three major components : materials , personnel , and

lesson sequence .

En the self—paced training system that was designed, the content

was organized into distinct lessons , each covering a short segment of

instruction . Each lesson contained a Lesson Sheet, a Practical Exercise Sheet , a Practical Exercise Answer Key , a Criterion Exercise

Sheet, and a Criterion Exercise Answer Key .

The Lesson Sheet was divided into three sections : Objective,

Purpose, and Concept. The Objective section stated what the operator

should be able to do at the end of the lesson, the Purpose section

stated the importance of the lesson, and the Concept section contained

the instructional materials with examples . A sample lesson sheet is

shown in Appendix A.

The Practical Exercise Question Sheet contained questions on the

material discussed on the Lesson Sheets . These questions were intended to allow the operator to test his mastery of the concepts in

the lesson.

t S
____________ — . — ,.

,;_ ~~~~~ — ~~~~~~~~~~ ~~~~~~~~~~~~ —.

~~~

--- 

,—

— — ~~~,. —-—--~~- -

~~-~~~w:~ ~~~~~— -

MfTERS

N I Iii 700 250 300 350 400 43*1 554 500 050 7e 750 050 534 *44 *000

~

i TrT

~

T

~ 

1TT~

1T~

tr

~

T METERS

I—) I I I I .

~

- ‘ f ~~f .._.,. ~~~~~~~~ ~~

_ + .-.-4 —~

-’-.. i_:~

_+ :,,

~ -

r

~J rr i; , ,,:, i , • ‘ •• i ,, i . ‘ , ;: - I , -I - 1 1 • : ::: I : : I : : : t : :: I : ’ : : I : ~~~~: ’ : ’ ; ~~~: ‘ - ~

~

- : ‘ : ‘ : f : ( :: : I :: f

~: : : 1~:: ::: : u -s.

g ~

-

~

-+

~ 

- -

~~+~ -

~~~

-

~ +~

-

~: !

~~~~~

- I~~~ i i I --- ~~~~~~~~ p

- 400

‘ 3- ~~~~~~~ I-- I

~~~~~~• I ~ —II 

~~~ I j

~~~ 4 t

~ *

—~~~ - - I : ~~~ ~ f

.13 - _ 

~~—r - t t~

—~ 

t1 t—~~t - ~

-

~

—~t ~~ -300

‘ p • ‘

~~~~~ l ,‘ ‘ —

-

• 

- ; t + : : . : : 3 5 - - - - - 44 ~~~ —.

t 4 ’’ ‘‘ ‘ ,— -4 - • -

~~~~~~

~

: : * ~~~so — 444 i— - . . - - - . , - 

-

I ~ ~~~~~~~~~~~ 

.— - - 

‘ p f .

~~~ 

. 

~~~~~~~~~~ 

-

• -H no

p -7* 0

-

~ — - t - —- ~ ~~- -‘a

~ _LL .

~~~~~.... ~~~~~~ _~~~~

~~~ ~

~~~~~~~~~~~ 

-

-‘53

- - ~~~~ ‘ ‘ ~~~~~~~~~~ : •. — ‘so

4;, ;L.~~~~ 4 ; .

~~~, 14$

~~ L. — - 4 — •

~~~~~~ — *41

4 :

~~~~~~~~~~~ ~

t i- 4 4 . 3 1*

~ ~~~~~ 

HH 3S...I

~~L,JJJ.[ ILLL~

H~,90.i.ij~’ .

~

.90, ~

LO ‘

~ 1

\ , - - - - - -\ - 4 - ~ \- ~~ ~ • -\,, • •\~ ~ ~~~~~~~~ -

~\4. \- -\ P~

-4 -

~

-

~

I tO 5 IS 5 00 35 50 45 SC 35 N 90 75 75 00 00 00

METERS PER MINUTE

55 CALCUUTIOII SCALE INOMOCRAPtII LC WT5 1T111 - COR

Figure 1 . Jot, alds——U (’.S ruler combined with nomt-*grnph .

h

Pt’

-

- - 

- 

-

~~~~~ ~~~~~~~~~~~~~~ :.~~~~~ - — -- - ~~~~~

- 

~~~~~~~~~~~~~~~~~~ 

- 

- j
4— .—-. — C .--- -

‘ —~~~~ S ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~ -_4-__ -~

The Practical Exercise Answer Key provided the operator with the

correct answers to the questions and allowed him to check his own

knowledge. The operator could restudy the lesson sheet if he did not

do satisfactorily .

• The Criterion Exercise Sheet contained questions relatinq to information on the Lesson Sheet. These questions tested the same thing

as the practical exercise questions and were the final check for mastery before going on to the next lesson.

The Criterion Exercise Answer Key enabled the traininci monitor to

evaluate the operator’s performance on the Criterion Exercise. If the

operator performed within preestablished criterion levels, he or she

was given the next Lesson Sheet. If not , he/she was sent back for

additional review or to an instructor for individualized help .

In this training system, there were three test monitors and one

subject matter expert. They helped the operators and monitored their

progress through the system . For normal usage in the field the material

could be bound into a booklet and monitoring would require only one

person (3, below):

1. One test monitor was at Station 1 to control handout of the

Lesson Sheet and score the Criterion Exercises .

2. Iwo test monitors were at Station 2 to control all activities

having to do with the Practical Exercises .

3. The subject matter expert was at Station 3 to provide subject

matter assistance whenever it was needed .

The operators were required to follow five basic steps to complete

each lesson:

1. Read the Lesson Sheet for comprehension .

2. Answer the Practical Exercise Questions .

3. Check the answers with the Answer Key and reread the Lesson

Sheet if necessary .

4. Answer the Criterion Exercise Questions .

5. Have the answers checked with the test monitor . If the criterion was met, the operator was allowed to progress to a new

lesson; if the criterion was not met, the operator was asked

to review the Lesson Sheet and Practical Exercises and/or confer with the subject matter expert and then rework the Criten on Exercise.

- -- 

~~~~~~ .- ---~~ 

~~~~~~~~~~~~~~~~
Development of the Test Scenarios

In order to assess objectively the performance of the operators, scenarios were developed from scenarios used in a previous study

(Martinek , Pilette, and Riggs, 1978) . These were developed from magnetic tapes of typical activations produced by personnel, tracked and

wheeled vehicles, and typical noise such as artillery and aircraft

collected during Project 1030 and BASS III tests conducted at the • Modern Army Selected Systems Test and Evaluation ( MASSTER) project , Fort Hood, Tex. (U.S. Army , 1970 and 1972). Because these were controlled exercises, target location and time were known and could be

related to sensor activations .

Eight 30-minute Thigh—noise” segments were selected : four seqments for each of two scenarios. These scenarios were matched with

respect to string size and target activity . Two of the segments in

each scenario contained three targets each (low activity) and two I 

contained six targets each (high activity) . Sensor string size was

systematically varied , i.e., in the three—target case , one target involved a 2—sensor string, one involved a 3—sensor string, and one in—

volved a 4—sensor string . In the six—target case, these were doubled. Targets were selectively arranged within each 30—minute period to

provide variation and balance .

High-noise means that the segments simulated a battlefield environment containing typical activations caused by fixed-wing and

helicopter activity , live artillery firing , weather activity , and

unreliable (noisy) sensors . These activations were not eliminated

when they occurred with the origi nal target patterns and , in many

cases, were added to scenario segments to obtain a reasonable balance

in amount and type across all the segments .

Training Validation Design

The effectiveness of the training materials was assessed by means

of a pretest/posttest comparison of tiGs monitor performance on opera—

tionally realistic sets of scenarios using simulated R0376 event recorders . The effectiveness of the job aid (nomograph) was assessed

by a pretest/poettest comparison of 10 problems requiring the operators to derive target speed and quantity from typical activation

patterns .

Independent Variables. The independent variables planned for

analysis are defined below .

1. Sessions-—Pretest versus posttest. The pretest was administered durinq Session I and the posttest was administered

after training (Session II).

8

- - ~~~~~~~~~~~~~~~~~~~~~~~~~ -—- -—

- - • - 

- -

--- - - - - - -p

~~-~~~ -- -
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- - .•--.—---

~~~~~~~•--- - -,-~~

---~

-4•--,.---- ’...-—- -

2. Sensor String Size——Two—sensor strings versus 3-sensor strings

versus 4—sensor strings.

3. Target Activity— -Low target activity (3 per 30 minutes) versus

high target activity (6 per 30 minutes). A target was defined

as one or more vehicle(s) or person(s) traveling in a column

on a trail.

4. Scenarios--Two 2—hour scenarios were matched with respect to

string size and target activity . They were administered in

a counterbalanced order as either pretest or posttest . Equivalent scenarios were not guaranteed because different amounts

of noise were present and different targets were used in each

scenario . The scenario variable was of interest mainly to

increase the sensitivity of the statistical analysis and provide data for future scenario develo~

msent.

5. Groups--The 20 operators were randomly assigned to two groups

to control for scenario effects by counterbalancing . Thus ,

one group received scenario “A” during the pretest and the

other group received scenario “B” ; the reverse was used for

the posttest.

The independent variables were analyzed by using a mixed-model p analysis of variance design . By this design, individual differences

are controlled across sessions (training effects) and all important

variables analyzed are within—subject comparisons, The three factors

of groups, sess ions , and scenarios were counterbalanced in a Latin

square, and as a result, interactions of these factors could not be

obtained. While the main effect of sessions alone would indicate the

effectiveness of the self—paced training materials, the main effects

of sensor string size, target activity , and the higher order interactions were included in the detection and identification analysis

mainly to determine the specific areas of usefulness .

In a previous study, it was shown that U(

~ operator detection and

identification rights were significantly lower with 2—sensor strings

than with 3— or 4—sensor strings (Martinek , Pilette, and Riggs, 1978)

In addition, percentage completeness on hiqh—taraet—activity scenario

segments was significantly lower than on l..,w—tarqet—activity scenario

segments . The self-paced training materials were developed to help - improve performance with the 2-sensor string condition. The overall

effect of the training should also help to improve performance in the

high-target-activity condition more than in the low-activity condition,

because there is more room for improvement and more weight was given to

errors in the high—activity condition in the determination of problem

areas . Thus, interaction effects with these variables and sessions - - were expected.

Dependent Variables. For the assessment of the training materials,

nine separate dependent measures were planned . The first three dealt

— 9

- - - —-- -

‘-~~~~ -. -

~~ 

- - 

~~~~~~~~~~ -

~~~~~~~~~~~~~L —

~~~~

—-
~~~~~~~~~~~~~~~~~~~~~~

--

~

-

~~~~~~--- ——— --~~~~~~

- —-—--— -

~

-

~~~~~~~~~~~~~ - --- - • --~~--- --- --—- --- -

~~~~~

-

~~~

--

with the overall measures of performance; the last six were related to

special aspects of the operator ’s job.

Using the independent variables listed above, separate analyses

of variance were planned for the following dependent variables:

Detection rights-—If an operator reported an activation pattern

on the display that was a valid target, the

response was classified as a detection right,

This variable is perfectly correlated (for

most variables) with detection completeness

because completeness is a ratio of the targets

detected to the total number of targets present. Percentage completeness figures are given

where they have special meaning.

Wrongs (false -—If an operator reported an activation pattern

alarms) on the display that was not a valid target,

the response was classified as a false alarm.

Identification --If, in addition to detectinq a valid target,

rights the operator classified it correctly by type

(personnel or vehicle), the response was also

classified as an identification right.

- 

The following four dependent measures relate to the ability to use

the formulas in arriving at additional intelligence information once a

target has been detected,

——Total time (in minutes) of the first sensor

activation pattern of a sensor string .

T

M --Time difference (in minutes) between the mid— - point of the first sensor activation pattern

and the midpoint of the second sensor activation pattern.

Speed --Estimated speed (meters/minutes) with which

the target moved through the sensor string .

Speed is usually calculated from time (TM)

and the distance between two sensors .

Quantity --Number of targets estimated to be in the target column; obtained by dividing the total

length of the column by the estimated distance

between tarqets.

The final two dependent measures were

Reporting time --Time between target detection and completion

o~ the report as written by the operator.

10
— ~~~ •7-_ - __--:.-~ ~~~~~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Confidence --The operator’s estimates on a four—unit scale

as to how confident he or she is that the

activation pattern he/she is reporting on is

a valid target.

Job Aid (Nomograph ). The evaluation of the job aid involved oivina

paper—and—pencil tests in which 10 target activation problems were prosented on the pretest and posttest. The operator did not detect targets

but only made the necessary measurements and calculations necessary for

reporting additional intelligence information (speed and quantity) .

The posttest was prepared by randomly rearranging the targets of the

pretest and chanqing the numerical values of the distances between the

sensors and sensor detection ranges. Performance was compared on the

fol lowing dependent measures : TT1, TM’ speed of target, target type,

length of column, quantity, and reporting time .

Criteria for Scoring Formula Dependent Variables. The intervals

of correctness were established by selecting and setting values above

and below the established school solutions for TT1 and TM. The criteri~ n for accepting TT1 and TM values for the individualized trai!-inq

was within 2/10 of a minute above and below the established school

solution. The result of accepting a range of values from TM introduced -i range of values for speed . Likewise, accepting a range of

values for T’r1 introduced a range of values f~

-

~

r length of column and

quantity . It was necessary to compute ranges for each tarcet for

scoring purposes.

The objective of the job aid evaluation was to assess the worth

of the ncmograph/UGS ruler combination, using the highest standards

that could be needed in the field . The range of acceptance, therefore,

for the pretest and posttest of the job aids was set at 1/10 minute

above and below the established school solution of TT1 and TM.

Test Procedure

The test plan covered a 4-day period , as shown in Figure 2. During the first day , 10 operators received test procedure training plus

the pretest in the morning : 10 received the same in the afternoon .

This procedure was necessary because only 10 R0376 simulators were

available. For both the morning and afternoon , five operators received Scenario A and five received Scenario B for the pretest.

During the second day, all 20 operators received the eight lessons of individualized training . This training did not require the

use of the drive mechanisms. At the completion of the training , all

operators completed a course evaluation questionnaire . The questionnaire provided a measure of operator acceptance of the materials and

the training system , In addition , this information and the operator

course performance data provided input for future revisions of the

course material.

11 

-

•--- ---—.:

~~~~~~~ -

~

- --- -- — -— .-
Day 1 Test procedure and pretest

7:30 a.m. - 9:30 Training on test procedure with the R0376

simulator for 10 operators .

10:00 - 12:00 Pretest--Each operator interpreted 18 targets

in a 2—hour scenario using the R0376 event

recorder simulator . Five operators received

Scenario A and five operators received Scenario B.

1:00 — 3:00 (Same as 7:30 — 9:30)

3:30 — 5:30 (Same as 10:00 — 12:00)

Day 2 Individualized , self—paced training

8:40 a.m . - 12:00 Four lessons--hard copy-—20 sub~ects

1:00 - 4:00 Four lessons--hard copy--20 subjects

4:00 - 5:00 Student training evaluation questionnaire--

20 operators

Day 3 Posttest

9:00 a.m . — 11:00 Posttest-—Each operator interpreted 18 targets

in a 2—hour scenario using the R0376 event

recorder simulator . Five operators received

Scenario A and five received Scenario B.

Operators and scenarios were switched from

the pretest.

2:00 — 4:00 Same as above for second group

Day 4 Nomograph, pretest, training , and posttest

7:30 a.m . - 8:30 Pretest--hard copy format--20 operators

9:00 — 11:00 Training——lecture/discussion——20 operators

1:00 - 3:00 Posttest——hard copy format--20 operators

Figure 2. Test administration schedule.

12 

-- ~~~~~~~~-

~~~~---—-~ ~~~~~~~~~~~~

-- -

~~~

--

~~~ ------ -- - - -
~

The posttest was administered on the third day . Durinq the morning, the five operators who received Scenario A as the pretest now

received Scenario B, and those who initially received Scenario B now

worked through Scenario A. The same procedure was followed in the

• afternoon with the remaining 10 operators.

• During the final day, all 20 operators received a pretest in the

morning, training on column-length formula calculations using the

nomograph, and a posttest in the afternoon.

For the individualized training , Appendix B presents the monitor

instructions for the ( 1) initial test procedure training, (2) pretest

and posttest training, and (3) actual traininq administration procedure .

The job aid (nomograph) monitor instructions are not provided because

they were given in a spontaneous lecture—discussion approach . For the

-: nomoqraph training, a series of practical problems was presented to

and solved by the class.

RESULTS AND DISCUSSION

Overall Performance Measures

Detection Rights. The following variables and interactions are of

primary importance and are discussed in the order given : sessions,

string size, sessions x string size, target activity, and sessions x

target activity . Table 1 presents the training experiment analysis

of variance results for detection rights . Of most interest is the

significant difference (.01 level) found for the sessions variable.

Out of a possible 18 rights, the average operator ’s correctness for

the pretest (Session I) was 10.4 rights while that for the posttest

(Session II) was 12.3 rights. This represents an 18% improvement or

about one target per operator per hour and demonstrates the effective- —

ness of the training . Appendix C presents a discussion of why this

increase is attributable to training as opposed to a practice effect.

The string—size variable (Table 1) is signi ficant at the .01 level.

Out of a possible 6 rights for each string size, the 2—sensor 5trinqs

resulted in 2.6 rights, the 3—sensor strings resulted in 4.7 riqhts ,

and the 4-sensor strings resulted in 4.1 rights . The significantly — fewer target detections when using 2-sensor strings agrees with the

results of a previous study (Martinek , Pilette , and Riggs , 19 74) .

• The interaction of sessions and sensor string size is nonsignificant, suggesting that the increases in performance for the three string

sizes (from Session I to Session II) are similar. However , individual

statistical tests conducted on each sensor string size for differences

between sessions indicates that this condition is true for the 3-sensor

and 4-sensor string sizes but not the 2-sensor string size (t = 2.61,

df = 19, p = .05). The average scores for this interaction are pre—

sented in Table 2.

13

—=~~~~~~ — ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ,
- -,--—‘-—,- -~~~--.-- --—•.--—---- -

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.- -~

------- —-.----

Table 1

Analysis of Variance Table for Detection Rights

Sig.

Source df SS MSS F ratio level

Between— subjects 19 30.184

Groups 1 .417 .417 .252 NS

Error 18 29.76 1.654

Within—subjects 220 213.000

Sessions 1 6.0166 6.017 16 .395 .01

String size 2 45.0333 22.517 61.354 .01

Session x string size 2 1.2334 0.6167 1.680 NS

Target activity 1 48.600 48.600 132.425 .01

Sessions x target

activity 1 1.6667 1.667 4.542 .05

— String size

x target activity 2 12.7000 6.350 17.302 .01

Scenario 1 12.1500 12.150 33.106 .01

Scenario x target

activity 1 5.4000 5.400 14. 714 .01

Scenario x string size 2 0 0.000 0.000 NS

Scenario x string size

x target activity 2 5.200 2.600 7.084 .01

Session x string size

x target activity 2 0.233 0,116 0.316 NS

-

~ String size x groups 2 1.7334 0.867 2.362 NS

Target activity

x groups 1 0.000 0.000 0.000 NS

String size x target

activity x groups 2 0.4000 0.200 0.545 NS

Error (w) 198 72.633 0.367

Total 239 243.184

14

__________________ — —-- i- -
~ 

~~~~~

— _ -

~

-•

~

--—

~~~~~~

-,-— -—-

Li

i

i •

Table 2

Mean Number of Detection Rights by Session and

Sensor String Size

2—sensor 3—sensor 4—sensor Sum

Session I 2.1 4.5 3.8 10.4

Session II 3.1 4.8 4.4 12.3

Average 2.6 4.7 4.1

One of the objectives of the training was not only to increase the

numbe r of targets detected across all the string sizes, but especially

to increase the number of targets detected with 2-sensor strinqs . As

shown in Table 2, the number of targets detected with 2—sensor strinc~ did increase from 2.1 to 3.1. The latter value , however , is still well

below the corresponding figures for the 3- and 4—sensor s

~rin~s (4.8

and 4.4 respectively) indicating that this objective was not completely

-

~ satisfied . One obvious possibility ~ that 2—sensor strings are more

difficult to interpret. This result reinforces the validity of the

I - doctrine calling for three or more sensors in a string .

The target activity variable is significant; each operator detected, on the average , 4.4 targets during low target activity and 7.1

during high activity. These figures are based on a possible total of

6 targets for the low—activity and 12 for the high-activity condition .

Interpretation of all experimental effects involving the target activity

variable must consider that twice as many targets are detectable in the

high— target-activity condition as in the low . Thus, significant difference in detection rights was expected . Ot more importance is the

detection completeness of targets detected in each activity condition .

During low target activity, 72% of the targets were detected but during

high target activity only 59% were detected, a difference which is

statistically significant (t — 4.94, df — 19, p < .01).

The interaction of sessions and target activity is significant,

which sugqests that the increases in performance for the two tarqet

activity conditions from Session I to Session II are different . Indivtdual statistical tests indicate that the increase in performance

is attributable to the high—target—activity condition (t 2.18,

df — 19, p < .05). The raw score values for this interaction are

presented in Table 3, along with completeness scores reported in

parentheses.

15
Table 3

Mean Number of Detection Rights and Completeness by

Session and Target Activity

Low target activity High target activity Sum

Session I 4.1 (68%) 6.3 (53%) 10.4

Session II 4.6 (76%) 7.8 (65%) 12.3

Average 4.4 7.1

One of the objectives of the training was to increase the completeness of targets detected , especially during high tarqet activity . As

shown in Table 3, the significant completeness increase from 53% to 65%

satisfied this objective . This latter figure, however, is still below

the corresponding figure for the low-activity condition (76%). These

results indicate that the training was effective in increasing per—

forinance, predoninantly in the high-target-activity condition .

The string size by target activity interaction presented in Table 4

is significant at the .01 level . Thus, although there are large differences in the mean number of detection rights in the 3— and 4—sensor

strings, there is only a small difference in the 2—sensor string . When

these figures are corrected for the different numbers of targets in the

high- and low--activity conditions, i.e., completeness, the effect on

operator performance is clearer. As first observed in a previous study

(Martinek, Pilette, and Riggs, 1974), completeness is lower for all

sensor string sizes in the high—activity condition, but the significant

interaction indicates that, as the nunther of se.isors in the string become s smaller (i.e., from 4 to 2), it becomes more difficult for a

monitor to detect targets during high target activity as opposed to a

low target activity. For the 4—sensor condition, a 4% difference

exists between the low- and high—activity conditions; for the 3-sensor

condition, the difference increases to 13%, and for the 2—sensor condition the difference increases to 23%.

These results should be interpreted with caution . The poorer performance associated with the 2—sensor string condition may be due to a

combination of a disproportionate personnel/vehicular tarqet ratio and

various conditions prevailing at the time the target activation data

were collected. In general, personnel do not produce as good an activation pattern as do vehicles. The target types for the 2—sensor string

condition were S personnel, 6 vehicles , and 1 combination; for the 3-

sensor string condition target types were 3 personnel, A vehicles, and

16

__ ~~~~~--

—•-- --- -— --

~

-- - -_

~ I- - —- --- _ — - — 

-
~~~~—--—-- - I - ~~~~~~~~~~~~~~~~~~~~~~ --- - - - - -~~~~~- _-

~~~~~~~~

-

~~~~~~~~~~~~~~~~~~

- _ - _

~~~

--

~~~~~

_

Table 4

Mean Numbe r of Detection Rights and Completeness by

String Size and Target Activi ty

2-sensor i

~

-se nsor 4-sensor Sum

Low—targe t activi ty 2.4 ( 59%) 3.5 (86%) 2.9 (7 1%) 8.7

High—targe t activity 2.° (16%) ~~~~~~~ (73%) 5.4 (1’7%) 14.0

1 combination; and for the 4-sensor string condition targe t types we re

2 personnel and ° vehicles . In addition , because all personnel target

activations were collected during t he BASS III tests (U.S. ~rmy , 1 970)

and all vehicular target activations were gathered durinq the Project

1030 tests (U.S. Atiny, 1972) with a 2—year interval separating these

exercises, diffe rences in equipment reliability, implantation techniques,

oround location , weather , and soil type could have had a marked infl uence on target presentation quality and hence upon operator performance .

Thus, the di fferences found for string size may be due to differences

in the level of target difficulty .

The scenario variable is significant at the .01 level , which m dicates that Scenario A , with an average of S rights, was more difficult

than Scenario B , with an average of 6.4 rights . ~

‘breover, the siqnificant interaction between scenario and target activity (.01 level) indicates noticeably lower performance (5.9) in the high-target-activity

condition of Scenario A (see Table 5) . Although not important to the

objectives of this evaluation, a thorough analysis of the causes of

this outcome could be useful for future scenario development.

The triple interaction of string size, scenarios, and target activ—

Ity is significant at the .01 level. The mean number of rights by condition is presented in Table 6.

Table 6 reveals more detailed information concerning significant

scenario differences. The scenario and target activity inte raction

seen in Table S indicate that the high-activity condition of Scenario

A contained more difficult-to—detect targets. This is especially true —

of the 2-sensor string condition within the high-activity condit ion of

Scenario A. However, this conclusion and others which can be drawn do

not have a direct bearing on the objectives of the study and are important only for future scenario development. All the other interactions tested were found to be not signi ficant.

17

________ __ 

- -
Table 5

Mean Number of Detection Riqhts by Scenario

and Target Activity

Scenario A Scenario B Sum

Low— target activity 4.1 4.6 8.7

High-target activity 5.9 8.2 14.1

Ave rage 5.0 6.4

Table 6

Mean Number of Detection Rights by String Size,

Scenario, and Target Activity

Low target activity High target activiti

String size Scenario A Scenario B Scenario A Scenario B

2—sensor 1.3 1.1 .9 2.0

3—sensor 1.5 2.0 2.7 3.2

4—sensor 1.3 1.6 2.4 3.0

Average 1.4 1.6 2.0 2.7

Wrongs (False Alarms). Of major interest to this study is the

small ntmther of false alarms: only three occurred during Session I and

seven during Session II , too few for conducting an analysis of variance .

The false alarms we re distributed in a random fashio~ across eigh t

operators. Because of the randomness with which the errors we re

coumtitted, it is not possible to draw generalizations concerntnq the

causes for false alarms .

The small number of false alarms that occurred in this study represents a sharp contrast to t)

~

e large number of false alarms reported in

a multidisplay comparison study using similar scenarios (Martinek ,

Hilligos , and Lavicka , 1978) . In the present study the averaqe number

18 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

_ _

~~~~~~~~~~~ -~~~~~~~~~~~~~ - — -- — —- --
of false alarms per 2-hour scenario was .2 while that for the R0376

from the multidisplay study was 5.6. Such a large difference , even

though similar scenarios were used, could be caused by differences in

(1) Instructions (set) or context of the study (comparison of displays

versus training) , and/or (2) school orientation on accuracy of target

reporting . The latter reason is probably more signi ficant, because

the average operator in this study had graduated (several months since)

from the U.S. Army UGS school . The Army school typically uses classical patterns (stairstep shapes) for practical exercises, and this may

automatically ingrain accurate target-reporting habits among the

trainees. They are less likely to make errors of commission, but

they may be more likely to make errors of omission .

The multidisplay study used l

~ Navy personnel who had been trained

at another UCS school but, perhaps more important, had been on the job

for several years and had received field experience on several maneuvers . - These operators committed more errors of commission , but they also detected more targets on the average , which may reflect the Navy’s em—

phasis on completeness.

Identification Rights. Table 7 presents the analysis of variance

results for identification rights . Of most interest is the significant

difference (.01 level) for the sessions variable . The average operator’s identification rights score for the pretest session was 8.3 while

that for the posttest session was 10.9. This represents an average improvement for each operator of 2.55 targets or a 31% gain over pretest

performance . This is a greater increase than would be expected from

the detection results , which showed an 18% improvement . The identification improvement of 31% seems to indicate that the training was more

effective in improving target identification than it was for improving

target detection, because detection improvement was only 18%. However,

identi fication completeness (the ratio of targets identified to those

detected) indicates that there are no significant differences between

Session I and Session II.

The string—size variable is significant. Table 8 shows that with

6 targets possible for each string size, operators identified 2.2 of

the targets activating the 2—sensor strings, 3.9 of those activatinq

the 3-sensor strings, and 3.5 of those activating the 4—sensor strings .

The interaction of sessions by string size in Table 8 is nonsiqni fi-

- cant, which suggests that the increases in performance from Session I

to Session It for the three string sizes are similar.

19

- - r . T - T r ~~. L... . 

~~ _~~ _ ~~~— - - ~~

-

~~~ -.~ - ---—--— — ~~

- ~~~~~~~ - -
--

~ - - - -

~~~~~~~~~~~~~~~ - - 

I-’

Table 7

A*~

alysis of Variance Table for Identification Rights

siq.

Source df SS r.t

~

S F ratio level

Between-subjects 19 34.7125

~roups 1 7.7041 7.7041 5.1344 .05

Error 18 27.0084 1.5005

W ithin—subj ects

Session 1 10.8375 10.8375 25.2740 .01

String size 2 34.8083 17.4042 40.5882 .01

Session x string size 2 0.9250 0.4625 1.0786 NS

Target activity 1 22.2041 22.2041 51.7820 .01

Session x target

activity 1 1.8375 1.8375 4.2852 .05

String size x target

activity 2 6.3084 3.1542 7.3559 .01

Scenario 1 5.1041 5.1041 11.9032 .01

Scenario x target

activity 1 2.6043 2.6043 6.0735 .05

Scenario x string size 2 0.8584 0.4292 1.0009 NS

Scenario x string size

x target activity 2 4. 7582 2.3791 5.5483 .01

Sessions x string size

x target activity 2 0.0250 0.0125 0.0292 NS

String size x group 2 11.4084 5.7042 13.3027 .01

Target activity

x groups 1 1.2043 1.2043 2.8085 1.18

String size x target

activity x groups 2 1.3082 0.6541 1.5254 NS

Error (w) 198 84.8916 0.4288

Total 239 222.796

___________ -- 

- - -

:0
Table 8

Mean Number of Identification Rights by

Session and String Size

2-sensor 3—sensor 4-sensor Sum

Session I 1.7 3.7 3.0 8.3

Session II 2.7 4.2 4.0 10.9

Average 2.2 3.9 3.5

Identification completeness (i.e., the ratio of the number of targets identified to those detected) for this interaction is shown in

Table 9. There are no significant differences between Session I and

Session II. Individual t—tests were computed for each string size to

te8t the significance of the difference between the sessions means .

The values for the 2— , 3-, and 4-sensor strings were 1.66, 1.07, and

1.67 respectively (n = 9). The results indicate that once a target

was detected, the sensor string size was immaterial: rouqhJy 80% to

90% were identified . This is similar to the previous study (Martinek ,

Hilligos, and Lavicka, 1978) in which the R0376 display resulted in

90% completeness. For both the Army and the Navy operators , therefore,

once a tarqet has been detected, the probability of a correct identification is similarly high .

Table 9

Percentage Identification Completeness by Session

and String Size

2-sensor 3—sensor 4—sensor

Session I 72 82 78

Session II 88 87 92

21

~

- - -

- _

-

.-

~ 

~~~~~~~~~~~ - - ~~~~~~~~~~~~

-

-

~~~~ _
-

~

The target activity variable is significant. As with the rights

(detection) results, this outcome was expected for the identification

rights. Of the targets detected in the low—tarqet-activity condition,

90% we:e identified, whereas in the high—target-activity condition 81%

were identified.

Of more importance is the significant interaction between sessions

and target activity (Table 7). The means are given in Table 10. The

significant interaction suggests that performance did not increase

similarly for both activity conditions but that the high-activity condition resulted in a larger increase . The training objective of better

performance for the high—activi ty condi tion was satisfied.

Table 10

Mean Number of Identification Rights by

Session and Target Activity

Low-target activity High-target activi ty Sum

Session I 3.5 4.8 8.3

Session II 4.3 6.6 10.9

Average 3.9 5.7

Table 11 presents identification completeness (i.e., percentage of

targets identified to targets detected). There are no significant differences between Session I and Session II. The t-t-est values (df = 19)

were 1.409 for the low— target-activity condition , ~nd 1.20 for the hightarget-activity condition.

The string size by target activity interaction presented in Table

12 is significant at the .01 level. This table reveals that the number

of identification rights is substantially lower for the 2-sensor string

condition for both the low- and high-target-activity conditions . These

results are very similar to those for detection rights .
Table 11

Percentage Identification Rights Completeness by

Session and Target Activity

Low target activity High target activity

Session 1 85 79

Session II 94 85

Table 12

Mean Number of Identification Rights by

String Size and Target Activity

2-sensor 3-sensor 4—sensor

Low—target activity 2.0 3.1 2.7

High—target activity 2.3 4.8 4.3

Average 2.1 4.0 3.5

The scenario variable was significant at the .01 level , which indicates that Scenario A (with an average of 4.4 rights) was more difficult than Scenario B (with an average of 5.3 rights) . The difference

in scenarios is probably due to a difference in target difficulty , as

revealed in the high- and low-target-activity conditions shown in

Table 13. This table suggests that the interaction (significant at the

.01 level) is due to the noticeably lower performance in the high-targetactivi ty condition (4.9 rights) of Scenario A. As with the detection

outcome, a further analysis of these data is beyond the objecti ves of

this research but could be useful for scenario deve1o~ nent in future

research .

The triple interaction of string size, scenario, and target activ.-

ity is significant . The results are presented in Table 14. For each

target activity level , note the lower number of correct identi fications

for targets involving 2-sensor strings, and the differences across

scenarios for 2-sensor strings .

Ii 23
- ~~~~~~~~~~~~ -::-:-

~~;~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~ 

- 

~~~~~~~~~~

Table 13

Mean Number of Identification Rights by

Scenario and Target Activity

Scenario A Scenario B Sum

Low— target activity 3 .8 4.0 7 .8

High—target activity 4.9 6.5 11.4

Average 4.4 5.3

Table 14

Mean Number of Identification Rights by String Size,

Scenario, and Target Activity

Low target activity High target activity

String size Scenario A Scenario B Scenario A Scenario B

2—sensor 1.1 .9 .7 1.7

3—sensor 1.4 1.8 2.3 2.6

4—sensor 1.3 1.4 2.1 2.3

?~

verage 1.3 1.4 1.7 2.2

The interaction of sessions x string size x target activity was

not significant, which indicates that the previous discussions of these

variables need not be modified due to a triple order effect.

The remaining interactions are string size x scenario, and those

involving groups. They have little bearing on the objectives of this

- study and are therefore not discussed .

Confidence Ratinqs. The analysis of variance results for the confidence variable is presented in Table 15.

ilt Jt __________ J
---——~~~~~~~ —- - I

—-— ~~~~--- - - - -- --—I

~~~~~~~-

Table 15

Analysis of Variance Table for Confidence Values

Sig.

Source df SS F ratio level

Between-subjects 19 6.61

Groups 1 .90 .90 2.84 NS

Error (b) 18 5.71 .38

Within—subjects 20 1.25

Sessions 1 .004 .004 .Ot-’2 NS

Scenarios 1 .08 .08 1.25 NS

Error (w) 18 1.17 .065

Total 39 7.86

No significant differences occurred . Either there is no difference

in confidence or there are differences in confidence but the rating

technique for measuring an operator ’s confidence was inadequate . The

latter case is probably true, since the variation between—subjects

(Table 15) is far greater than the variation within—subjects. This

occurred because each operator tended to select and use only a particular confidence level, even though some target activation patterns were

more difficult than others.

Measuring the confidence of operators could be useful to the field

commander when he is weighing fragments of intelligence from several

sources in a battle situation . Knowing how confident the operator is

in his judgment could influence the importance that the commander will

place on the information . This was the first time that these monitors

had been asked to rate their confidence . The rating system perhaps

could have been more useful if the operators had been given more extensive instruction and had been helped to develop adequate anchor

points across the scale.

Reporting Time. The F-max test of the homogeneity of variance

indicated that this basic assumption in use of the analysis of van —

ance model was not satisfied. Therefore, a log transformation of the - - reporting time scores was performed . Table 16 presents the results

of the analysis of variance, using the log transform of reporting time .

Both the groups and sessions variables were significant at the .05

level. The results on groups has no impact on the objectives of the

25

___________________ 

=

- - j _ ~~~~~~~~~~~ -~~~-—~~~- - -
study except for control purposes . However, the results on the sessions variable indicates that the training resulted in a time saving

in the reporting of targets. An operator required 5.9 minute s to

report a target before training and 5.0 minutes after training--a time

saving of 15% for each target reported . Operationally this means that

a target would be reported 1 minute earlier. In addition, during an

overload situation when operators have more targets than they can

report, an operator who has received special training will be able to

report more targets.

Table 16

Analysis of Variance Table for Reporting Timea

Source df SS MS F ratio P

Between-subjects 19 .4151

Groups 1 .1780 .1780 13.59 .01

Error (b) 18 .2371 .0131

Within—subjects 20 .2181

Sessions 1 .0440 .0440 4.94 .05

Scenarios 1 .0127 .0127 1.43 Ms

Error (w) 18 .1612 .0089

Total 39 .6332

C

Log transform of reporting time used .

Additional time savings (4 minutes) could also be incurred by

having the operators send out a target detection report immediately

after target detection and then by a followup target identification

report several minutes later. For target acquisition, these few minutes could be very valuable for processing the information and alerting

gun crews.

Measurement and Computational Procedures

Use of the column-length formula provides two bits of information

used for military intelli gence purposes, target speed and target quan- —

tity (the number of targets in a column). Solution of this formula requires two target—pattern measurements which are takt~

n directly from

26

- —~ --— ~~~~~~~~ -—- -
~~ ~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -

~~~~~~~~~~~

the activation readout. These are T’~1 and TM, which have been discu~

.sed previously. Analyses are reported below for TT1, TM, target

~peed, and tarqet quantity . The school—solution criteria of correctness are discussed in the Me thod section .

TT]. The analysis of variance for ‘rr1 is presented in Table 17.

The s~i~ions variable is significant at the .01 level , which indicates

that the training was effective . The posttest results of 6.45 average

correct measurements represents a 63% improvement over pretest results

of 3.95. The groups and scenarios variables are nonsignificant.

Table 17

Analysis of Variance Table for TT1 Values

• Sig.

Source df SS MS F ratio level

Between-subjects 19 109.40

Groups 1 3.60 3.60 .613 NS

Error (b) 18 105.80 5.87

Within—subjects 20 184.7

Sessions 1 62.5 62.50 9.27 .01

Scenarios 1 .9 .90 .134 NS

Error (w) 18 121.3 6.74

Total 39 294.1

TM. The analysis of variance for TM values is presented in Table

18. ~l’~e sessions variable is statistically significant at the .01 level

which indicates that the training was effective. Posttest results of

6.8 average correct measurements represent a 58% improvement over pretest results of 4.3. The groups and scenarios are nonsignificant .

The results of ‘Nl and TM are almost identical. This was not unexpected because both involve measurements taken directly from the X-T

display and , therefore , demonstrate the effectiveness of the training

given for the endpoints (Lesson 3) and the use of the ~~~ ruler job

aid.

27

_____________ 

— —----- - ---I---- - - ——-— —

~~~~~ 

- —-— —
I I

~~~~~~~~~ I~~~~~ I

~~~~~~ !1 

--

Table 18

Analysis of Variance Table for T

M Values

Sig.

Source df SS MS F ratio level

Between—subjects 19 88.90

Groups 1 0 0 0 NS

Error (b) 18 88.90 4.94

Within—subjects 20 217.00

Sessions 1 62.50 62.50 8.35 .01

Scenarios 1 19.60 19.60 2.62 NS

Error (w) 18 134.90 7.49

Total 39 305.90

Target Speed. Table 19 presents the results of the analysis of

variance for the target speed variable . Again, there is a significant

(.01 level) increase of posttest results over pretest results .

The average operator performance on the pretest is 3.9 correct and

that for the posttest is 6.4 correct, representing a 64% increase in

accuracy of reporting the speed of the targets. However, most of this

increase is attributable to the increased accuracy of obtaininq T

M .

The groups and scenario variables are nonsignificant.

Quantity. Table 20 presents the results of the analysis of variance for quantity . Again the session variable is significant at the

.01 level. The average operator performance on the pretest was 3.95

correct and that for the posttest was 6.45 correct, a performance increase of 63%. The difference between operator performance on the two

scenarios (4.5 correct on Scenario A and 5.9 correct on Scenario B)

was significant at the .05 level .

Within—Course Performance

In developing instructional systems, it is standard procedure to

collect data that can assist the course developer to effectively revis

his materials . In addition to pretest and posttest performance, opera

tor performance on individual lessons can provide effective guidelines

for revisions that can result in improved posttraining performance on

28 

— --

~ - -~~~~~ ,-

~

..-

--

—--I

~ 

- - — 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~L _

~~~~~~~ Z~~~~~-~

- - 

~. z r -~r — -
- --- - I

-

Table 19

Analysis of Variance Table for Tarqet Speed Values

Sig. Source df SS MS F ratio

H~ twc~’n— .’~ub j L ’

~~ t ~ ~ 

-- 

82.10

_

Groups 1 .40 .40 .09 NH Error (b) 1,8 82. 70 4.3 4

• Wi t in—~~u1-~ j

~~’t~ 20 157 .00

• Sossion~ I (-

~ 2 .50 (-

~ 2 . 0) 1 3 . 31 .01 $cenariou 10.00 10.00 2 .13 NH

Error (w) 18 134 .50 4.”

Total ~ 21~~, 10

Table 20

Analysis of Variance Table for Quantity Values

Sig. Sourc~’ df 58 MS F ratio level

Retween—~thj~ct~

Groups i 2.50 2.50 .34 MS

Error (b) 18 130.90 7.27

Within—stth-1c’e t~ 20 151 .00

Sessions I 62.50 ~

1 .50 1t~ .32 .01

Scenarios 1 1’~~ .60 l’) .60 5 . 12 .03

Error (w) 18 68.90 3.83

Total ~ ) 284.40

- 

- - ——

. ~~~~~~~~~~~~~~-

~~~

— — ~~~~——~~~ - ~~~~~~~ --~~~~~~—- - —- -~ ---- - - --
-. -—-. - -—~~ ~~~ --- ~ -I

~~~~~~~~~~~ -~~~ ~~~~~~~~~~~~~~~~~~~~ —~~~~~~~-I-- ------——

subsequent administrations . Although a formal evaluation using operator performance data was not within the scope of this effort, the data

on individual lesson performance are provided, to present a complete

picture of the effect of the training and to allow for improvements of

the course materials in subsequent efforts .

Table 21 presents the scores on the criterion exercise for each

lesson for each operator . Each operator ’s first attempt is given as

well as scores on subsequent recycles . It should be recalled that an

operator had to get 100% correct on each criterion exercise before

being allowed to proceed to the next lesson. It is apparent from

Table 21 that Lessons 3 and S presented the greatest amount of difficulty for the operator . Over 50% of the operators had to recycle on

these two lessons, which indicates that the training materials should

- . be revised .

As with mos t individualized , self—paced training systems , opera—

tors varied widely in the amount of time they required . One operator

had completed half of the lessons by 10:15 a.m ., and over 50% of the

operators finished Lesson 4 by the end of the morning . One operator

completed all eight lessons by 2:30 p.m., (1—1/2 hours early), 75%

of the operators were finished by 3:00 p.m. . and only one operator

took the full time allotted .

Student Queslionnaire Results

The questionnaire was administered to obtain t measure of student

acceptance of the individualized , self-paced training system and to

provide further data that could help guide revisions of the training

system and materials . Summaries of the questions and answers are

presented below .

Overall Assessment of the Training System. Only 2 of the 20 operators indicated that they had ever received like training , and this had

occurred in high school. When asked for their overall impression o~ the training compared to the conventional classroom method, 13 indicated it was “definitely better,” and 7 indicated it was “somewhat

better.” None indicated it was “worse” or “no difference .” Nineteen

operators indicated they would like to receive more training using this

technique. Overall, the subjects were quite favorably disposed toward

[ 

the training given.

Best and Worst Things About the Training~ System. The following

answers were given to the question “Wh,it did you l ike the most about

this system of traininq?”

• Tolerance allowed in answers;

• Open—minded instructors ;

30

k ~~~~~~ - --- -—- -- ~~~~~~~~~~~~~~~~~~ ~- - - - - ______ —
I 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

o — 0 0 0 I) I I I .4 I .-4 I $ I W I I I .-4 I I I

H 01.4 00 0 0’ O W 0 CC I$~ CO O C C’ 0’ 0 0C C

8 ‘.~ ‘~~ ‘.‘ ,‘ .‘ ‘.‘ .‘

0

-4 0 4) -4 0’ r4 - Ii 0’ 0’ • 0 C • 0 0 0 C C C ’ C 00 C

0 0’ -4 ‘-4 N .4 I I -4 I .-4 .-4 I .-4 I —4 ~-4 I 0 4 )

t~14

W 0’ 0’ 400 ’ 0 00’ C) NO’ 0 0 ~ 0’ 0 NW 6’ (‘ 0’

4)

-4

0 000 0 00 0 0 0 0 0 U I .4 I I ..4 .4 ‘-4 I .-4 I I I —4 -4 —4 .-.4 .-4 .4 I

OWOOS O ’ 0 ’O N O C ’ O W W O’ W 0 ’O ’ 0 0 ’

H ).

~

o 40 40 40 40404040404040404040 ~ 404040 4)

0 0 1.) 14 4) C. 0 0 C’ 0 0 -

4) C U ‘-4 .4 .‘4 ,-4 .4 4 0’ 04 .4 >. 0 CC’ • 0 . . 00 C’ • 0 I) QU .4W ‘-4 .-4 0’ .’4 I 040’ 0’ -~ .4 N —4 04W I “-I

0’ a. 4I

I -.4 “ I I —4 I ‘-4 I .‘4 I 4 -~ I

41 1k 14

0000’ 0’ 0 0 0’ 0 0’ 0 0 0 4 0 0 0 ’ 0’ 0’ 0 0 ’ p -.4 -4 .4 -4 .4 _4 -4 .4

41 $4 . 4 4 ) 0 . 4>. 0 0 0 C’ 0 0 I.) ..4 I I ‘4 I ‘4 I I —-4 I I ‘-4 ‘-4 4 I I I

14

0’ 0 0 0 0’ 0 00 ’ 00 N 0 0 040 ’ 0000 0

. 8 -

~ .

~ -

~ .4 ..4 ‘..4 ,.4 ..4 ...4 ..4 .-

~ .4 —

.414

C

2 ‘..4 N 4~ ~~ 414 40 NW 6’ 0 -.4 N~~~~ ~ 4(4 40 NW 0’ 0 .4 — .-4 4 ~ 4 .4 .4 -4 ..4 .4 (44

31

~~~~~~~~~~~~~~~~~- ~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~- - -~~ -- -
• Worked at own pace;

• Made learning a little more interesting ,

• Lessons were easier to understand;

• Question sheets were not too long ;

• Checking own answers ;

-~~ • Relaxing atmosphere, informality , not as boring;

• Refreshed memory ;

• Lesson, then application and help from instructors;

-

~ • Had examples on paper to help you , whereas in the classroom

you don’t;

• Absence of boring lectures and excess verbiage; and

• Learn a little more because you get tested right then.

The following answers were given to the question “What did you

dislike most about this system of training?”

• Nothing;

• Did not have time to get important items into personal notes

for reference;

• Not enough breaks;

• Too much tolerance allowed on some answers;

• Some of material was incorrect;

• Tired at the end of the last couple of lessons , and you don’t

always grasp the full understanding;

• Some of the material was not presented very clearly-- too easy

to misunderstand questions;

• Too much time waiting in line between stations to get your

materials,

• Not enough discussion;

• Too long, too many tests over too short a period;

32

‘ - . ‘.‘..- -

- - -- —att a

~~~~~~~~~~~~~— _ -
- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~ 

I -- ~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ -- - -

~~~~~~~~

• Went into too much detail on instructinns at times; and

— • The feeling of hurrying to get done.

Suggestions for Improvement of the Training System. The following

answers were given to the question “If you could make some changes in

how this training system operates what would you change?”

-‘ • 

• Use better equipment (note : two machines in the testing phase

caused difficulty);

• Give 10—minute break every hour;

• Increase the probability that the student actually uses the

material, perhaps with longer or more complicated test;

• Leave it the way it is;

• Have more instructors available (to reduce waiting in line);

• Give out the lesson and practical exercise at the same time;

• Include lecture and group question and answer period , also

hands—on time for new students;

• Make some of the more difficult questions more simple; and

• Do not know .

Student Assessment of Specific Lessons. In answer to “Were there

any lessons that you felt you did not need?” all 20 operators answered

“No.” In answer to “Were there other areas of content that you feel

should have been discussed or reviewed?” 16 operators answered “No .”

Topics suggested by the remaining four operators were (1) physicalsecurity sensors and (2) sensor-implantation information .

The ninth question requested the operators to rate each component

of the lessons in terms of clarity of presentation and difficulty to

answer. Table 22 shows the results of the operator ratings . The

weighted sums were computed to make some elementary within-lesson comparisons of lesson content, practical exercise questions (PFQ) and

criterion exercise questions (CEQ) as well as across-lesson comparisons. The weighted sums were also totaled for each lesson in order

to assess the overall effect of the lesson .

Note that the lower the weighted sum, the greater the clarity as

perceived by the operators. However, the lower the weighted sum, the

greater the difficulty as perceived by the operators. Therefore, a “1”

on the rating scale means very clear or very difficult. The use of

these data, along with the within-course performance data and the

33

~ 

- ---- — --- -

- —---—-- - - - 

-

—— —~

___;_____~

______ __-___- —--
__________________ I ~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~--~ -

~ ----- ---.-—- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Table 22

Operator Ratings of Clarity and Difficulty on Lesson Materials

Clarity Difficulty

Very Not Weighted Very Not Weighted

clear clear sums difficult difficult sums

Lessons 1 2 3 4 5 1. 2 3 4 5

1. U~~ ruler 15 4 1 28

PEQ13 4 2 1 32 1 1 3 213 85

CEQ 13 3 2 1 30 1 4 2 12 82

90 167

2. Two—sensor

strings &

formula

review 10 7 2 1 35

PEQ 1O 8 1 1 35 2 8 6 4 72

CEQ B 7 2 1 2 42 4 3 8 4 69

112 141

3. End—points 11 S 2 1 1 36

PEQ 9 8 3 34 5 3 7 5 72

CEQ 8 5 5 1 1 42 66 4 2 58

112 130

4. Column length

quantity 10 5 2 2 1 39

PEQ 1O 5 2 2 1 39 1 3 3 5 8 76

~EQ l2 4 1 2 1 36 1 3 2 310 75

114 151

5. Non—stairstep

activation

patterns 10 6 3 1 36

PEQ 11 6 1 1 1 35 3 6 5 5 66

CEQ 1O 5 3 1 34 3 1 5 4 5 61

127

6. EMID 12 4 3 1 33

P~Ql0 6 2 2 36 1 3 2 6 7 72

CEQ 7 4 4 4 43 3 5 7 4 69

iii 141

7. MAGID 7 9 2 1 1 40

PEQ 9 7 3 1 37 1 4 2 7 5 68

CEQ 11 6 2 1 34 2 1 4 6 6

111 138

8. DIRID 11 3 3 2 1 39

PEQ 1O 6 2 1 1 37 1 3 2 3 8 65

CEQ 1O 4 4 1 35 1 2 3 2 9 GS

111 130

34

~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . - - ~~~~~~~~~~~~~~~~~~~~~~~ -
- - - ~~•

~‘•‘-•~‘I I - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~ - -- 

~~~~~~~~~~~~~~~~~~~~~~~ 

- I- ~~~~~~~~~ ~~~~~~~~ •I --

pretest and posttest results, would allow for a more effective revision

of the materials .

The following sumearizes the answers to “List any suggestions you

miqht have for improving a specific lesson or component.”

• Have this kind of work available more often;

• Pick up all materials in one trip;

• Make note-taking possible;

• Use better equipment (machines in training course),

• Give 10-minute information to all of them;

• Reword some of the lesson sheets to increase clarity ;

• Add a little more information to all of them;

• Break all sessions down to about 3 or 4 hours at a time; and

• Read lessons aloud and explain them.

.Iob Aid Performance Measures

Six target variables were statistically analyzed to determine the

effectiveness of the nomograph job aid in combination with the tiCS ruler :

TT1, TM, target type, column length, target quantity, and reporting

time . Table 23 presents the average operator performance on the pretest and posttest for each of the variables . As shown in the table,

all of the variables tested are statistically siqni ficant. It is coneluded that use of the UGS ruler and noutoqraph substantially improved

operator performance.

The UGS ruler, which is used to measure T’P1 and TM values, resulted

in 63% and 121% improvement, respectively . Note that the operators had

previously been trained and given uc

~ 

rulers (separate from the nomograph) to use during the self—paced training . The effect of this on

the above-mentioned comparisons is difficult to judge. However, the -

~~ improvement scores are probably lower than they would have been had

the operators not been exposed to the first experiment . Because the

operators had been exposed to the idea of accurately measuring TT1 and

TM to within .1 minute by using the tiCs ruler, they were inclined to

achieve this same accuracy . Prior to the training experiment the

operators had been taught to round off their time estimations to the

nearest half-minute . However, over half of them were estimatina their

L time values to the .1 minute during the above pretest. If they had

retained the half—minute criteria , their T’r1 and TM correctness values

35

I

-— 

--

~~~~~~~~

- —-- - - - 

-
~~~~~ I

~~•~ ~~~~~~ r 

~~vi~

-

~’

-

— — ,-4 — — — — tn

o o o o o o o . . . . . . .,

~~~~~~

~~~~~~~~~~~~~ ~~ . ~~~~ N N 5’ N ~

5 m .

~~

j E 

~ ~~~~~~~~~~~~~ 

? ~~

L!.

~ ~~~~~ ~~~~~~~ 

.

~~

~ I - tI

~ (5 & aU

~ 

0 0

(4; (,; .

~

— 36

-- iI~~~~ ~~~~~~~~~~~ - - -- --.—•- -

_______ - -

~ --- - -

- 

_

~~~~~. 

— -

~~~~ — ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
r’u~

(and related computations) during the pretest would have been poorer

than they actually were . Thus, exposure to the UGS ruler and the concept of accurate measurement probably reduced the pre- and posttest differences (which were substantial) .

In summary , the following percentage improvements occurred : target speed, 97%; target type, 7%; column length, 67%; and quantity , 41%.

The relatively low percentage improvement shown for target type should

not be misunderstood. Target speed is the major determiner of target

type, and the percentage improvement for target speed is substantial .

The criterion of 150 meters per minute was used in this study , i.e.,

targets traveling slower than this speed were called personnel and targets traveling equally as fast or faster were called vehicles . The • number of times in which increased information accuracy (as provided

by the noniograph) will make the difference between calling a particular

target personnel or vehicle will differ depending upon how close the

speed is to 150 meters per minute. The number of times in which it

oc - urred in this study was left to chance, as it occurred in the materiaJ obtained from the field.

A 15% improvement over pretest time occurred in time taken to report on each target. The percentage improvement is lower than had been

expected . However, analysis of what the job aids do for an operator

indicates that reporting time might not be greatly affected . The job

aid certainly should reduce the time required for performing calculations. However, measurement of TM x ‘N’1 with the ruler should require

more time than just estimating those values .

The training and testing of the utility of the nomograph/UGS ruler

job aid combination occurred after the data collection on the individualized training study. Therefore, the operators had received a lot of

practice in making the necessary measurements and computing the speed

and quantity of targets . However, it was not known whether or not the

operators had reached their “normal” level of competence. If they had

not, then the additional practice received durinq Session I could increase their scores in Session II but could appear to be due to the

use of the nomograph/UGS ruler.

If learning or practice effects occurred in Session I (and to some

extent Session II), this increase should be evident in a comparison of

performance on the problems in the first half of each session with

that on the problems in the last half of the sessions . This assumes

that the problems in the first half are equal in difficulty to those

in the second half. Table 24 presents these data for the speed and

quantity computations. Because the second half of the problems in

each session actually resulted in lower operator performance scores,

it is concluded that no learning/practice effects occurred , but that

the differences between Session I and Session II were due to the use

of the ncmograph/UCS ruler lob aid.

L I

— ~~~~~ a S. .-’-- — —4 ISh~~~~ I

~~~~..2 ~ _J
- ~ 1

~~ 

~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —~~ -—- -- - - —- ~~-.------ ------- -- ____

Table 24

Number Right of First Half Versus Second Half

of Test on Speed and Quantity Estimations

Session I Session II Total

1st Half 2d Half 1st Half 2d Half 1st Half 2d Half

Speed 42 34 78 64 120 98

Quantity 45 43 72 68 117 111

Practice effects apparently occurred on the time variable. Table

25 shows a marked difference between the first and second halves of

both Sessions I and II, with the biggest difference occurrina in Session I. A plot of the individual items (Figure 3) shows this effect

more clearly. From items 1 to 5 in Session I there is a decrease in

time, after which it tends to remain constant . The average time for

the first five items wa~ significantly hiaher (.05 level ; t 2.34.

df ~ 8) than the average for the first five items. In Session II there

is an initial “warmup” or perhaps practice on th~ first or first four

items offered , after which time/item remains fairly constant . Therefore, it is concluded that most, if not all, of the session differences found in the time variable are attributable to practice (learning ,

warmup, etc.).

Table 25

Average Time for First Half of Test

Versus Second Half of Test

Session I Session II Total

1st Half 2d Half 1st Half 2d Half 1st Half 2d Half

Average

time 22.1 16.9 18.5 15.5 40.6 32.4

38 

-

~-—-

. 

--
_________________ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~1_

I

5

4 -

MINUTES 3 I

2

1 -

12 3 45 6 76910 1 2 3 4 5 6 7 8 9 10

-* $E$S~NI SESSION U

ITEM NUMBER

Figure 3. Reporting time by item.

CONCLUSIONS AND RECOMMENDATIONS

Cone lus ions

1. The individualized training program developed from an analysis

of operator errors improved the performance of UGS operators, as

follows:

a. An 18% increase in the number of detection rights (which is

one target per operator per hour on the average).

b. about half of the above increase occurred in the 2-sensor

string condition.

c. The largest increase in detection rights occurred in the hiahtarget-activity condition.

d. A 31% increase in the number of identification rights (which

is on the average, 2-1/2 targets per operator every 2 hours).

This outcome inc1~

udes the increase attributable to the increase

which occurred for detection rights.

-
__________ - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --~~~- --- ----- -

~~- • ~~~~~~~~~~~~~~~~ - •~~~~

e. Most of the increase in identification rights given above

occurred in the high-activity condition.

f. Session I resulted in 58% detection completeness (rights

divided by total number of targets) which is comparable to

the 55% completeness of a previous study (Martinek , Pilette,

and Biggs, 1974) using similar (but not identical) target

scenarios and operators. Session II performance increased

to 68% as a result of the training .

2. No significant number of false alarms was found .

3. If a target is detected, it will be identified right (as a vehicle

or personnel) about 85% of the time.

4. The following results occurred with respect to additional intelligence information associated with detecting and reporting targets :

a. A 63% increase in the number of correct TT1 values;

b. A 58% increase in the number of correct T

M values;

C . A 64% increase in the number of correct speed values;

d. A 63% increase in the number of correct quantity values; and

a. A 15% reduction in the time for target reporting .

5. Answers to a questionnaire given to the operators after the

training indicated a strong favorable reaction to the self-paced ,

individualized training approach . Thirteen operators indicated that

the training approach was “definitely better” than the conventional

military training they had received. The remaining seven operators

indicated it was “somewhat better.” None of the operators indicated

it was “worse” or that there was “no difference” between the two.

Operator reactions were obtained on all aspects of the training as

reported in the Results section. Although acceptance of the tiCS

ruler was not specifically treated in the questionnaire, the researchers noted that such acceptance was high , judging from the amount of

use it received and the difficulty encountered in getting the rulers

back from the operators.

6. In the second study, the nomoqraph/UcS ruler job aid combination

was favorably received and improved operator performance on the following variables.

a. 63% increase in correct TT1 values,

b. 121% increase in correct T

M values ;

40

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~

—-_-—— - -:-- -

_- _ -_-- - -- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~ ~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~ .,-I~ - - ~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~ ~~~ ~ ‘ - -

I C. 97% increase in correct speed values;

d. 7% increase in target identifications;

e. 67% increase in correct length of column values;

f. 41% increase in correct quantity values; and

I I g. 15% reduction in reporting time.

7. Use of 3— and 4-sensor strings produced about the same operator

completeness (80% and 69%) and better completeness than use of 2-sensor

strings (48%). However, more vehicle targets (presumably easier) occurred in conjunction with the 3- and 4-sensor strings than with the

2-sensor strings.

8. Considering both sessions, a higher completeness is found durinq

I periods of low target activity (76%) than high target activity (65%).

9. After training, operator performance on the 3- and 4-sensor strings

was 77% completeness with virtually no false alarms .

10. Scenario differences were found, despite the attempt to eliminate

them as a source of variation .

-

~ Recommendations

1. The improvements in operator performance that occurred as a resuit of the training lead to the recommendation that the lesson materials (or portions), together with the individualized traininq approach , — be refined and implemented as follows:

a. Provide review and on—the-job training in the field , and

b. Be integrated into the required training for tiCS operators

-

~ at Fort Huachuca, Ariz.

2. It is further recommended that the nomograph and tiCs ruler be

— 

- a. Taught during required training for tiCS operators at Fort

-

~ Huachuca, Ariz., and

b. Included as standard issue with the R0376, the BASS III, and

any newly developed recorders.

3. An additional study , controlling for target differences, should

be done comparing the 2-, 3-, and 4-sensor string conditions. If

1 operator performance is the only criterion, this study indicates that

41

L ___
- : -— ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ —- --- - —-- -

three sensors produce as much target information as four. Conceivably,

two sensors may be as good as three or four.

4. A report of the detection of a target should be sent forward

using two sensors (or more in cases where two are not sufficient)

with a followup report giving speed, number of objects, and type

of target. This will provide an alert function at the earliest possible time.

5. Operator performance is at an acceptable level .

6. In future tests and research concerninct UGS, scenario differences

probably will occur and should be controlled experimentally and/or

statistically .

42 

—-—------ — _____
rI

f - - I - - - r

REFERENCES

Craven, M. N. Human Factors Evaluation of the Terminal Message Proces—

sor (TN?) , Sandia Labs., N. Mex., July 1972.

h DCPG Phase III Field Experiment Report. Project MASSTER. December

1970 .

Lindquist, E. C. Design and Analysis of Experiments in Psychology and

Education. Boston : Houghton Mifflin Company , 1956 .

Martinek, H., Hilligos, R., and Lavicka, F. comparison of Three Display

Devices for Unattended Ground Sensors. ARI Technical Paper 299,

-; 1978.

Martinek, H., Pilette, S., & Biggs, B. The Effects of Work -Rest,

Target Activity , Background Noise, and String Size on UGS

Operator. Interpretation of Unattended Ground Sensor Records .

ARI Technical Paper 300, 1978.

Operator and Organization Maintenance Manual for Recorder, Signal Data

R0376/USQ Resdel Model No. 91662—1, DAABO7—69-0195, June 1969.

U.S. Army. Armored Cavalry Troop Test Report , HQ, Modern Army Selected

Systems Test and Evaluation (MASSTER) at Fort Hood, Tex., Vol. II,

January 1972 (a).

U.S. Army. Operational and Tactical Concepts for Employment of Unattended Ground Sensor, February 1971. U.S. Army Combat Surveillance and Electronic Warfare School, Fort Huachuca, Ariz.

ST—30-20—2. February 1971 (b).

U.S. Army . Program of Instruction of 17M20 Unattended Ground Sensor

Specialist Course and Unattended Ground Sensor Maintenance , Draft.

November 1972 (b).

U.S. Army . UGS Experimentation Test Plan (No . 1030) Report. Deputy

Commanding General (DCC) Project MASSTER, May 1972 (c).

1 1 U.S. Army . Unattended Ground Sensor Devices. U.S. Army Combat Surveillance and Electronic Warfare School, Fort Huachuca , Ariz.

30— 20-1. Revised September 1971 (a).

West, H. W. Effects of Environment on Microseismic Wave Propagation

Characteristics in Support of SID Testing at Fort Bragg, N.C.

Technical Report M-73—2. June 1973.

43

- - - I - ____ 

-—I-~~ .-—.—’ - - A
~ _•1__ • • —-- I ~~~~~~~~~~~~~~~~~~~~~ .- --_. 

~~ 1

Appendix A

LESSON--TWO-SENSOR STRINGS AND FORMULA REVIEW

c~~jective

To enhance awareness in detecting 2—sensor strings and to provide

recall or basic training in computing the column-length formula.

Purpose

Detecting target patterns from 2—sensor strings are important because they can frequently provide as much information as strings with

more sensors.

Concept

Two—sensor strings may be deployed for the following reasons:

1. Along roadways when there is a shortage of sensors,

2. On mountain or hilltop areas to detect movement, and

3. Where only two sensors are functioning out of a larger string

because of failure or damage . -

When 2—sensor strings are deployed together in the same sensor - field with 3, 4, and 5—sensor strings, it is easy to miss their activation patterns . Two—sensor string patterns can easily be overlooked because the larger string patterns are much more obvious . The

2—sensor string patterns may yield equally relevant information con—

cerning enemy movements, but to detect them sometimes requires more

careful observation .

In order to maximize the chances of detecting 2—sensor strings

the following guidelines are helpful:

1. Check the Sensor Data Record and Sketch Maps carefully for

H the location and pen numbers of 2-sensor strings.

2. When activations occur on both pens in the familiar stairstep pattern, the chances are good it is a target.

3. When activations occur on both pens, but not in the familiar

stairstep pattern, they may or may not represent a target.

Check other sensors in the area to determine if the activations could be the result of noise activity such as artillery

or aircraft.

45

- - I — 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- - — — 

~a-r~~~ 1

~~~~~~~_ -I

~fl ~~~~~~ ==~fl ~~~~ I

~~~~ 7 r,t ~~~:-
________________________ - - .—,.-—--.._— ,

~~_ --- y _. _— 

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~ 

— - - - - FThe Column-Length Formula requires a minimum of 2-sensors; there—

fore , it can be used on the 2-sensor strings.

To review, the Column-Length Formula is

LC = 

~~~~ 

- CDR , TM

where

L

c = length of the target column (in meters),

D = distance between the sensors (in meters,

T

M = time difference (in minutes) between the mid-point

of the first sensor activation pattern and the midpoint of the second sensor activation pattern.

= total time (in minutes) of the first sensor activation

pattern, and

CDR = Combined Detection Range (in meters) . The sum of the

detection range of the first sensor and the detection

range of the second sensor . CDR is used instead of DR

to eliminate confusion between the detection range (DR)

of one sensor and the detection range of two sensors

when they are added together (CDR) .

The following values were measured from the 2-sensor string in

Figure 1 (pens 3, 4). Check them for correctness.

T

M =2min.

= 3 m m .

The Formula is solved using the following assumptions :

D = 500 meters (m)

CDR = 600 meters (m)

L ~~ — (TT ) - CDR

c T 1

500zn (3 mm .) - 600m

2 m m .

= 250 ~~ ~ mm .)— 600m m m .

= 750m - 600m

L = iSOm

C

46 

- ~~~~~~~~~~~~~~~~~~~~~~~~ 

-
~~~__

Since the speed of the target is 250 rn/mm , it is considered

vehicular. To determine the number (quanti ty) of targets in the

column the L0 is divided by 50m as shown.

Quantity (Q) — — 15015 — 3 vehicles.

of course, if the speed is less than 150 rn/mm the would be

divided by Sm.

WHEN YOU ARE FINISHED WITH THIS LESSON TAKE IT TO THE TRAINING

MONITOR (STATION 2) AND PICK UP YOUR PRACTICAL EXERCISE QUESTION SHEET.

4-7

— ~~~~~~~~~ SlI&~ I?-II~~ It - III

~~~ I I- S 

— ~~~~~~~ - ---~~~~~~~~. 

- 

-L:~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - - -

Appendix B

SUBJECT INSTRUCTIONS

A. Test Procedure Training

MONITOR: Read the following instructions:

As you already know, you have been selected to participate in a

research effort to study UGS performance under various conditions as

they might exist in the field . We have been asked by the Departhent

of the Army to administer this exercise to you, collect performance

data, analyze the results, and submit a report. I want to make it

clear at the outset that our objective is not to isolate each of you

and attach a proficiency score to each of you. Our objective is to

- I determine the total capability of the UGS information potential as it

combines both the person (you) and the machine output (the UGS record).

However, your performance will be scored and given to your platoon

leader so that he will have some idea about how good you are compared

to the rest of the group in the readout of the I~~376 event recorder.

Since other groups of UGS operators have taken or will be taking

this exercise, I will be looking frequently at my notes to be sure everyone gets the same information. You will have opportunities throughout this entire program to ask questions.

To summarize the entire exercise, you will be given UGS sensor

H - records (X- T plots) to interpret. These plots, for the most part, are

target activations that have been collected under various field exercises primarily in the Fort Hood area. Involved in these exercises

were varying numbers of tanks, Armored Personnel Carriers (APC’s), and

personnel. These tests also contain noise activations which are typical

of wartime operation such as: fixed-wing and helicopter activity,

malfunctioning and unreliable sensors, radio interference, weather !

wind activity, and artillery shell bursts. All we ask is that you

interpret the X—T plots to the best of your ability and try to make

sense out of what sometimes might appear to you to be rather difficult.

H Let me stress that we have tried to make these records as realistic

H as we could.

H You should interpret these X-T plots as though you are under battlefield conditions. However, in wartime it will be a lot worse. You

won ’t slow down then, because your life may depend on how well you do

your job. However, at the risk of sounding melodramatic, keep in mind

that you should try, because there is a great need for the Army to

find out how typical UGS specialists will be able to function under

battle conditions that we selected .

You are important because you, as a group, represent the hundreds

of specialists that have graduated and will graduate f rots the UGS school

- 

—-—- ~~ - — -- - -— -

~~~~

--

=- -

~~~~~~~~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~ 

- -

~~~~~~ 

~~~ -H~~ 

— 

- - 

- A
— —~ — ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~ ~~

F - -. I-• I - -

-

~ i for a long time to come . Army deployment plans for UGS equipment and

personnel will be partly influenced based upon what y~~~~ can do.

MONITOR: Pass out the packets and read the following instructions:

The next sheet should be a TARGET LOG. On this sheet you will

record most of the target information you would regularly record on a

SENSOR ACTIVATION SPOT REPORT plus additional information . Note the

first column of the target log. This is the target pattern number.

For activation patterns on the chart paper that you think are targets,

circle the target on the chart paper and number it. Record the number

in the first column . For each target that you detect, always number

it and record it on your target log. If you detect a target, but

- ; later feel it is not a target, do not erase what you have written,

but simply write “No Target” and start recording the next target on

the space below. Number all your targets consecutively in sequence.

In column 2 record the clock time that you made a target detection and round it off to the nearest minute. You may use your wrist

watch, but be sure it is synchronized with the wall clock.

In column 3 record the pen numbers of the sensor string .

In column 4 record how confident you are that what you think is a

target really is a target. Use the terms for your confidence that are

shown at the bottom of the target log sheet: positive, h igh , 50/50 , and

low. Tell us how confident you are that the activation pattern you are

looking at really is a target , assuming that you will be sending this

information to your Company Commander during a battle situation . He

knows you are doing your best but he wants to know how confident ~~~ are in the information you are sending to him. If ~~~ are not conf ident in your information then he will place a higher weight on the

information being supplied to him by other intelligence sources such

as realtime IR reconnaissance, radar or electronic intelligence. The

confidence estimate should relate only to your detection of a target

and not necessarily to any of the following information.

.1 In column_5 record the direction of movement of the target . Use

the terms NW , NE , SW , and SE.

In column 6 compute the speed of the target using the formula

Speed (S) - fl/TM

where D = distance between sensors, and

TM = time of activations from center of the first sensor

activation to the center of the second sensor pattern .

Show your calculations for speed in the space provided.

~~und off your speed to the nearest meters/minutes.

So

- —- —- I--- I -- - - - 

-
- 

— ~~~~~~~ ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~— .—.----— - 

~~~~~~~~~~~~~~~~~~~~~~ - ---- -- —:~

-;

In column 7 record the target type, whether personnel or vehicu-

- J lar . If the speed is less than 150 meters/minute call the target

“personnel” and write P. If the target speed is greater than 150

meters/minute call the target “vehicles” and write V.

In column 8 compute the length of the target column using the

formula

L = S ( ? rl

)

~~~~DR

where S = speed calculated and recorded in column 6,

= total time the first sensor was activated, and

detection range which is the detection radius of the

first sensor plus the detection radius of the second

sensor added together . In this test the detection

radius of all sensors in a string is the same .

Substitute the values in the formula in the spaces provided and show

your work. Do your actual multiplication on scratch paper. Round off

your numbers where appropriate .

In column 9 record the quantity or number of units in the target.

If the target is personnel divide the length of the column by 5 meters

to determine the number of personnel. If the target is vehicles divide

the length of the column by 50 meters to determine the number of

vehicles.

In column 10 record the clock time in which you are finished with

the target and are ready to send the information to your Company

Commander.

The sequence of the information required on the target log was

planned , and we want you to try to use this sequence . Does anyone

have any reservations about this sequence?

MONITOR: Discuss the implant sketch (Field I) and read the following :

Remove your target log and place it aside. The next sheet should

be an IMPLANT SKETCH , which is marked on the upper right-hand side--

Pretest Part I. As you know, implant sketches show the location of

- - 

the sensor strings.

During the next couple of days, you will be using implant sketches

very similar to this one. The implant sketches will show the location

of the sensors along the roads and trails. Give the pen number for

each sensor, the string number, and the distance between the sensors .

51
- 

~~~~~~~~~~~~~

As you can see, the spatial relationships between the sensors are

not drawn to scale. This is the case with all the implant sketches

you will be given during this entire exercise. The purpose of this

implant sketch is merely to present you with the information that you

will need . There will also be times in which you may disagree as to

the placement of the sensor strings in the various deployment patterns.

Although certain string deployments may differ from what you feel or

have been taught is correct, accept the deployments presented, keepinq

- 

- in mind that in operational situations there are many factors involved

in deploying sensors. The sensor-string deployments are the ones used

in the Fort Hood tests and are adequate for purposes of this exercise.

The implant sketches will also provide you with information as to

whether the string contains a MAGID, EMID, or DIRID sensor. In this

implant sketch (Pretest--Field I) there is one MAGID.

Place the implant sketch aside. The next sheet should be a SENSOR

PROGRAM RECORD. This sheet provides the string number and field designation, sensor type, and detection ranges of the various sensors. This

sheet contains four sections to provide this information for four different sensor fields. On this sheet three sensor fields are shown:

I, II , and III. Right now we will be concerned with only the first

sensor field labeled Roman Numeral I. The sensor record or read-out

sheets you probably used to work with contain a great deal more information including geographical coordinates, sensor ID, sensor frequency,

mode, and inhibit times. The ones we will work with are simplified for

purposes of this exercise.

Notice that for detection radius, a value is given for vehicles

and for personnel. Detection radius varies greatly for vehicles and

personnel depending upon the sensor type, soil type, and gain setting .

MONITOR: Administer Pretest Part I and read the following:

Place your target log, implant sketch, and sensor record in a

convenient place near your event processor which is already loaded

with our X—T chart paper . Place your packet aside. We wil]. work

through a 30—minute practical exercise, in which we will use the materials we have just discussed . Fill out your target log and use your

implant sketch and sensor record. As you detect patterns that you

think are targets, circle the pattern on the chart paper w ith your

pencil and place the number of the target next to the circle. Then

fill out your tarqet log by first recording this number in column 1.

Are you ready? Start monitoring .

MONITOR : Turn on drive mechanisms and check each subject to be sure

all are working with the proper materials. When the first group of

noise activations has completely appeared , find out if anyone called

them targets. When the first, second , and third targets have each

S2

— —_.

~~~~.~~~~~~~~~~~~ _ _ _ _ 

~~

— - -— —

- - --- 
-- _~~~~ .1I

~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - 1

~l•~~~~~~ ~~~~~~~~~~ I~~ l__~~l ~~~~~~~~~~~~ Ill ~._~~~l ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -~ 1

appeared (in different time frames), find out who did not detect them

as targets, and assist these people. For the first target, work out

the calculations on the blackboard if necessary. Find out how this

class is performing and give assistance individually or for the entire - - class as required. ~nphasize aspects of the procedure that are not

being performed . Check each man to be sure he is properly filling in

all the information blanks on the target log.

At the end of the 30-minute period, turn off the event processors

no more than 1/2- inch above the line separating the 30-minute periods

and review procedures where appropriate .

MONITOR: Read the following :

During the practical exercises you will be given today, you will

be monitoring various 30—minute chart preparations. After each 30-

minute period you will be given a new implant sketch . We will begin

the next 30—minute period shortly . Get your packets and take out the

implant sketch marked Pretest--Part tI in the upper right-hand corner .

Do you have any questions concerning this one? Remember, you are numbering your targets consecutively (in sequence) so if you need additional target logs, let us know and we will supply them. Be sure you

are working with the appropriate Sensor Record . For the next 30-minute

segment you will be working with the Sensor Record, labeled Pretest II.

Are there any questions before we begin the next 30—minute segment?

All right, we will now start.

MONITOR: Conduct Pretest--Part II

Conduct in a similar manner as Part I except with somewhat less

involvement of the monitors during the test. c~

heck what procedural

problems are still occurring . At the end of the 30 minutes, turn off

the equipment. Address the entire class on procedural problems and

reiterate where necessary .

S. Pretest and Posttest Training

MONITOR: Read the following:

- You will now be given four 30-minute segments. You won ’t be

stopped after each one, but you will proceed through all four segments

as though you were working a 2—hour shift in the field. Take the matena ]. out of your packets. The first two sheets will be the Sensor

Record Sheet and Implant Sketch for your first 30—minute period, and

as you can see , they are the same as those you have used previously .

I would like to remind you that the wall clock is right there

(Monitor : point to it) to get the times required on the target log .

- - - 

~~~~~~~~~~~~~~~~~~~~~~~ - i

~~~~

._ . - 

- 

J
___

If you are going to use your wristwatch, set it with the wall clock

now. We will not be assisting you in any way to aid you in your target analysis. We will , however, be assisting you to make sure you

are looking at the right sensor record and implant sketch throughout

the exercise. If you have any questions with regard to whether or not

you are using the proper sensor record or implant sketch, please do

not hesitate to ask. Be sure to circle the activation patterns that

you think are targets. Number these patterns consecutively and record

in the first column of the target log. Are you ready? Begin.

MONITOR: Turn the equipment on. Check to see that all the equipment

is running satisfactorily. Every 30 minutes the monitor will make

sure that each subject is using the proper Implant Sketch and Sensor - Record.

C. Individualized Training Administration

MONITOR: Read the following instructions :

Yesterday you worked on a practical exercise that required you to

read—out an X-T plot. We haven ’t scored it yet, but from past experience with UGS operators such as yourself, we can say that -~1l of you

made errors. Today ’s training session will be directed toward errors

previously made by other UGS operators. We assume that you also made

- : similar errors. We believe this training plus the practice you re-

-

~ ceived will increase your effectiveness as an UGS operator . It also

keeps you out of PT and clean-up details.

This type of training has been used successfully in other educational systems and has been approved by training specialists . Mow

well it works out here is one of the questions we are trying to answer .

You will be asked to assess it, afterwards, in a questionnaire to get

the UGS operator’s point of view .

For most of you this training will be quite different from any

training you have received in the past. The training system has been

designed to allow you to proceed at your own pace, test your own know].-

edge after short segments of instruction, obtain immediate feedback

on your mastery of the concepts, and receive expert assistance when

you need it and want it. Those who are familiar with the concepts

can proceed through the lessons at a fast pace, while those who need

assistance can receive it when they need it.

In this self—paced training system the content is organized into

distinct lessons, each covering a short segment of instruction . Each

lesson contains a Lesson Sheet, a Practical Exercise Question Sheet,

a Practical Exercise Answer Sheet, and a Criterion Exercise Question —

Sheet.

~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ 

- - 

-
-

~ -

1. The Lesson Sheet is divided into three sections: Objective ,

Purpose, and Concept. The Objective section states what you

should be able to do at the end of the lesson; the Purpose

section discusses the importance of the lesson, and the

Concept section contains the instructional materials with

examples.

2. The Practical Exercise Question Sheet contains questions on

the material discussed on the Lesson Sheets. These questions

are intended to allow you to test your mastery of the concepts

in the lesson.

3. The Practical Exercise Answer Sheet provides you with the

correct answers to the questions and will allow you to check

your own knowledge.

4. The Criterion Exercise Sheet contains questions relating to

information on the Lesson Sheet. These questions test the

same thing as the practical exercise questions and are the —

final check for mastery before going on to the next lesson.

For this exercise there are two monitors and one subject matter

expert. They will assist you and monitor your progress through the

system.

1. One monitor will be located at Station 1 and will control

handout of the Lesson Sheet and he will score your CRITERION

EXERCISES.

2. The second monitor will be located at Station 2 and will

control all activities having to do with the PRACTICAL

EXERCISES.

3. The subject matter expert will be located at Station 3 and

will provide content assistance whenever it is needed .

There are five basic steps to be completed for each lesson: Read

the Lesson Sheet, work the Practical Exercise Questions (PEQ ’s), check

your PEQ’ s, answer the Criterion Exercise Questions (CEQ’s), and have

the Test Monitor check the CEQ’s.

The specific steps for each lesson are as follows:

1. Get the Lesson Sheet from Station 1, return to yo’~r desk ,

and study.

2. Take the Lesson Sheet to Station 2 and give it to the Test

Monitor.
— - ~~~- --.-I ---.--.——- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

a. Pick up the Practical Exercise Questions (PEQ’s).

b. Return to desk and answer the PEQ’s——answer all questions

on the sheet.

3. Take completed PEQ sheet back to Station 2.

a. Show the Test Monitor that you have answered the questions .

b. Pick up PEQ Answer Sheet and Lesson Sheet.

c. Return to desk , check your answers, and resolve all incorrect responses. Do not erase or scratch out any of your

original answers. We need to know your first respon~es

so that we can improve the materials. If you need assistance, go to Static-n 3.

4. When you feel you und rstand all the material take the Lesson

Sheet, the PEQ ’s, an1 ~

‘EQ Answer Sheet back ‘.

~ Station 2.

5. Take completed CEQ’s to Station 1 (N~te: Station 1 is where

you picked up the Lesson Sheet at the very beqirninq ); the

Test Monitor will check your answers.

a. If you got them all correct , pick up the next tesson Sheet

from the same Test Monitor and ccmplet~- it as -iou did this

one.

b. If you missed any questions , return to Station 2, and cet

all your materials for the 1 -’sson. Return to your desk

and re—answer the “uestions y’~.. “dssed . Do not erase

your original answer. We need th&t information in order

to improve the materials. Write the number of the question

you missed on the back of the CEQ She~

’t and put your second answer there . Take your CEQ ’s back to Station 1 to he

checked. If you missed them again , you will be required

to go to Station 3 for assistance . (Take all your materials with you.) If you got thom correct, you may proceed

with the next lesson.

Are there any questions you may have about the materials, personnel ,

and/or procedures? Go to Station 1 and begin.

56 

I— 

- 

-
F ~~~~ •1 -- — - - - -

~~~~

---

~~ -- — -

~

--

~—-—

t

Appendix C

ASSESSMENT OF PRACTICE EFFECTS IN INDIVIDUALIZED TRAINING

Because of the shortage of operators to participate in the individualized training study, it was impossible to run a control group

to determine the effect of practice . Although some practice obviously

occurred, it is possible to show that its effect on detection and

identification performance is negligible. Table C—]. shows detection

and identification rights as a function of 30—minute scenario segments.

Operator performance for detection rights and identification rights is

presented across the 30—minute segments. Segments 1, 2 , 5, and 6 are

low—target-activity segments, and segments 3, 4, 7, and 8 are highactivity segments.

Table C—i

-; Detection and Identification Rights

Session I Session II Total

30—minute segments 30—minute segments number

in order administered in order administered of

1 3 5 7 rights

2 4 6 8

Detection rights

Odd segments 51 58 52 75 236

Even segments 31 68 39 80 218

Identification

rights

Odd segments 43 46 50 64 203

Even segments 27 50 35 68 180

~~~~~ For learning or practice effects to have taken place , the total

numbe r of rights for the even—numbered segments would have to be

greater than the total number for the odd—numbered segments (assuming

- ; equal target difficulty over all segments) . However, for both detection and identification rights, the total number of rights for the

even-numbered segments is below that for the odd—numbered segments.

These results strongly support the contention that practice or learning

effects did not have an important impact on the operator performance

increases and that, therefore, these increases are due to the training .

57 

—

_

~~~~~~~

_

~~

_ — — - A
l

y 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ ‘

~~~ -~~-— ~~~~~~~~~~~

- - — -

- - In addition, previous ret~

earch (Martinek , Pilette, and Biggs,

- : 1974) has shown no increases in performances associated with 8 hours

- of practice on scenarios highly similar to those used in this study.

Moreover, the operators used in that study had just graduated from

school and, therefore, should show the largest increases attributable

to practice (i.e., experience).

:;J

I

~~

58

- - ~- - - -- - ____ .- -—~~ -- - I~~~~~~~~~

- ~~~~~—

—~~~~~—-— —- __________
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

DIST1IBUTION

- A~~

I Dsstnbutson List

4 OASO (M&RA( 2 HQUSACOEC. Ft Ord. ATTN: Library

2 HODA (DANa-CR) . HOUSACOEC. Ft Did. ATTN : ATEC-EX— E -Hum Factor,

1 HQDA IDAPE-PBR 2 USAEFC, Ft Isnianrin Harri.on, ATFN~ Llbrsi-y • I HODA (DANA-An) I USAPACOC. Ft Sen(ammn Harrison, ATTN: ATCP-IIR

1 I’IOOA (DAPE-HRE-POI I USA Comm-Elect $cts, Ft Monmouth, ATTN : ATSN-EA

I HODA (SGRO-ID) I USAEC. Ft Monmouth, ATTN : AMSEI,—CT--HDP

I HODA (DANI.OOT-C) I USAEC, Ft Monmouth, ATTh : AM$EL-PA-P

I HODA (OAPC-PMZ-A) I USAEC, Ft Moninouth, A1TN: AMSEL.—S (—CB

I HODA (OACH.PPZ-A) I USAEC. Ft M*nmouth. ATtN~ C, Fad Div Br

I HODA (DAPE•HRE) I USA Materials Sys AnsI Agcy , Absrt’aen , ATTN~ AMXSY—P

I HODA (OAPE-%4PO.CI 1 Edgewood A,wtel , Aberdeen, ATTN: SAREA--aL—H

I HODA (DAPE-DW) 1 USA Did Cu B Salt, Aberdeen , ATTN: ATSL-TEM--C

I HODA (DAPE.HRL) 2 USA Hum Engr Lab , Ab,rde.n, AIIM: Library/Dir

I HODA (DAPE.CPS) I USA Combat Arms Tog Bd, Ft Banning, A’TTN: Ad Supervisor

I HODA (DAFO-MFA) I USA Infantry Hum Rich Unit, Ft Banning, AIIM: Chi.f

HODA (OARO-ARS.P) 1 USA Infant ry Bd, Ft Banning, AIIM: STEBC—TE—T - - I HODA (DAPC•PAS.Al I USASMA, Ft Bliss, ATTN: ATSS—LRC

I HODA (DUSA-OR) I USA Air Del Sch, Ft Bliss, AiIM: ATSA- CTD--ME

I HODA (DAMO-ROR) I USA Air Del Salt, Ft Bliss, ATIN: Tech Lib

1 HODA (DASGI I USA Air D,t Bd , Ft Bliss, A’rrN: FILE S

4 I UOOA (DA1O.PI) I USA Air Def Bd, Ft Bliss , ATIM : STESD—PO

I Chief, Consult Div (DA.OTSG), Adelphi , MD I USA Cmd & G.n,ral Sit College, Ft Leavenworth , AIIM: Lib

I Mil Ant . Hum Rae, 000RBE , OAD (E&LS) I USA Cmd & Ganeral Sit College , Pt Leaven worth, ATTN : ATS* -ãE—L

I HO USARAL , APO Seittle , ATTN: ARAGP-R I USA Cmd & General Sit Collega, Ft Le.v.nworth, ATTN : Ed Advisor

I HO Flits Army. AIIM: AFKA-OI-TI I USA Combined Arms Cmbt Des Act Ft Leavenworth , AIIM: OepCdr

2 HO Filth Army, Ft Sam Houston I USA CoMbined Arms Cmbt Div Act, Ft Lesvanwo,th . AIIM: CCS

t Dir. Army Stf Studies Ott , AIIM: OAV CSA (DSP) 1 USA Combined Arms Cmbt Div Act, Ft Leavenworth , ATIN : ATCASA

I Ofc Ch ief of Sit, Studies Ofc I USA Combined Arms Cinbi Div Act, Ft Leavenworth, AlIN: ATCACO-E

I DCSPER, ATTN: CPS/OCP I USA Combined Arms Cmbt Div Act , Ft Leavenworth, AIIM: ATCACC-CI

I The Army Lib, Pentagon , ATTN : RSB Chief 1 USAECOM , Night Vision Lab , Ft Balvoir, ATTN: AMSEL—NV—SO

I The Army Lib, Pentagon, AIIM: ANRAL 3 USA Computer Sys Cmd , Ft Belvoir, ATTN: Tech Library

1 Ofc, At Sect of the Army (R&D) I USAMERDC , Ft Belvoir , AIIM: STSFB—DQ

I Tech Support Otc, OJCS I USA Ersg Sch, Ft Balvoir. AIIM: Library

I USASA, Arlington, ATIN: lARD-I ¶ USA Topographic Lab , Ft Belv oir , ATTN: ETL - ID—S

I USA Rich Otc, Durham AIIM: Life Sciences Dir I USA Topographic Lab, Ft Belvoir , AIIM: STINFO Canter

2 USARIEM, Natick , ATTN: SGRD.UE.CA I USA Topographic Lab . Ft Belvoir , AIIN: ETL— O SL

I USATIC, Ft Clayton. AIIM: STETC-MO.A I USA Intelligence Cu & Salt, Ft Huachuca, AIIM: CTD—MS

1 USAIMA , Ft Bragg, AIIM: ATSUCID-OM I USA Intelligence Ci, & Salt, Ft Huachuca, AIIM: AIS—CTD - -MS

I USAIMA, Ft Bragg, AIIM: M arquat Lib I USA Intelligence Cu & Salt , Ft Huachuca. AITN: ATSI—TE

I US WAC Cu & Sits. Ft McClellan , AIM: LIb 1 USA Intelligence Cu & 5th, Ft Huachuca, AIIM: ATSI-TEX--GS

I US WAC Cu & Sits, Ft McClellan, AIIM: Tog Dir I USA Intelligence Cu & Sch, Ft Huachuca , AIIM: ATSI—CIS--OR

I USA Quartermaster Salt, Ft La., ATTN: ATSM.TE I USA In~

elligence Cu & 8th. Ft Huachuca, AIIM: ATSI-CTO-DT

I Intelligence Matin al 0ev Ofc, EWL, Ft Holablrd I USA Intelligence Ctr & Sch. Ft Huachuca , AIIM: ATSI—CTD—CS

1 USA SE Signal 5th , Ft Gordon, AIIN: ATSO.EA 1 USA Intelligence Cu & Salt, Ft Huacisuce , AITN: DAS/SRD

1 USA Chaplain Cu & Sits, Ft Hamilton , AIIM: ATSC.TE-RD I USA Intelligence Cu & Sch, Ft Huachuce, ATTN : ATSI—TEM

1 USATSCH. Ft Eustis, AT1’N: Educ Advisor I USA Intelligence Ctr & 8th , Ft Huachuca , ATTN : Library

I USA War College, Carlisle Barracks, AIIM: Lib I CDR . HO Ft Huachuca, ATIN: Tech Ref Div

2 WRAIR , Nauropsyclsiatry Div 2 CDR . USA Electronic Prvg Grd , AIIM: STEEP-MT-S

I DLI , SDA .-Monterey I COR , Project NASSTER . AIIM: Tech Into Center

I USA Concept Anal Agey. B.the.da, AIIM: MOCA-WGC I Hq MASSTER. USATRADOC. l.N0

I USA Concept Anal Agcy, Batheada. AnN: MaCA-MR I Re.eerch Institute, HO MA$STER , Ft Hood

I USA Concept Anal Agcy. Bethe,de, AIIM: MOCA-JF I USA Recruiting Clod, Ft Sherdien, ATTN: USARCPM—P

I USA Attic Test Cu. APO Seattle. AIIM: STEAC-MO-ASI I Senior Army Adv., USAFA000P1AC. Elgin AF Au, FId No. 9

I USA Atti c last Cu, APO Seattle, AIIM: AMS’IE-PL.-IS I HO USARPAC . DCSPER . APO SF 9 558, ATIN : GPPE—SE

I USA Armament Cmd, Redstone Ars enal, AIIM: ATSK - IEM I Stimeon Lib , Academy of Ha.lth Sciences, Ft Sam Houston

I USA Armament Cmd, Rock Isl and , AIIM: AMSAR-T DC 1 Marine Corps Inst., AIIN: Dean-MCI

I PAA.NAF EC, Adantic City , AIIM: Library 1 HOUSMC. Commandant, ATm: Code MINT SI

FAA-NAFEC , Atlantic City, AIM: Hum Ersgr Br I HOUSMC, Commandant. AIIM: Code MPI-- 20

I FAA Aeronautical Cir, Oklahoma City. AIIM: AAC.44D 2 USCO Academy, New London, AIiM: Admission

3 USA FM Arty Salt, Ft Sill , AIIM: Library 2 USCO Academy, New London, AIIM: Library

USA Armor Salt . Ft Knos , AIIM: Library I USCO Training Cu, NY , ATIN: CO

I USA Armo,Scls.Ft Knos ,AIIN: ATSS-0I-E I USCG TrainingCu.NY , AIIM: Educ Svc Ofc

I USA Armor Bali, Ft Knos , AIIM: ATSB-DT-TP I USCG, Psychol Re, Br, DC. ATm: o

~ 

1/62

I USA Armor Sits, Pt Knott , ATIN: AISB-CD-AD I HO Mid—Rangt -

~ v . MC Del, Quantico, ATm: PBS Div

59

- — — -- -

~~ -
— 

~~

‘‘

— _. -_

~~~~~~~~~~

-

- - I US Marine Corps Liaision Ofc , ANC. Alesendria, AIIM: AMCGS-F I Del B Civil Inst ot Envito Medicine, 

Csnada

I USATRADOC. Ft Monroe, AIIM: ATRO

— ED I AIR CRESS, Kensin

gton, 

AIIM: Info 

Sy, Br

6 USATRADOC . Pt Monroe . AIIM: ATPR 

- -AD 1 Milita.n

peykoloqlsls 

T(eneste, Copehage

n

I USAIRADOC. Ft Monroe, 

AIIM: AITS—SA I Mi

litary Attache. French Eml.,ssy. AIIM: Doc Sec

1 USA Forces Cmd, Ft McPherson. 

AIIM: Library I Medecin Chef, C.E.RI

P

.

AI -Arsenal, Toulno/Naval Fiance

2 USA Aviation Test Btt , Ft Pucker . 

AIIM: STEBG—P0 I 

Pun Scientific Off. 

Appt Hum Engr Rich Div. 

Minnl,y

I USA Agcy lot Aviation Safety, Ft Ruth.,, AIIM: Library oI Defense. New Delh

i

I USA Agcy for Aviation Safety , Ft 

Pucker . 

AIIM : Edut fts&

~ie~ I Pens Rsch Oft Libr ary, AKA . 

Isr ael Defense Forces

- - I 

USA Aviation 5th, Ft Pucker, AIIM: P0 Drawer a I Minittiult van 

Del ensle, 000P/l(t. Afd Sociaal

- 

- - I HOUSA Aviation Sys Cmd, St Louis, 

AIIM: AMSAV—ZDR ~~~~~~~~~~ Zalten. The Hague. Netherlands

2 USA Aviation Sy, Test Act1, Edw~~ds APB. AIIM: SAVIE—T

1 USA Air Oaf Salt, Ft Bliss , AIiM: 

ATSA TEN

- - I USA Air Mobility Rsth & Dew Lab, 

Moffeff Flit, AIM: SAVDL—AS

I USA Aviation Sth, Ret Tog Mgt . Ft Ruckar. AIIM : AIST—T-RIM

I USA Aviation Sits, CO. Ft Pucker, AIIM: A1’Sl’—O--

-A

I HO, USANC. Alesandria . ATTN: AMXCD-TL

I HO, USAMC , Alesandnia, AIIM: COP

I US Military Academy, West Point. AIIM: Serials Unit

I US Military Academy, West Point , AIIM: Ofc of Milt Ldcsisp

- - - I US Military Academy, West Point. 

AIIM: MAOR

* USA Standardlaation Op. UK, FPO NY , 

AIIM: MASE-OC

I Oft of Naval Rsch, Arlington, 

AIM: Code 4S2

3 Ofc of Nav.lRsch

.Arlington.

AIN:Coda 4BB

1 Oft of Naval Rich. Arlington, AIIM: Code 450

I Oft of Naval Rich, Arlington. 

AIIM: Code 441

1 Naval Aaroepc Med Re, La

b, Pensacola, 

AIIM: Acouu Salt Div

1 Nasal Aeraepc Med Re, Lab, Pensacola, 

AIIM: Coda UI

¶ Naval 

Aerovpc Med Ret Lab , Pensacola, 

AIIM: Coda LB

I Chief of NavPers. AIIM: Pert-OR

I NAVAIRSTA, Norfolk , 

AIIM: Safety Cu

I Maw Oceanographic, DC, AIIM: Code 6261. Charts & Tech

I Center ed Naval Anal , 

AIIM: Ooc Ctr

I NavAirSysCom, AIIM: AIR-63I3C

F I Ma y 

BuMed. AiIM: 113

I MavHelicopterSubSqua 2, FPO SF 90601

I AFHRL 

(FIl William AFB

I AFHRL(II) LowcyAFB

1 AFHRL (AS)

WPAFB

. OH

2 AFMRL 

(00,12) Brook, 

APD

I AFHRL IDOJN) Lacklend AFB

I HOIJSAF (INYSO)

I HOUSAF (DP)(XA

I

I AFVTSI IRDI Randolph APB

3AMRL HE WPAFB

, OH

2 AF Inst of Teals, WPAFB . OIl, AIIM: ENE/SL

1 Alt 

(XPTD) 

Randolph AEB

1 USAF AeroMed Lib, Brooks AFB 

ISUL—4), 

AIIM : 0CC SEC

1 APOSR (ML), Arlington

I AF Log Cmd, McC

lellan AFB. AIIM: ALC/DPCR

B

1 Air Force Academy, CO 

AIIN: Dept of Bel Son

S NasPens & Des Cit. Sen Diego

2 Navy Med Neuropvychiatrlc Path Unit, San Diego

1 May Electronic La

b. San Diego, AIIM: Re, Lab

1 Nay TrngCen. San Diego, AIIM: Code S000-Lib

1 MavPo.IOnaScts. Monte

rey, 

AIIM: Code SSAa

I MavPoetGraSch, Monterey, AIIM: Code 2124

I NavtrngEquipCtr, Orlando, AIIM: Tech Lib

1 US Dept of Labor, DC, AIIM: Manpower Admin

I US Dept of Justice. DC, AIIM: Drug Enforce Admin

1 Nat Bun of Standards. DC, 

AIIM: Computer Info Section

I Net Clearing Houte for 

NH --Info , Roc*vllje

I Denver Federal Ctr , Lake

wood. AIIM: BLM

13 Defense DocumentatIon Center

4 Dir Psych, Army Hq, flu

ietlI Ofte, Canberra

1 ScIentific Advu. 

Nil Sd, Army 

Hg. Russell 

ONe, Canberra

I Nil and Air Attache, Austrian 

Embassy

I Centre de Reche,che Des 

Patteum, Humalne da a Defense 

—

Natlonale. Bruteals

2 Canadian Jo

int Staff Washington

I C/Air Staff . Royal Canadian Ad, 

AIIM: Per, Bed Anal Sr

3 Chief, Canadi an Del 

Rich Staff . 

AIIM: 

CiVROS(WI

4 BrItish Oaf Staff. 

BritIsh Embetsy. 

Washinqtoh

60

A