Research Report 1253 1.EVr~~

SUMMARY OF ARI RESEARCH ON

REMOTELY MONITORED SENSORS

0Harold Martinek DTIC

SA~PR 1 5 M98

-Co

HUMAN FACTORS TECHNICAL AREA I

U. S. Army

Research Institute for the Behavioral and Social Sciences

July 1980

Approved for public release, distribution unlimited.

81 4 15 023

--. . . . ......',,,,'
U. S. A-RMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES j

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

FRANKLIN A. HART

JOSEPH ZEIDNER Colonel, US Army

Technical Director Commander

NOTICES

DI STRIBUTION: Primary distribution of this report has bean made by ARI. Please address correspondence

concerning distribution of reports to: U, S. Army Research Institute for the Behavioral and Social Sciences,

ATTN: PERI.TP, 5001 Eisenhowver Avenue, Alexandria, Virginia 22333.

FINAL DISPOSITION: This report mey be destroyed when it is no longer needed. Please do not return it to

the U. S. Army Research Instimute for the Behavioral and Social Sciencs.

NOTE The findings in this report are not to be construed as an official Department of the Army position,

unless so designated by other authorized documents.

, I

.L 11 : Jl -I' I 1 • 1.. .. n .. -,- - -
Unclassified •AO". • •5

So 'URIY CLSIIAIO u iTrroJ FTI PAGE (Wo (- 6 for .A IN,, P,--3- . 0'

REPORT DOCUMENTATION PGI -.-- U--1AWbWSTRUCTIONS BEFORE COMPLETING FORM 1. REPORT NUMISER ;. GOVT ACCESSION No 3. RECIPIENT'S CATALOG NUMBER

Research Report 1253/ A6.J t --

. TITLE( Subt... .d - ... . -RIOD COVERED

PUMMARY OF ARI RESEARCH ON REMO'TELY MONIT ORED 9 Fnal 3 - Bg6

ANSORS. M -ER

7. AUTHOR(e) S. CONTRACT OR ORANT NUMBER(O)

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10- PROGRAM ELEMENT, PROJECT, TASK

U.S. Army Research Institute for the Behavioral AREA A WORK UNIT NUMBERS

and Social Sciences ?

5001 isenhower Ave., Alexandria, VA 22333

It- CONTROLLING OFFICE NAME AND ADDRESS

Deputy Chief of Staff for Personnel IS. NUMER OF PAGES

Washington, DC 20310 40

14. CvITRI NAME It different fromi Controlling Office) IS. SECURITY CLASS. (of this report)

-- Unclassified 7 7 75-4. DECLASSI ICATION/ DOWNGRADING

SCHEDULE -

1S. DISTRIBUTION STATEMENT (of this Report)

I Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abrptact entered in Block 20, If different from Report)

ti. SUPPLEMENTARY NOTES

III. KEY WORDS (Continue on reverse slde It nceseary and Identl)' by block number) 4

Unattended Ground Sensors (UGS) Seismic sensors MOS 17M

Remotely Monitored Sensors (REMS) Training REMBASS

UGS, REMS surveillance Displays Personnel

Target acquisition Operator aids requirements

Acoustic senscis _

11L A9&rRAcr (cmit~ae maft oer ek~ Nf noweeemy wd Ideetir AT btick number) . The future battlefield will demand an increa3ed ability to acquirc,

transmit, process, disseminate, and utilize surveillance and target acquisi- tion information. One relatively new source of intelligence information is

remote monitoring of the battlefield, using seismic, acoustic, and magnetic remotely monitored sensors (REMS). When enemy personnel or vehicle movement

activates these remote sensors, a monitor display located behind our lines

indicates the activity. The operator can derive from this display not only

I _ ,_..... (Continued)

JA N 47 EW'noWOFt OV SIOUSOLET Unclassified

i SECU"rTY CLASItFICATIO OW • THIS PACE (Mbe. D anmE e

4 . , . . . . -Q . .. . . . I71"/

S. ..... ...; ' 1
Unclassified

SECURITY CLASIIICATION OF THII PAGC(om, Due enee,,

\Item 20 (Continued)

the enemy's presence but also sjch information as direction and speed of

convoys and personnel, number' .f vehicles' in a convoy, and convoy composition--e.g., armored versus .%eeled vehicles.

This publication summarizes ARI research on REMS user problems, including direct operational applications of present and future utilization of

REMS. Major findings are categorized into five areas--training, operator

aids, operational procedures, REMS system design, and personnel requirements.

The appendixes give information in the form of briefs for each pertinent ARI

document.

/

I

InTIS G" j) I ,(,•.. T'•

i3TIann

D ! . . .. ,.

Li

ii Unclassified

SECURITY CLASSIFICATION OF THIS PAGE('WPhui Dola Entered)

S77--.- -~

- •-- ..---- - - - -

- • -... . .. . ; -
I'

Research Report 1253

A

SUMMARY OF ARI RESEARCH ON

REMOTELY MONITORED SENSORS

Harold Martinek

Submitted by:

Stanley M. Halpin, Acting Chief

HUMAN FACTORS TECHNICAL AREA

Approved by:

Edgar M. Johnson, Director

ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY,

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Awnue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnes

Department of the Army

July 1980

Army Project Number Sensor Systems

20163739A793

A e f iii

ilA prd for pe,•ic r•ulesN distribution unlimited.

: r-I.
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks i, id for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally am. conveyed to appropriate military

agencies by briefing or Disposition Form.

I

4 , .,
FOREWORD

The Human Factors Technical Area of the Army Research Institute (ARI)

is concerned with future battlefield demands for increased ability to ac- quire, transmit, process, disseminate, and utilize information. The research focusei on the interface problems and interactions within command and control centers and concerns such areas as tactical symbology, information management, user-oriented systems, staff operations and procedures, and sensor systems integration and utilization.

Of special interest is the problem of human factors in the presentation and interpretation of surveillance and target acquisition information. One

relatively new source of intelligence information is remote monitoring of

the battlefield, using seismic, acoustic, and magnetic remotely monitored

sensors (REMS). When enemy personnel or vehicle movement activates these

remote sensors, a moniti-,r display located behind our lines indicates the

activity. The opery.tor can derive from this display not only the enemy's

presence but also such information as direction and speed of convoys and

personnel, number of vehicles in a convoy, and convoy composition--e.g.,

armored versus wheeled vehicles.

This publication summarizes ARI research on REMS-related human factors

problems, including the direct operational applications of present and future utilization of remotely monitored sensors. Major findings are summarised in the body of the report--more details and other results are given in the appendixes as briefs describing each experiment.

Research on remotely monitored sensor systems was conducted both inhouse and under contract with HRB-Singer, Inc., in response to requirements of Army Proj4ect 2Q163739A793 and to special requirements of the U.S. Army Intelligence Center and School, Fort Huachuca, Ariz.; Project AVID GUARDIAN,

U.S. Army, Europe; and the Remotely Monitored Battlefield Sensor System Project (REMBASS), Fort Monmouth, N.J.

EPH Z

hr.ical irector

4 iV

/

" d v . . .. !I

i.
SUMMARY OF ARI RESEARCH ON REMOTELY MONITORED SENSORS

BRIEF

requirement:

To aid in the development and utilization of remotely monitored sensors

(REMS) by investigati.ig and developing appropriate operator training, operational aids and procedures, system design from a hummn factors standpoint,

and personnel requirements.

Research Products:

In the training research, three self-administerab.'.e, on-the-job or

school training packages were developed and shown to result in improved

operator performance. Similarly, three operctor aids ware developed and

shown to significantly improve operator performance. Ro'search on operational

procedures determined preferred work-rest cycles, the minimum number of sensors required for both string and grid sensor employment, and the effects of

signal/noise ratio on the performance of operators using the acoustical remote sensor. Research on user requireit ts in new REMS cystems determined

the best type of display from several design options, the number of sensors

that an operator can handle in a display, improved concepts for the acoustic

sensor system, and operator-based bandwidth requirements. Much of this research affects personnel requirements, by reducing number of operators re- quired, enhancing operator performance, and helping predict current and

future needs.

Utilization:

Much of the ARI research on REMS-related human factors problems directly

applies to operational sensor utilization. The training materials and operator aids have been integrated into the U.S. Army Intelligence Center and

School, Fort Huachuca, Ariz. Research results on user requirements have

been incorporated in the design of the Remotely Monitored Sensor System

(REMBASS). These and other products are in use by other agencies in the

Army and the Federal Government.

vii

-ran Asui.ýg slam.
SUMMARY OF ARJ: RESEARCH ON REMOTELY MONITORED SENSORS

CONTENTSPage

BACKRON K G R ... O U .. ND.. . . ... ......... . 1

Introduction. ........................... . .. .. .. .. .. .... 1

The REMS System .. .. .. .. .................................. .. 1

Research Requirements .. .. .. ....................................... 1

Re search Design . . . . . . . . . . . . 2

organization of the Summ . ary .. .. .. .. ...... .. .. .. .... 2

TRAINING. .. ................................................. .. .... 4

Determination of Requirements..... .. .. .. .. .. . . . .*. .. 4

String Employment Text .. ............................................. 4

Grid Employment Text .. ................................................

Acoustic REMS Training .. ..............................................

OPERATOR AIDS. ........................................................... 6

Nomnograph/Ruler Aids. ................................................. 6

Grid Patching Techniques .. ........................................... 8

IOPERATIONAL PROCEDURES .. ............................................... 8

Work-Rest Cycle. .. ................................................... 10

Numnberof Sensors Needed. ............................................ 10

Signal/Noise Ratio. .................................................. 12

REMS SýýTEM DESIGN. .. ................................................... 12

Requirement. .. ................................ .. .. .. .. .. ... 12

Sensor Display Concepts. .. ......................................... 12

operator workload. .. ............................................... 14

Acoustic Sensing Concepts .. .......................................... 14

Bandwidth. .. ....................................................... 14

PERSONNEL REQUIREMENTS .. ................................................ 16

SUMMARy .. ............................................................... 17

REFERE1NCES .. ........................... . .. .. .. ... . . . . .. 19

APPENDI!X A.* THE VALUE OF SPECIAL TRAINING AND JOB AIDS FOR

IMPROVING UNATTENDED GROUND SENSOR OPERATOR PERFORMANCE . 21

B. THE VALUE OF SPECIAL TRAINING FOR THE INTERPRETATION

OF UGS EMPLOYED IN AGRID. .. ................... .. .. ... 23

ix

~s -aM&M 11100
rage

APPENDIX C. VEHICLE IDENTIFICATION USING THE ACOUSTIC SENSORt

TRAINING, SENSING CONCEPTS, AND BANDWIDTH . . . . . . .... 25

D. OPTIMUM PATCHING TECHNIQUE FOR SEISMIC SENSORS

EMPLOYED IN A GRID . . . . . . . . . . . . . 27

E. THE EFFECTS OF WORK/REST, TARGET ACTIVITY, BACKGROUND

NOISE, AND STRING SIZE ON OPERATOR INTERPRETATION OF

UNATTENDED GROUND SENSOR RECORDS . . . . . . . . . . . . . 29

F. THE EFFECT OF SIGNAL/NOISE RATIO AND BANDWIDTH ON

VEHICLE IDENTIFICATION, USING THE ACOUSTIC SENSOR ..... 31

G. COMPARISON OF THREE DISPLAY DEVICES FOR UNATTENDED

GROUND SENSORS . . . . . . . . . . . . . . . . . . . 33

H. COMPARISON OF FOUR UNATTENDED GROUND SENSOR DISPLAYS . . . 35

I. THE EFFECT OF WORKLOAD ON PERFORMANCE OF OPERATORS

MONITORING UNATTENDED GROUND SENSORS .... ........... .. 37

DISTRIBUTION ............................................. 39

LIST OF TABLES

Table i. Operator improvement due to training text .......... 5

2. Detection performance expectations using grids . . . . . . . . 6

3. Expected operator performance with continuously emitting

acoustic REMS ........ ........................ 7

4. Expected completeness detection performance by grid

density, target activity level, and patching technique . . . . 9

5. Expected detection accuracy performance by grid density,

target activity level, and patching technique ............. 9

LIST OF FIGURES

Figure 1. Research products ............ ...................... 3

2. Total correct detections for each work/rest cycle ........ .. 11

3. Total corrcct detections for work/rest cycle pairs ... . . 11

4. Plot of identification completeness by signal/noise

ratio and target categories ................ ............ 13

i,.1

""- •-----
Page

Figure 5. Effect of workload on percent detections . . . . . . . . . . 15

6. Research producte and users. ......... ............... . 18

I' xi

....
SUMMARY OF ARI RESEARCH ON REMOTELY MONITORED SENSORS

BACKGROUNE

Introduction

In September 1966, the Secretary of Defense directed the armed forces to

develop an infiltration interdiction capability to assess vehicle and personnel flow in Southeast Asia (Army Manual ST 3L0-20-1, 1971). The remotely

monitored sensors (formerly called unattended ground sensors, or UGS) developed in this program provide the Army's battlefield surveillance and target

acquisition system with another versatile capability of obtaining real-time

information about enemy movement.

Recent Army research and development efforts to improve the capability

of remotely monitored sensors (REMS) have been principally under the aegis

of the Remotely Monitored Battlefield Sensor System (REMBASS) program. Operational considerations on the use of REMS in the European environment have

been extensively studied under NATO's Avid Guardian program. Human factors

research by the Army Research Institute (ARI) to improve operator related

performance aspects of REIS is summarized in this report.

The REMS System

The different types of REMS in the Army inventory are categorized according to the method of remote sensing: seismic, acoustic, magnetic, electromagnetic, and infrared. Upon detection of an enemy movement, the sensor

transmits the data to an unattended relay or directly to the sensor monitoring station. The operator (MOS 17 M20) interprets the data presented on the

RO 376 event recorder (current) or the REMBASS sensor monitoring unit and

reports, at most, the target type, speed, direction, location, and number.

REMS may be tactically employed for offensive and defensive operations by units ranging from small independent patrols to full divisions for target

acquisition, landing (drop) zone monitoring, combat sweeps, ambush, monitoring of avenues of approach, base camp defense, convoy security, and border

surveillance. REMS can be employed in a regularly spaced "string" along

potential transportation routes, in a "grid" pattern to cover a large geographical area, or in various "alerting" patterns for simple detection of

activity in an area (e.g., drop zone).

Research Requirements

Significant technical achievements have been made in developing REMS.

Initially, little effort was directed toward the study of the operator and

the operator/equipment interface. Experience in Vietnam demonstrated both

the value and the human factors prolems of REMS. Depending upon field conditions, human errors of omission and commissiox were fairly substantial.

Planned extension of a sensor system, developed for the Vietnam conflict,

S1
to worldwide applications :ould increase existing problems and cause additional ones. In response to general and specific rsquirements for solutions

from REMS systems users and developers, ARI's research program (summarized

here) was initiated. Specific needs for research were generated by Proiect

Manager (PM) REMBASS, the U.S. Army Intelligence Center and School (USAICS),

and Project Avid Guardian; they were also derived from problems identified

by the ARI research program.

Research Design

The general objectives of the following rasearch were to better train,

assist, and manage REMS operators and to more effectively design and employ

equipment from a human factors standpoint, while considering operational

conditions. The underlying philosorhy of the research was operational realism and controlled, objective measurement of pertinent variables.

The operators participating in the experiments were always school

trained and usually experienced in the field; however, practice in the field

was often limited because of lack of equipment. Equipment used by the operators participating in the research was always a close simulation to

existing or proposed devices, at least for the critical dimension of coný4 cern. Sensor data presented to the operators via the above equipment were

accurate recordings of data collected in the field under controlled conditions, using operational equipment (except for the closely simulated acoustic

sensor equipment). The sensors were activated under operational procedures

and conditions by U.S. Army vehicles and personnel traveling in normal wartime modes. The above (operators, equipment, and sensor data) were combined

to represent actual wartime use of REMS. Perfect representation was neither

achieved--nor was it possible.

Measurement of criterion variables--detection rights and wrongs, identification rights and wrongs, time, direction, speed, and numbers-,were completely objective, based on actual knowledge of existing conditions (rights,

wrongs, time, direction, and, sometimes, number of targets) or on school

solutions (speed and numbers of targets). Assessment of whether or not true

differences existed between the conditions tested always relied on accepted,

statistical design and decision rules--not on someone's opinion of the

difference.

Organizati6n of the Summary

The research is summarized in terms of training, operator aids, operational proced.-es, REMS system design (RtMBASS), and personnel requirements

(Figure 3). Major findings in each area are summarized along with some of

the more pertinent data. More detailed description of the research is pro- vided in Appendixes A through I, which contain briefs of the cited research

reports and additional data.

Products of the research program have been utilized by USAICS, I-M

REMBASS, and others. While some of the operational applications are valid

across a broad spectrum of situations, others pertain only to specific circumstances or contexts. Additional use of research findings should be made

2

- ........-- *--,
PO 4) J'

jJ14 ju 4J .4

U)

0

$44

04 $40 C: 4.)1

S) r i.-

$4J$ U) H

E-4CJ U)U) 4J-

$4 v40

W -H 0 0G)0~

U2tA CflU
with caution when oonditions differ substantially from those under which

the research was performed. In no case should operational applications of

the results be instituted without a detailed review of the research. Interaction with ARI scientists is recommended to help insure effective, optimal

application of results.

TRAINING

Determination of Requirements

The first step in the training research program was to determine training requirements on the basis of a review of documentation, interviews of

senior instructors, and, most importantly, actual problems encountered by

operators in interpreting REMS output. The data and conclusions from analyzing operator errors in using the string and the grid deployment are reported

in Martinek, Pilette, and Biggs (1978a) and Edwards, Pilette, and Martinek

(1977), and for the acoustic sensor (the target identifier REMS), in Pilette,

Biggs, and Martinek (1979). In each, actual operator reports were analyzed

to document and categorize the factors leading to human errors. These analyses were the basis for development of appropriate training material that could

be used at the school for formal training or in the field for proficiency

maintenance, involving little (if any) instructor participation. In each experiment, the operators were pretested, trained, and posttested to determine

the gain in proficiency attributable to the training materials.

String Employment Text

The 6-hour-long, self-administerable training text developed under this

research project (Pilette, Biggs, & Martinek, 1978a) resulted in significantly improved operator performance (Table 1). Performance improved as

follows:

* Target detection, 18%;

e Target identification, 31%;

o Speed estimation, 64%;

* Number of targets estimation, 63%; and

* Time required for reporting, 15% (reduction in time may be due to

practice).

The gains in target detection and identification generally were greatest in

the more difficult operational situations tested--the condition of high target activity and the use of 2-sensor strings. In addition, all participating

operators reported that the training was "somewhat" or "definitely" better

than conventional training.

Secondary findings pertain to training and the operational use of REMS.

Contrary to experience in Vietnam, the trained operators reported very few

false alarms. Moreover, once a target was detected, it was identified correctly (vehicle or personnel) about 85% of the time. (See Appendix A for

more detail.)

4

- * :

/
Table 1

Operator Improvement Due to Training Text

Performance Operational No. correct Percent

variable situation PGetest Posttest improvement

Detection rights Low target

activity 4.1 4.6 12

high target

activity 6.3 7.8 24

Identification Low target

rights (vehicle activity 3.5 4.3 23

vs. personnel)

High target

activity 4.8 6.6 34

Correct speed A

estimation 3.9 6.4 64

Correct number

of targets 4.0 6.4 63

Time for target

reporting (in

minutes) 5.9 5.0 15

Grid Employment Text

Two days of training substantially improved operator performance (Pilette,

Higgs, & Martinek, 1978b). Target detection completeness improved by 38%,

speed-estimation accuracy improved by 23%, and target-direction-estimation

accuracy improved by 20%. The initially low false-alarm rate showed no significant change. After training, operators detected about 95% of the targets

during low workload conditions (5 to 8 targets per hour), and 61% when the

worklcad reached 27 targets per hour. Detailed existing and expected performance using the grid deployment is given in Table 2. These data indicate

that operators can provide highly useful intelligence information from REMS

in a grid pattern. (See Appendix B for more detail.)

Acoustic REMK Training

Research on acoustic REMS training was initiated because nnly this sensor in today's inventory (which does not include REMBASS sensors) has been

shown to reliably differentiate between vehicle types. A multiobjective experiment (Martinek, Pilette, & Biggs, 1978b) demonstrated that with about

5 ..........................
4 hours of training, operator performance with acoustic a~ensors improved

substantially for each of several target identification levels required.

The training package (self-administerable tape recordings) increased operator target identification performance between 20% and 50%, depending on

the level of target detail required (Table 3). An additional increase of

about 13% can be achieved in the field by using only the top one-third of

operators selected on their ability to interpret acoustic signals. (See

Appendix C for more detail.)

Table 2

Detection Performance Expectations Using Grids

Existinga Operator with 13 additional

Workload operator hours of training

5 tgts/hr 81% (4.0 tgts) 97% (4.8 tgts)

8 tgts/hr 88% (7.0 tgts) 94% (7.5 tgts)

15 tgts/hr 28% (4.2 tgts) 66% (10.0 tgts)

27 tgts/hr 34% (9.2 tgts) 61% (16.5 tgts)

aschool-trained and with some basic introduction to the grid employment.

OPERATOR AIDS

I The error analy:ses that led to the development of the need and problem

areas for the training research also indicated that certain simple operator

aids would significantly enhance operator performance. The use of these aids

would be taught at the school (see preceding section on training) and would

become part of the operating procedures (see the following section). However, this section on operator aids is different in that it concerns devices

or techniques that a single operator would use in the field.

Nomograph/Ruler Aids

These two simple devices are grouped together because they would normally

be used serially for the determination of speed and number of targets. They

were developed and validated (Pilette et al., 1978a) together for the string

deployment of REMS and were used successfully in several of the research

projects, in the school for training, and in the field during maneuvers.

Analysis of operator errors indicated that measurement of time on the

RO 376 tactical recorder and subsequent computations to determine speed and number of targets resulted in unreliable figures.. Use of the devices in a

controlled experiment (Pilette et al., 1978a) resulted in the following improvements in operator performance:

6

~~.-~~~--.-~~ ~ ..............-.--.--.-.--
"44

00

(q) 0 HD W O 0 '4' e N N

44041

00

044

*d 4.4* a

10 0

~4)

kc J4J ( 0% % . M 4

4.) 0

4)04

HH

$4 4

H H4

000

Fe 21

0 .0 -A4

V ~IU

-~~ 2 to 

4~r4 110 HO 0

OH ., 80 HH i

E 41 V1 4i>4

H242 MA N ) H0) >4 , 411 7

I-0U

> - - - 4j - - - - - -

-/-ý
0 A 97% increase in speed estimation,

"* A 7% increase in target identification (vehicle vs. personnel),

"* A 67% increase in length of convoy estimation,

"* A 41% increase in number of targets estimation, and

"* A 15% reduction in reporting time (probably due to practice).

Operator acceptance of the devices was high; many devices disappeared

aftex the experiment was over. The calculator envisioned for the sensor

monitoring unit 'SMU) in Project REMBASS would obviate the need for the

nomograph except for a manual backup and in situations (e.g., long-range patrols) not utilizing the SMU. The REMS rule; would continue to be useful in the new systems. (See Appendix A brief for more detail.)

Grid Patching Techniques

As the grid employment of REMS was a post-Vietnam-conflict concept, it

had not been taught at school or used in the field. The operators' problem

wit i this method is transforming and using the two-dimensional space of the

grid in the field to the one-dinensional representation of the grid on the

display device (the RO 376 tactical recorder or the REMBASS SMU). Four techniques were developed and validated (Pilette, Biggs, Edwards, & Martinek,

1978). Although clear-cut superiority for one technique across all the

variables in the experiments could not be demonstrated, it was concluded that

the row patching technique was best with operators who have received special

training, particularly when using low-density grids. T1,e column technique

is initially better (before training on this concept) because of its similarity to the patching technique used for the string deployment of REMS.

It was concluded that the major problem in employing sensors in a grid

is the completeness (not accuracy) of the operator's performance (omissions,

not false alarms). Tables 4 and 5 present more detailed data on expected

performance by relatively untrained and inexperienced operators using a

grid employment. (See Appendix D brief for more detail.)

OPERATIONAL PROCEDURES

Operational procedures are part of the task of the management of the

REMS system--how the managers "work" the operators and "work" (utilize)

the sensors. For any size REMS unit there is a limited supply of resources:

a given number of man-hours and sensors are available to cover a particular

area assigned to that unit. The manager must optimally work and rest the

operators to insure that fatigue does not reduce monitoring performance.

The numbers of sensor strings (and grids) monitored by one operator must

also be controlled, since too great a workload will reduce monitoring performance. Similarly, the number of sensors assigned to an avenue of approach (in a string) or to an area (grid) usually must be kept to a minimum,

based on acceptable operator performance and the requirement to cover all

avenues/areas with the limited sensor supply. Finally, the manager must

carefully assess which avenues or areas he can cover. The assessment is

based on many factors that include line-of-sight distance, especially when

using the acoustic sensor, because each relay used increases the signal/

noise (S/N) ratio and thereby reduces operator identification performance.

These requirements initiated the following research.

J8
Table 4

Expected Completeness Detection Performance by Grid Density,

Target Activity Level, and Patching Technique

9-sensor grid 24-sensor grid

Low target High target Low target High target

activity activity activity activity

Technique (%)

Column 88 46 91 45

Perimeter 90 34 84 36

Row 84 48 94 42

Zone 85 39 94 42

Average 87 42 91 41

Table 5

Expected Detection Accuracy Performance by Grid Density,

Target Activity Level, and Patching Technique

9-sensor grid 24-sensor grid

Low target High target Low target High target

activity activity activity activity

Technique (%) (%) (%) (%)

Column 96 96 79 84

Perimeter 89 91 86 95

Row 93 98 86 95

Zone 93 93 85 33

Average 93 95 84 92

9

.. 4

i' - - - -
Work-Rest Cycle

Typical findings in the research literature indicate that work without

periodic rest periods results in performance decrement. Similarly, this

research (Martinek et al., 1978a) demonstrated that REMS operators given

rest periods every 2 hours performed better than those given only a lunch

break. The drop in performance occurs during the second 4-hour period,

particularly in the last 2 hours (Figures 2 and 3). This indicates that

an operator could be kept monitoring (during critical times) for 4 to 6

hours without a drop in performance. No difference was found between a

15-minute break and a 1-hour break. (See Appendix E for more detail.)

unmber if Sensors Needed

General. Past doctrine recommended that 3 to 7 sensors be used in a

string for avenue-of-approach surveillance. Similarly, it was thought that

a grid with sensors spaced close together would provide better information

to the operator. However, the more sensors used, the greater the cost, the

more manpower required to emplace them, the higher the detectability, the

more radio-space required, and the greater the number of displays needed. Two

exp.riments (Martinek et al., 1978a; Pilette et al., 1979) investigated the

effect of string size on REMS operator performance, and one (Pilette et al.,

1979) compared a 9-sensor grid to a 24-sensor grid (each covering the same

area and targets).

String Size. The results of the two experiments comparing 2-sensor,

3-sensor, and 4-sensor strings are consistent. Under low-target-activity conditions, there were no differences in operator performance attributable

to string size. Under high target activity, the use of 3-sensor and 4-

sensor strings resulted in equal operator performance, but better than the

use of 2-sensor strings. Considering both operator performance and costeffectiveness, and given reliable sensors, 3-sensor strings are preferred.

However, a case could be made for 2-sensor strings. First, there were some

confounding experimental conditions acting to lower performance using 2-

sensor strings. In addition, assuming a shortage of sensors compared to

the number of approaches that the commander needs covered, 2-sensor strings

might be required and are as good as the longer strings for low target activity. (See Appendixes A and E for more details.)

Sensor Density in a Grid. Two densities of sensors were compared: a

9-sensor grid (500 meter spacing between the seismic sensors) and a 24-

sensor grid (250 meter spacing except for one row which had only 4

sensors). In the operational case the maximum spacing between sensors

would have to be determined on the basis of observed detection radius of

sensors, gain settings, etc. The experiment did not test the spacing of

sensors but, rather, whether or not redundancy of sensors (24-sensor grid)

aided the operator. However, it was found that the additional sensors (and

thus information) did not aid in the detection of targets but actually resulted in more false alarms, especially during low target activity when the

operator had more time to make errors. (See Appendix D for more detail.)

10

10
9o

\ Work/rest cycle 11 =• \\2 hr work, 15 min rest

N . Work/rest cycle I

o., 2 hr work, I hr rest

70 '° / \ 1hr rest

00 Work/rest cycle IV

4 hr work,

15 min rest

2 hours 4 hours 6 hours 8 hours Time period

Figure 2. Total correct detections for earn work/rest cycle.

S~~180 ,|

Work/rest cycles I & II

160

Wot'k/rest cycles I & III ooFig.re 2 ,\\, Work/rest cycles II&I V

Work/rest cycles II & IV

W'ork/rest cycles III & IV

120 I , II

2 hours 4 hours 6 hours 8 hours

Time period

Figure 3. Total correct detections for work/rest cycle pairs.

11J
Signal/Noise Ratio

The acoustic sensor was given special attention in the research program

because of its impoý..tance in the identification of vehicles in a convoy.

The research (Martinek, Pilette, & Biggs, 1979) concerned the effects of S/N

ratio on the classification of vehicles. The results indicated that the

following rule of thumb can be used when developing procedures for the employment of relays (increasing noise level) and designing new sensors: a 1%

decrease in operator performance occurs for every 1.5 dB loss in S/N ratio.

(See Figure 4 for performance curves.) It was also found that there is a

tendency for operators under high noise conditions (such as when using many

relays) to report any vehicle sounds as light- and medium-wheeled vehicles.

This implies that the operators use relative sound intensity in discriminating between vehicles and, therefore, use of complete automatic gain control in a

new sensor is contraindicated. (See Appendix F brief for more detail,.)

REMS SYSTEM DESIGN

Requirement

The design of most U.S. Army hardware systems entails many decision

points and trade-offs involving the interface of equipment and the operator.

ARI research in this area for REMS was done in close coordination with the

Combat Surveillance and Target Acquisition Laboratory, Fort Monmouth, N.J.,

concerning requirements for operator displays and the acoustic sensor for

Project REMBASS. These often were investigated in conjunction with nonequipment requirements (e.g., training) to increase the cost effectiveness

of the research. Similarly, data gathered on equipment related research

were used to determine needs for research in other areas.

Sensor Display Concepts

Two research efforts (Martinek, Hilligoss, & Lavicka, 1978; Edwards,

Rochford, & ShverA, 1977) compared the operational type of display (RO 376

Tactical Data Recorder) to several variations of a situation map display

proposed for possible inclusion in REMBASS. The situation map display consists of a map placed over small lights that correspond to the location of

the sensors in the field. The light blinks rapidly for each activation of

its associated sensor and then goes off until the next activation. The

first experiment (using the string employment) compared use of the RO 376

display, use of a situation map display, and use of time compression with

the iituation map display. With time compression, the operator could review in ax time any of the past activations; in addition, he had a forced

compressed time review (4 minutes long) every 30 minutes of all activations

during that time period. The second experiment (using the grid employment)

compared the RO 376 to three variations of the situation map display. Results of both experiments indicated that the operational type of display

as represented by the RO 376 was as good as or better than the situation

map display and its variations in terms of the accuracy and completeness

of target detections. The RO 376 type of display was selected for the

Sensor Monitoring Unit of REMBASS. (See Appendixes G and H for more

detail.)

12

I.
cc0. ~. ~V

~~ u0r%

I ~00

00

cc H

v40

131
Operator Workload

The number of sensors that one operator can monitor without serious

deterioration in performance determines, in part, the requirements X.or the

number of sensors that should be shown on new displays such as the SMU.

An investigation was carried out to determine operator performance as a

function of the workload (Edwards, Pilette, Biggs, & Martinek, 1978).

Workload is defined as the number of targets an operator must monitor due

tj the combination of the number of sensors and the target-activity level

for those sensors. Results indicated that operator performance decreased

rapidly as a function of workload (Figure 5). Clearly, an operator can

handle 27 sensors under low target activity (85% detection completeness)

and cannot handle very well 54 sensors under high target activity (50% detection completeness). However, the above figures pertain to relatively

inexperienced operators using REMS employed in a grid. It was believed

that, with special training and/or using the string employment, operators

could usefully monitor 60 sensors under high target activity. These data

provided a partial basis for the initial SMU design displaying the output

of 60 sensors. (See Appendix I for more detail.)

Acoustic Sensing Concepts

Th3 operational U.S. Army acoustic REMS, designed for the Vietnam conflict, is turned on automatically by seismic activity, transmits for 15

seconds, and is silent for 20 seconds. Use of this type of sensor in Europe

would result in a loss of at least 50% of the information about convoys,

since a typical aggressor vehicle column takes about 60 seconds to pass a

sensor. Two different sensing concepts were experimentally evaluated

(Martinek et al., 1978b), using various requirements for vehicle detail; both were judged to be better than the Vietnam type of sensor. Under the

most stringent reporting requirements, a small but significant difference

was found in correct identifications for the intermittent concept (34.4%)

over the continuous concept (32.1%). A greater difference is expected for

actual operational conditions because of an artifact in the experiment

favoring the continuous concept. Larger differences (not tested for statistical significance) were found for less stringent reporting requirements.

(See Appendix C for more detail.)

Bandwidth

Two experiments (Martinek et al., 1978b; 1979) investigated the effects of acoustical bandwidth on operator performance with the acoustic

REMS. One indicated that a bandwidth of 50 to 2,000 hertz (Hz) is better

than 50 to 1,500 Hz for vehicle identification. However, no difference

was found between 50 to 2,000 Hz and 50 to 4,000 Hz. This result was

chlecked in the second project where 50 to 2,000 Hz was compared to 50 to

4,500 Hz--no differences were found. Thus, the use of the operational

bandwidth of 50 to 2,000 Hz is sufficient for venicle identification. This

is not to imply that a different range (e.g., 450 to 2,400 Hz) but same

bandwidth would be adequate, since the lower 'requencies are probably

critical tc the identification of vehicles. (See Appendixes C and F for

more detail.)

14

4' 2).

j.1
100-

90-

271

80- 27, 54, 108, Numhr of pens

L - Low target activity

541 H • High target activity

70-

z

0 60- \ 1081

27H

I,-

a 50-

2

LU

"c 4U- Correction of hlgh-target-activity

"%*.- scores to eqjalize scenario

differences in target difficulty

30- 1 1 08H

20-

10-

0- ' I I I . .. ' " i I I " I

5 10 15 20 25 30 35 40 45 50

NUMBER OF TARGETS PRESENTED/30 MIN. (WORKLOAD)

Figure 5. Effect of workload on percent detections.

I

15

K,~ __ _
PERSONNEL REQUIREMENTS

The results of this research program indicate the complexities involved in forecasting personnel requirements. They imply that the number

of operators required depends on several factors, such as target-activity

level, number of sensors employed, type of employment, number of displays

used, training, and fatigue effects. Thus a first step in forecasting personnel requirements is to determine the threat (number and kind) in order

to determine the target-activity level expected in most operations. Aenother threat factor (number of approaches and area to be covered) must be

considered to determine the number of sensors to be employed and the number of displays required. These data, combined with the research findings,

give a first indication of personnel requirements.I

Another factor entering the equation is related to the research on

training and work-rest cycles. Increased performanece through training reduces the numbers of operators required. On the other hand, the fatigue

factor found in the work-rest cycle research increases the personnel

requirement.

One variable often ignored in forecasting personnel requirements is

the commander' s requirement for detailed information. The research on the

acoustic sensor demonstrated the large reduction in identification accuracy

due to requirements for detailed information. Lowering the requiremenit for

detail increases the accuracy of information. Likewise, part of the improvement found in the training research may be attributable to the operators

ignoring the requirements for calculating speed and numbers of targets.

Elimination of this requirement alone could account for several minutes of

time per target. Target reports would be more timely, and operator time

per target reduced, thereby reducing the number of operators required for

Combining the threat factor with operator performance data also can

identify some potential areas of serious system breakdown. Based on research on the grid and string employment of sensors (Pilette et al., 1978a,

1978b; Edwards et al., 1978), a trained operator would detect about 60 to

70% of the targets if the activity level is 27 targets per hour. Assuming

that the operator's display portrays 60 sensors, and 3-sensor strings are

used, one operator can cover 20 roads. Unfortunately, enemy columns (10

vehicles) travelirg at the normal convoy speed would pass a sensor string

every 3 mir'utes, or 20 columns per hour. in a rare case, this means that

400 targets (20 roads x 20 columns) could appear on one display in 1 hour

during a major attack. Even if the enemy were to use only 5 roads, 100

columns per hour would have to be processed by the operator. Clearly, more

operators or special processing and reporting procedures for the operator

are required in this critical overload situation. Another approach would

be to reassess the use of a computer for auitomatic target reporting.

The above workload figures do not necessarily apply to the new REMBASS

acoustic/seismic identifying sensor. This sensor provides more information,

but the operator's overload point may be reached earlier because there is

more information to process and a m~ore complicated display. Additional research would be helpful to assess the impact of new sensors and displays on

operator performance (and personinel requirements) to derive doctrine for

all situations.

16
SUMMARY

This report summnarizes ARI research on REMS in terms of training, operator aids, operational procedures, REMS system design, and personnel requirements. In the training research, three on-the-job or school selfadministerable training packages were developed, on the basis of the error

analyses of commnon operator problems, and validated. These training packages resulted in demonstrated improvement in operator performance, in both

the statistical and the practical sense. Similarly, three operator aids

were developed and shown to significantly improve operator performance.

Research on operational procedures determined preferred work-rest cycles,

the minimum number of sensors required for both string and grid employment,

and the effects of signal/noise ratio on operator performance with the

acoustical remote sensor. Research on user requirements in new REMS sys-'

tems determined the better type of display from several design options,

the number of sensors an operator can handle on a display, improved concepts for the acoustic sensor system, and operator-based bandwidth requirements. Many of the above findings also impact on personnel requirements.

Research on HEMS systems led not only to improved operator performance but

also to a reduction in the numbe.r of operators required and assistance in

predicting current and future personnel requirements. A summary of these

products and their uses is shown in Figure 6.

17
U) U

U))

H .

144)~ 4J0 0-

A00l I )

0 0 Me 0e

4 -

4.) 0

U 8 14 il

-4 (0 4- (-)

.0 r. tp ~

04 4J44J)

0~ 000

b) 0 4- U4

0 zId 4) 0 W

*4 4.) 0 C

44 .1 

0U U

)4J (d r(

0 P4

181

4 4.)________

ý42
REFERENCES

Edwards, L. E., Pilette, S., Biggs, B., a Martinek, H. The effect of workload on performance of operators monitoring unattended ground sensors.

ARI Technical Paper 231, September 1978. (AD A061 694)

Edwards, L. E., Pilette, S., & Martinek, H. Error analysis of unattended

ground sensor operators' reports. ARI Research Memorandum 77-1,

November 1977. (AD A077 921)

Edwards, L. E., Rochford, D. S., & Shvern, U. Comparison of four unattended

ground sensor displays. API Technical Paper 281, April 1977. (AD A039 056)

Martinek, H., Hilligoss, R. E., & Lavicka, F. Comparison of three display

devices for unattended ground sensors. ARI Technical Paper 299,

August 1978. (AD A060 558)

Martinek, H., Pilette, S., & Biggs, B. The effects of work/rest, target

activity, background noise, and string size on operator interpretation

of unattendad ground sensor records. ARI Technical Paper 300, June

1978. (a) (AD A061 043)

Martinek, H., Pilette, S., & Biggs, B. Vehicle identification using the

acoustic sensor: Training, sensing concepts, a.d bandwidth. ARI

Technical Paper 334, September 1978. (AD A062 601) (b)

Martinek, H., Pilette, S., & Biggs, B. The effecrt of signal/noise ratio

and bandwidth on vehicle identification using the acoustic sensor.

ARI Technical Paper 377, June 1979. (AD A073 715)

Pilette, S., Biggs, B., Edwards, L. E., & Martinek, H. Optimum patching

technique for seismic sensors employed in a grid. ARI Technical Paper

320, September 1978. (AD A061 573)

Pilette, S., Biggs, B., & Martinek, H. The value of special training and

job aids for improving unattended ground sensor ope3rator performance.

ARI Technical Paper 304, August 1978. (AD A061 046) (a)

Pilette, S., Biggs, B., & Martinek, H. The value of special training for

the interpretation of UGS employed in a grid. ARI Technical Paper 328,

October 1978. (AD A063 594) (b)

Pilette, S., Biggs, B., & Martinek, 11. Requirements for target identification training for the acoustic sensor operator. ARI Research Memorandum 79-4, April 1979. (AD A077 977)

19

- I[
APPENDIX A

THE VALUE OF SPECIAL TRAINING AND JOB AIDS FOR IMPROVING UNATTENDED

GROUND SENSOR OPERATOR PERFORMANCE

Pilette, S., Biggs, B., & Martinek, H. ARI Technical Paper 304, August

1978.

Pequirement:

To determine the value of unattended ground sensor operator training and job aids derived from an analysis if error -sources and the effect on operator performance of target activity level and numbers o

sensors used in a string.

Procedure:

Based on an analysis of operator errors made in a previous study,

a training program and two job aids were developed. To test the value

of the training and one of the job aids (measuring device), two 2-hour

scenarios were constructed for pre- and posttraining evaluation. Typical target patterns at two levels of target activity and three levels

of sensor string size were systematically varied within 30-minute segments. Authentic fixed-wing and helicopter activity, artillery shell bursts, and random noise were included to simulate operational, nontarget activations. The second job aid, a nomograph, was evaluated using

the pretest and posttest design. Two special tests reqairing only the

measurement and computations necessary for estimates of speed and enemy

number(s) were developed for this purpose.

Twenty school-trained Army enlisted men (UGS (unattended ground sensor) operators), were given test procedure training and a short refresher in UGS interpretation. Two 10-man groups were formed and each

was given different scenarios for the pretest; these were then switched

for the posttest. The training proqram--given between the pretest and

posttest--stressed individualized instruction including self-pacinq,

immediate feedback, expert assistance when needed, and guaranteed

student-mastery using criterion testing. One job aid, an UGS ruler

for accurately measuring the length of activation patterns, was part

of the training exercise. The other job aid (a nomograph), which had

been developed to simplify arithmetic calculations and decimal point

placement, was tested separately after the above trainina and posttest.

Findings:

The individualized training program resulted in significantly improved operator interpretation performance in target detection rights,

21

- - DfleFLUs

S,

'I
identification rights, target speed, and target quantity estimation.

Use of the nomograph significantly improved performance in identification rights, target speid estimation, target quantity estimation, and

reporting time. Student acceptance of the individualized training approach and of both job aids was high. Operator performance on 3- and

4-sensor strings was 77% detection completeness with virtually no false

alarms. Detection of targets was better during low target activity

than during high target activity. Use of three sensors in a string

resulted in the same operator performance as use of four sensors.

Utilization of Findings:

The lesson materials together with the individualized training approach should be used to provide review and on-the-job training to oper-'

ational field personnel and should also be integrated into the UGS

course at USAICS, Fort Huachuca, Ariz.

The nomograph should be included as standard issue with operational event recorders and should be taught at UGS course USAICS and

given to each graduate to take with him to his assigned unit. A creditcard size UIGS ruler should be issued to everyone involved in monitoring

UGS target activation.

To increase timeliness, the UGS operator should send forward a

detection report based on two sensors (or more, if doubt exists) with

a followup report giving target type, speed, and number.

A research study controlling on target difficulty and comparing 2 versus 3 versus 4-sensor strings should be done to provide data for

determining the most efficient string size.

During high-target-activity conditions, operator reports should

be considered as greater underestimates than during low-target-activity

periods.

I

f. . 22

4,

/t
APPENDIX B

THE VALUE OF SPECIAL TRAINING FOR THE INTERPRETATION OF UGS EMPLOYED

IN A GRID

Pilette, S., Biggs, B., & Martinek, H. ARI Technical Paper 304, August 1978.

Requirements:

1. To determine the value of specialized training for improving

operator target detection and direction and speed estimation performance when using unattended ground sensors (UGS) employed in a grid.

2. To provide estimates of operator performance in an operational

situation for target detection completeness and direction and speed

estimation accuracy for UGS employed in a grid.

Procedure:

Based on an error analysis of operator target detection data from

previous research, a self-paced training program was developed to reduce

the frequency of operator errors. To assess the value of the training

program, a pre-posttest design was used. Two 2-hour scenarios consisting of various numbers and compositions of convoys traveling cross

country were constructed from activation data collected at a field ex- ercise. Four target workload conditions (2.5, 4.0, 7.5, and 13.5 targets per 30-minute period) were systematically varied within each 2-hour

scenario. Each operator monitored all four workloads during both the

pretest and posttest- scenario3.

Two groups of eight trained operators of the Remote Sensor Platoon

of the 2nd Armored Division at Fort Hood participated in 3 days of training and performance testing. Prior to the pretest, each group was given

an orientation briefing, an introduction to the grid employment of UGS,

row patching technique training, and test procedure training. Each

operator then interpreted one of the two scenarios to determine his

baseline performance in target detection and direction and speed estimation accuracy. He next completed the specialized two-unit training

program. The first unit dealt with solitary targets (vehicle or vehicle

convoy traveling alone through the grid), and the second dealt with

target clusters (several targets in the grid area at the same time).

The training program was individualized (self-paced), and expert assistance was rewered when needed. Student mastery was ascertained by the

monitors checking practical and criterion exercise answers.

23 d I - -'-- - - - - -
Operators used three aids in the training and in the pouttest; they

used the UGS ruler and the speed table to make time measurements and to

facilitate arithmetic calculations, and used the protractor to make more

precise target direction estimates.

Findings;

The specialized training program with operator aids significantly

improved operator performance in target-detection completeness and target speed and direction estimation accuracy. Target-detection v=pleteness improved by 38%, while speed-estimation accuracy improved by 23%.

The accur=acy of target-direction estimation improved by 20%. The initially low false-alarm rate showed no significant change. After training, operators detected about 95% of the targets during low workload

conditions (5-8 targets per hour), 66% when the workload was doubled

(15 targets per hour), and 61% when the workload reached 27 targets per

hour.

Utilization of Findings:

The training materials and operator aids have been integrated into

UGS training at U.S. Army Intelligence Center and School (USAICS), Fort

Huachuca, Ariz.

The lesson materials should be given to personnel of all remote

sensor platoons periodically (every 6-12 months) to provide practice

and review.

Intelligence personnel should be trained to properly assess opera-or's reports based on the operator capability data in this report, so

ci they can effectively employ UGS and use the rerulting intelligence

ormation.

The grid employment of UGS can be used as an early warning and

target acquisition system. The grid can also provide appicoximate speed

estimates for use in a target acquisition system.

The atypical target paths used in this experiment prevent a good

est ,tion of the operator's ability to provide target direction dataHowever, results from the few available "normal" target paths indicate

that the operator can give useful direction information (average deviation from true direction was ± 26*). The usefulness of direction estimates should be ascertained using additional activation data in a second

exper_.ient.

24

/
APPENDIX C

VEHICLE IDENTIFICATION USING THE ACOUSTIC SENSOR: TRAINING, SENSING

CONCEPTS, AND BANDWIDTH

Martinek, H., Pilette, S., & Biggs, B. ARI Technical Paper 334, September

1978.

Requirement:

The experiments were designed to meet the following requirements:

(a) to develop and validate a training program for using the acoustic sensor to identify vehicles in convoyl (b) to provide estimates of operator

performance in identifying vehicles, using the acoustic sensor; and (c) to

investigate the effect of different sensing concepts and bandwidth modifications on the operator's ability to identify vehicles.

Procedure:

Following orientation and procedure training, 18 school-trained operators of unattended ground sensors (UGS) were tested on their ability to

identify military vehicles in convoys. Magnetic tape recordings simulating

use of the acoustic remote sensor in the field were used. The taped simuE lation was developed from recordings collected in the field during maneuvers

of armored and motorized infantry units. Incorporated in the test tapes

in a counterbalanced arrangement were two acoustic sensing concepts, "continuous" and "intermittent." In the continuous mode, operators hear the

entire convoy as it passes the microphone. In the intermittent mode, they hear each vehicle for a period of only 4 seconds, with 2 seconds of silence

between each vehicle. Seven vehicle types were involved--jeeps, gamma

goats, 21⁄2-ton trucks, 5-ton trucks, 10-ton trucks, armored personnel carriers, and tanks.

The operators then received vehicle recognition training which used immediate feedback, self-scoring, paired comparisons, and practice. After the

training, the operators were retested to measure its effects. An exploratory

study was then conducted to compare operator performance when different bandwidths were used--50-1500 cycles per second (cps), 50-2000 cps (now in use),

and 50-4000 cps.

Findings:

The training package developed increased operator vehicle identification performance by 44% to 16%, depending on the level of target detail

required.

An increase of 6% to 10% in vehicle identi2ication can be achieved by

using the intermittent type of sensor rather than the continuous. A saving

of 33% in battery life would also result. Either type of sensor has a greater

information potential than the present-day Audio Add-On Unit.

25

: ..... ...-. --.. -. .1 4.1_

' • .. . . .. . . .... . . . . . .. .. ..... • • • " .. . ' ' F • •, t.'= Nu' J•
An increase of 13% in vehicle identification can be achieved by using the top third of operators selected on their ability to interpret acoustic signals, as measured by the initial test in this exercise.

The 50-2000-cps bandwidth currently used by the Army for the remote

sensor was better than 50-1500 and as good as 50-4000 cps for vehicle identification purposes.

Utilization of Findings:

The self-administerable training tape should be used at the U.S. Army

Intelligence Center and School for UGS operator training and in field units

for periodic refresher training.

Depending on field requirements, the remote sensor platoon leader

should selectively assign operators on the basis of their capabilities.

Both the intermittent type and the continuous type of sensor should Le considered for use in the Remotely Monitored Battlefield Sensor System (REMBASS).

The bandwidth currently used by the Army for the acoustic remote sensor is adequate.

I

26

----- --- ----
APPENDIX D

OPTIMUM PATCHING TECHNIQUE FOR SEISMIC SENSORS EMPLOYED IN A GRID

Pilette, S., Biggs, B., Edwards, L. E., & Martinek, H. ARI Technical Paper

320, September 1978.

Requirement:

The requirementa are to develop ways in which seismic sensors employed in a grid array can be patched to an RO 376 readout device, and to identify

the preferred technique for field use. In addition, (a) requirements are

to determine if specialized training of unattended ground sensor (UGS)

operators is required for interpretation of activations of seismic sensors

employed in a grid, (b) to find whether, or to what extent, operator performance is affected by two densities of sensors in a grid employment, and

(c) to determine the interactive effects of sensor density, target activity,

and patching techniques on operator performance.

Procedure:

Four techniques for patching seismic sensors employed in a grid array

to the RO 376 Event Recorder were developed. Operator performance using

the techniques was compared under two sensor density levels--9 versus 24

sensors per square kilometer--and two levels of target activity--high and

low. The value of training specific to the use of the patching techniques

and associated job aids--target log, speed chart, and a specially designed ruler--was determined. Five 2-hour scenarios based on materials collected

in field exercises were used in assessing operator performance in detecting

vehicular targets under the experimental conditions described above. Fixedand rotary-wing aircraft activity, artillery shell bur6ts, and random noise

were included in the scenarios to help insure operational realism. Twentyfour school-trained UGS operators participated in testing under the experimental conditions. An additional eight operators serving as a control group

were not given the special training.

Findings%

Row patching was identified as the preferred technique; it resulted in

fewer false alarms, greater accuracy in estimating target speed, and more

efficient use of equipment. It was also preferred by more operators. The

patching technique training significantly enhanced target detection from

36% to 51%, but did not reduce the number of false alarms. Operators indicated that all the job aids were useful, but that the scale on the ruler

was of value only while they were gaining familiarity with the way the

sensors were deployed in the grid. The percentage of targets detected under

the low-target-activity condition was twice that detected under the hightarget-activity condition. Use of the 9-sensor grid resulted in the same

detection performance and half the number of false alarms as use of the 24-sensor grid.

27

4,-....

'I!
Utilization of Findings:

The row patching technique is preferred for field use with operators

trained in the patching technique. If the operators have not had such

training, and if operational conditions require a higher detection rate

in spite of a possible increase in false alarm rate, the column patching

technique is preferred.

For detetting vehicular activity, the 9-sensor grid (500-m spacing between seismic sensors or MINISIDS) is preferred to the 24-sensor grid (250-m

spacing). This preference exists in view of the similarity of results,

and considering cost and equipment availability and occasional reduction

in the number of false alarms.

If high target activity is observed, procedural changes should be made

(such as assigning additional operators or increasing the number of targets

estimated by intelligence analysts). Training in the use of the patching

technique and associated job aids should be incorporated in the UGS school

content at Fort Huachuca. Knowledge of the system's capability can be useful to intelligence officers in the selection and utilization of field personnel to enhance the reconnaissance resources of the Army.

An error analysis should be conducted and a training package should be

developed and validated, to increase the detection completeness to higher

levels, reduce speed calculation error, and reduce error in determining

target direction.

28
APPENDIX E

THE EFFECTS OF W)RK/REST, TARGET ACTIVITY, BACKGROUND NOISE# AND STRING

SIZE ON OPERATOR INTERPRETATION OF UNATTENDED GROUND SENSOR RECORDS

Martinek, H., Pilette, S., & Biggs, B. ARI Technical Paper 300, June 1978.

Requirement:

The experiment was designed to (1) investigate the relationships between unattended ground sensor (UGS) operator performance and various

system-dependent and target-dependent parameters including work/rest cycle,

number of sensors in a string, target activity level, and background noise;

and (2) to identify sources of operator er'ror that can be reduced through

new interpretation techniques, procedures, and training.

Procedure:

Two 8-hour zcenarios were compiled from a data bank of taped recordings of UGS activations during field tests by Modern Army Selected Systems

Test, Evaluation, and Review (MASSTER) at Fort Hood, Tex. The tests were

run under typical operational conditions using groups of personnel and

wheeled and tracked vehicles as targets. Roughly 80% of the sensors were

seismic, the remainder confirmatory. All sensors were deployed in typical

string configurations. The scenarios included counterbalanced variations

of strings of three different sizes (2, 3, and 4 sensors), high and low

target activity, and high and low "battle" noise. The high-noise condition

contained the same targets as the low but also included typical aircraft, artillery, and high background noise taped during the above field tests.

Four work/rest cycles were used, each of which involved a total of 8 hours

of work with work and rest periods of different durations: 2 hours work,

1 hour rest, 2 hour's work, 15 minutes rest; 4 hours work, 1 hour rest;

and 4 hours work, 15 minutes rest. Sixteen students at the U.S. Army

Intelligence Center and School (Ground Sensor Department) at Fort Huachuca,

Ariz., were given test procedure training and a 1-hour review consisting

of instruction and practice in interpretation of seismic sensor records.

They were also given a background questionnaire and a 30-minute pretest.

Pretest results were used as the basis for assigning operators to the experimental groups. Each group was assigned one of four work/rest cycles

the first test day and a different one the second test day. Subject

performance was scored against the known target activity for correct detections, wrong detections, correct identifications, elapsed time, and

confidence.

29
Findingst

For 8-hour monitoring shifts, 2-hour work shifts interspersed with

1-hour or 15-minute rest periods were better than 4-hour work shifts interspersed with 1-hour or 15-minute rest periods. For important short-term

monitoring requirements, operator performance was satisfactory for the

first 4-hour shift, but deteriorated during the senond 4-hour shift.

Operator performance was equal usinq 2-, 3-, or 4-sensor strings in

the low-target-activity condition. However, in the high-target-activity condition, use of 3- or 4-sensor strings resulted in more correct target

detections than use of 2-sensor strings.

Operator performance during the high "battlefield" noise was equal

to that during low noise.

During high-target-activity conditions, operators detected more

targets than during low-target-activity conditions. They detected a

higher percentage of targets during the low condition, however.

Sources of operator errors included the use of 2-sensor strings fir

the detection of targets in a high-target-activity condition, endpoint

determination, measurement, arithmetic calculations, and use of confirming sensors.

Utilization of Findings:

The major findings regarding work/rest cycles can be used by all

operational UGS commands and school units for assignment to duty cycles.

Considering only operator performance and assuming high-sensor reliability,

2-sensor strings, rather than 3- or 4-sensor strings, can be used to reduce costs if only low-target activity is expected. If high activity is

expected, 3-sensor strings can be used rather than 4 with no operator

performance decrement., Operator problem areas identified should be used

as the basis for a new trdining program. A UGS ruler should be developed for measuring sensor activation times directly on the display; a job aid

(such as a nomograph) shculd be developed to simplify arithmetic computations and decimal point placement. A programed text for other operator

errors should be developed and validated.

4

I' i ,;J

30

M
APPENDIX F

THE EFFECT OF SIGNAL/NOISE RATIO AND BANDWIDTH ON VEHICLE

IDENTIFICATION, USING THE ACOUSTIC SENSOR

Martinek, H., Pilette, S., & Biggs, B. ARI Technical Paper 377, June 1979.

Requirement:

To determine the effect of variations of signal-to-noise (S/N)

ratio on the ability of remotely monitored sensor (REMS), formerly

called unatteiided ground sensor (UGS), operators to identify vehicles

in convoy.

To determine the effect of an increase in bandwidth on the REMS

operator's ability to identify vehicles.

P.rocedure:

Three experiments were conducted: S/N Ratio, Individual Target,

and Bandwidth. In the S/N Ratio Experiment, 20 operators received

special training, which covered four levels (+6 decibels (dB), +12 dB,

+18 dB, and +24 dB) of SIN ratio. After training, magnetic tape recordings simulating REMS outputs were used to determine the operators'

ability to identify military vehicles in convoys. Seven vehicle types were present in the convoys: jeeps, gamma ýioats, 2-1/2-ton trucks,

5-ton trucks, 10-ton trucks, armored personnel carriers, and tanks.

The test tapes were made from recordings Collected in field maneuvers

of armored and motorized infantry units. A 4 x 4 Graeco-Latin Square

design was used to counterbalance the variables of SIN ratios, opera- J

tor groups, order of presentation, and convoys.

The Individual Target Experiment presented operators with indi-

%idual targets using only vehicle sounds t-iat had similar signal

strength (t2 (35 difference). The operators interpreted sournds of

each vehicle type at each of the four S/N ratios in a randomized

sequence.

In the Bandwidth Experitent, the operators were given convoy

sound recognition training using both 50-2000 hertz (Hz) and 50-4500

Hz bandwidths. They were then tested on their ability to identify

military vehicles in convoys at both bandwidths using a 2 x 2 x 4

modified Latin Square design. The variables of this design are bandwidth, order of presentation, and operator groups.

31

, .. . |
Findings:

Operator identification completeness tends to decline as the S/N

ratio decreases approximately 1% per 1.5 dB of S/N ratio.

Operator identification completeness of light- and medium-wheeled

vehicles tends to increase as the S/N ratio decreases. This unusual

relationship results from a tendency to report (i.e., to guess) these

vehicle types more frequently as noise increases. Hrwever, accuracy

in reporting light- and medium-wheeled vehicles tends to decrease as

noise increases.

Operator identification completeness of 5-ton trucks, 10-ton

trucks, and tracked vehicles declines as the S/N ratio decreases.

The 50-4500 Hz bandwidth provides no advantage to identification

over the current-ly used 50-2000 Hz bandwidth.

Utilization of Findings:

Field commanders, training personnel, and operators should be

made aware of the tendency of operators under high noise conditions

(such as many relays) to identify any vehicle sounds as light- and

medium-wheeled vehicles. Specific training to counteract this effect

should be given both in the school and on the job.

From the standpoint of signal interpretability, there is no requirement for new acoustic sensors to use a greater bandwidth than

the current one of 50-2000 Hz.

For purposes of developing doctrine for the employment of relays

and designing new acoustic sensors, this rule of thumb can be used:

A 1% decrease in operator performance will occur for every 1.5 dB

loss in S/N ratio.

Use of automatic gains control should be limited to allow signal

loudness variations between vehicle types.

32

- .4
APPENDIX G

COMPARISON OF THREE DISPLAY DEVICES FOR UNATTENDED GROUND SENSORS

Martinek, H., Hilligoss, R. E., & Lavicka, F. ARI Technical Paper 299,

August 1978.

Requirement:

typical operating conditions of three methods of displaying activations

of seismic unattended ground sensors: us.e of the operational RO 376

event recorder, use of a situation map display, andluse of time compression

with the situation map display.

Procedure:

Three tape recordings lasting 2 hours each of the activations of

unattended ground sensors were compiled from the data bank of recordings

of unattended ground sensor activations taken during field tests under

simulated operational conditions. Typical patterns at two levels of target activity were selected to include both personnel and vehicle targets.

To provide realistic simulation, recorded activations of aircraft, artillery, and background noise likely to affect the interpretation of displays were included.

Twelve Naval personnel trained and experienced in the use of the

RO 376 were given 4 hours training in the use of the situation map display and the display used with time compression. Each subject then interpreted each of the three displays using a different set of recorded

activations each time, in counterbalanced order, and filled out a stan-j

dardized report form. The reports were scored for number of correct detections and number of false alarms in comparison with the known target

activity observed in the Modern Army Selected Systems Test, Evaluation,

and Review (KASSTER) tests at Fort Hood, Tex.

Findings:

Use of the operational RO 376 resulted in higher accuracy and

greater completeness of reports than did use of the other displays.

No differences were found between the situation map display and the

situation map display used with time compression.

Target activity level, order effects, and composition of the taped

activations affected operator performance.

33
Utilization of Findings:

The experiment reported here was an early effort in a series to

improve the interpretation of UGS activations. Because of its superiority, the RO 376 event recorder should be used instead of the situation

map display in interpreting activations of sensors deployed in strings.

An additional study that would deploy sensors using the area intrusion concept of sensor deployment ("grid" or "gated array") could

be an additional, useful basis for evaluation.

In view of the effect on performance of target activity level,

order effects, and composition of taped activations, these conditions

must be controlled in further evaluations of ground sensors in which

operators interpret and report intelligence information.

34

Ii

I

34
APPENDIX H

COMPARISON OF FOUR UNATTENDED GROUND SENSOR DISPLAYS

Edwards, L. E., Rochford, L. S., & Shvern, U. ARI Technical Paper 281,

April 1977.

Requirement:

To compare under operational conditions four different types of unattended ground sensor (UGS) displays, the RO-376 X-T plotter and three

variations of situation map display, in terms of their effect on monitor

performance.

Procedure:

The RO-376 X-T and the three situation map displays--a blinking light

to indicate an activation, a light that increases in intensity with each

additional activation, and the latter light plus the capability of reviewing previous activations in compressed time--were compared in terms of

operator performance. Four 2-hour UGS scenarios were com~piled fromt recorded

field tests run at Fort Bragg, North Carolina, using typical personnel and

vehicle target patterns, noise sources, and two levels of target activity.

The recordings were played back to activate the displays during

experimentation.

Sixteen Naval personnel (8 relatively experienced with UGS and 8 inexperienced) were given training on the displays. Each operator then monitored each display in turn for 2 hours, reporting target information as he

would operationally, except that prepared report forms were used. The reports were compared to known ground truth and were scored on total detections, false alarms, detection accuracy, and direction of target movement.

Findings:

Operator performance was unaffected by type of display used. Operators

were able to detect a higher percentage of targets during periods of low

target activity than during periods of high target activity. However, accuracy of detection was gieater during high target activity. Levels of experience, time effects, and scenarios did not have a significant effect on

performance.

Utilization of Findings:

The performance data from this effort provide the best available estimates of expected operator performance until more extensive field data are

available. Typically, in the selection of type of display for a given objective, operator performance with available displays should be a majorA

35

- -- -- - .- ~A
criterion 

shown not 

used. 

to affect 

In the present experiment# however, type Of display was

ferentiating among displays. 

operator performance and thus offers no basis for difcan provide 

Other measures, such as cost and availability, added basis for selection of displays.

I

36
APPENDIX I

THE EFFECT OF WORKLOAD ON PERPOP4MANCE OF OPERATORS MOWITORING

UNATTENDED GROUND SENSORS

Edwards, L. E., Pilette, S., Biggs, B., & Martinek, H. ARI Technical Paper

321, September 1978.

Requirement:

To investigate the effect of workload on operator performance as

defined by target activity level and number of unattended ground sensors (UGS) used.

To determine operators' target-detection ability, false-alarm

rate, and direction and speed estimation accuracy to 'help establish

system capability.

Procedure:

Following an orientation and training session, experienced UGS

operators monitored, in sequence, each of three event recorder displays

showing activations of UGS used in grids. The operators monitored 27

sensors (3 grids) on one display, 54 sensors (6 grids) on the second,

and 108 sensors (12 grids) on the third. Each grid was composed of

nine minisids, spaced 500 m apart to form a 1,000 n2 field. Operators

encountered periods of high and low target activity that were of equal

time duration. Operators reported each target they detected and estimated speed and direction of movement.

Findings:

The number of sensors monitored and the target activity level

significantly affects UGS operator performance. The operators' ability

to detect targets decreased as either activity level or number of sensors increased. Operators' ability to estimate target direction also

decreased as activity level increased. Although target speed was underestimated, no significant differences were found between any of the experimental conditions for this variable. The false-alarm rate was low

(one per 3 hours).

37

4I7

...-- -
Utilization of Findings:

Careful judgment should be exercised in assigning workloads to

UGS operators. Operators without special training or experience should

not monitor more than 60 sensors, and then only if target activity is

low. If operators are required to monitor more than 60 sensors or if

target activity is high, intelligence estimates of target activity based

on UGS operator reports should be adjusted upward.

The grid deployment of UGS is a valid method for surveillance of

large areas to detect vehicular movement. Operators' target-detection

performance was go-d even though they had received no training or experience in monitoring UGS employed in grids. The false-alarm rate

(one per 3 hours) and the 85% detection rate for the 27-sensor, lowtarget-activity condition demonstrates the initial capability of the

use of UGS employed in a grid. Although the true speed of vehicles

passing through the grid was underestimated for all conditions, the

"cross-country" speed estimate (used for predicting time of arrival)

is as accurate as that made for sensors deployed in the more typical

string configuration along roads or trails.

Special training should be instituted for target detection under

high-workload conditions and for the estimation of the target's direction of travel. Direction estimation was poor (*400 on the average);

but in view of the atypical target paths used in this research, the

above value should not be generalized to the usual operational

situation.

(

38

i .
DISTRIBUTIGN

I US ARMY CINCPAC bJPPORT bOUUP PHRSON4LL 0IVISION

I H•OA AITNI PMAC

I TAG/TAOCEN ATINI UAAG-EU

I HQ* TCATA ATTN: ATCAT°UP-w

4 HQ0A RESEARCH AND STUDIES OFC

I MILITARY OCCUPATI3NAL UEVLLOPMENT DIV OAPC-MSP-0, RM A52C HOFFMAN BLOG 1

4 OASO (MRA AND L)

I HO')A ATTN: UAMU-RUR

I MG ICATA TECHNICAL LIbRAHY

I HQDA CHIEF* HUMA4 RESOURCES DEVLLOPMENT DIV

I USRAUCO, STC

I HOW. ATTN: UAMI-TST

I USA AVIATION SYSTEMS COMO ATTN: DMSAV-ZI)R

I USA -ORAUCOM ATTN: AMSEL-PA-RH

I USA ARRAUCOM ATTN: ATFE-LO-AC

I HEADQUARTERS* US 4ARINE CORPS AITN: CODE MPI-28

e US ARMY EUROPE AND SEVENTH ARMY

IIST INFANTRY DIVISION AND IT. RILEY ATTN% AAF2N-DPT-T

I USA INTELLIGENCE. AND SECURITY COMMAND ATTN$ IAOPS-TNG

2? HO TRAInOC TECHNICAL LIORARY

I NAVAL TRAINING ýQJIPMENT GLN ATITN TECHNICAL LIBRARY

I MIuI.ARY OCCUPATIONAL UEVtLOPMENI DIV ATTNI UAPCoMSP-s, RM 852C, HOFFMAN BLOG I

I MILITARY OCCUPATIONAL UEVELOPMENT DIV ATTNt OAPC-MSP-Dq Rm 852C9 HOFFMAN BLDG I

I MILITARY OCCUPATIONAL UEVLLOPMENI DIV ATTNI DAPC-MSP-T, RM 852C9 HOFFMAN BLDG I

I USAFACFA CHIEF, OkOANIZATIONAL EFFECTIVENESS BRANCH

1 gT4 INFANTRY OIVISION

I HWI)A TANK FORCES MANAGEMLNT OFL

I NAVAL AIR SYSTEM COMM-wNU

I I)CSOPS (MIST 4) ATTN% UAMU-RQI * UAMO-ROC

1 12l30 UsARCOM HESERVE CENTLR

I FT. BENJAMIN HARRISON, IN 4621b

I DIRECTORATE OF AR4UR AVIAIION ATTN: ATSH-AAU

1 AVIATION DIVISION ATTN: ATZK-I)PT-AVN WUHIFS)

I USA FORCES COMMAND AFIN - OtPjtjY C OF S FOR INTELLIGENCE

I USA FORCES COMMAND AFOP - DEPUTY CHIEF OF STAFF FOR OPERATIONS

I US ARMY AIR UEFFNSE

I DIRECTORATE OF TRAIN1Nk AITNI ATZQ-T

I DIECTORATE OF C04BAT UEVYLOPMENIS AITN: ATZA-D

I HQI)ARCOM MAHINF. CORPS LIAISON UFC

I DEPARTMENT OI THE ARMY US ARMY INTELLIGENCE + SECURITY COMMAND

I US ARMY AGENCY FOR AVIATION SAFLTY ATTN% LIBRARIAN* BLDG 4905

I USA MISILE MA1EKILL HLAUINESS COMMAND ATTNI ORSMI-NTN

1 ARTADS AITN; UWCPM-IUS-TU

1 USA FORCES COMMAND

1 PM TRAoE /

I US MILITARY DISTRICT OF WA$HINGIUN OFC OF EQUAL OPPORTUNITY

I NAVAL CIVILIAN PERSONNEL COMO SOUTHERN FLO DIV

2U ART LIAISON OFFICE

1 7TH ARMY TRAINING CENTER

I HO USAREUR ATTtw: OCSOP5

I HQ0A9 OCS STUUY OFFICE

I U)ACDEC ATINt ATEC-LX-E HUMAN IACTORS

I USAFAbOS/TAC SENlOR ARMY ADVISOR

I INTER-oNIV SEMINAR ON ARMEU FORCES 4 SOC

I USA ELECTRONIC PROVING UROUNU ATTNI SlEEP-MT-S

I OASA (RUA) DEPUTy FOR SCILNCE AND TECHNOLOGY I OFC OF NAVAL RLSLýRCHI

I AFHRL/L RT

I AFHRL/I RL

I AIR FORCE HUMAN RESOURCeS LAB ATITN: AFHRL/TSR

I FE0ERAi. AVIATION AI)MINISTRATION CtNTRAL REGION LIBRARY, ACE-bb

39

-- . ... .
1 6570 AMRL/RB

1 6570 AMRL/HE

I NAVAL PERSONNEL H AND U CINTER COMMAND AND SUPPORT SYSTEMS

I NAVY PFRSONNEL R AND 0 CENTER /

1 NAVY PFRSONNEL R AND D CNNTER uIRECTOH OF PROGRAMS I NAVY PFRSONNEL R AND 0 CENTER/

I US ARMY AVN ENbINEERING FLIGHT ACTIVIIY ATTN8 SAVTE-TD

SoFc OF NAVAL RESEARCH PERSONNEL AND TRAINING RESEARCH PROGRAMS

I OFC OF NAVAL RESEARCH ASSTo UIRHCTOR PENS * TRAINING RSCH PHOGS

I OFC OF NAVAL RESEARCH PROJECT UPFICERM. ENVIRONMENTAL PHYSIOLOGY

I NAVAL AEROSPACE MEDICAL RSCH LAD ATT4.: (CODE L511

I RUREAU OF NAVAL PERSONNEL SCIENTIFIC ADVISOR (PERS-OR)

I NAVAL AEROSPACE MEDICAL HSCH LAO AEROSPACE PSYCHOLOGY DEPARTMENT I USA TRADOC SYSTrMS ANALYSIS ACTIVITY ATTN$ ATAA-TCA

1 HEADUUANrERSv COAST UUAHU CHIEFs PSYCHOLOGICAL FSCH Rp

I USA RESARCH AND) TECHNOLObY LAM ATTN: DAVDL-AS

I USA ENGINEER TOP03HAPHIC LABS ATTN: EIL-(oSL

I USA ENGINEER TOPS0RAPHIC LABS ATTN: STINFO CENTER

I USI ENGINEER TUPO6RAPHIC LABS ATTN: LTL-TD-S

I USA MORILITY EWUIPMENT R AND 0 COMO ATTNS ODMUE-TO

I NIGHT VISION LAH ATTN: UHSEL-NV-SUU

I USA TRAINING BUAHD

I USA HUMAN ENGINEERING LAU

I US HELIUSAAVNC /

i USA MATERIEL SYSTEMS ANALYSIS ACTIVITY ATTN: DRXSY-M.

I USA RESEARCH OFC /

I NAFEC HUMAN ENMI4LERING bRANCH

IATTELI.E-COLUMdU5 LA809ATUNIES IACTICAL TECHNICAL OFC

I USA ARCTIC TESI CEN ATTN: AMSTL-PL-TS

I USA ARCTIC TEST CEN ATTN& STEAL-PL-MI

I USA CONCEPTS ANALYSIS AbCY ATTN: MOCA-W1

I USA CONCEPTS ANALYSIS AbCY ATTN: MOCA-JF

I HU WHAIR DIV UI- NLUROPSYI.HIATPY

I USACACflA ATTN7 ATLLCA-CI-u I USACACI)A ATTN: ATZLCA-CI-M

I USACACOA ATTN: ATZLCA-CI-A

I USACACOA AiTN3 ATZLCA-CA

I USA ELFCTRONIC *ARFANE LAd CHItFl INItLt.IGENCE MATER DEVIL L SUPP OFF i ULSA RSCH UEVEL * SIANDAULLA P* U.*K.

I AFFDL/FGR (CDIC)

I USA NATICK RESLARCH ANL U.VELOPNN C)MMANU CHItF, dFHAV SCIENCES UIV, FOOD SCI LAR

I OASDO F AND E (F AND LS) MILIlAHY ASST FOR TNG * PERS TECHNOL

I TRAJANA ATTN: SAJS-UR

I HQUA /

I NAVAL AIR SYSTEMS COMMAND ATTN; AIR-5313

I ECO• ATTN: AMbEL-CT-U

I USACUECEC TECHNICAL LIHRARY I USAARL LIBRARY -

I SEVILLF RESEARCH CORPOHATIUN

I USA THADUC SYSTEMS ANALYSI/ ACTIVI(V ATTNI ATAA-SL (TFCH LIHRARY) j

I UNIFORMED SEHVICES UNIT UP THlE HLALYH SCI DPARIMENT OF PSYCHIATRY

I USA COMPUTER SYSTEMS COMMAND ATTN. CJMMANU TECHNICAL tIIRAHY

I HU4AN QESUURCES RSCH OUb (HUMRRU) 'I

I HUNIRRO WESTERN LIHRARY

I FUSTIS UIHECTUHAIE, USAAMNUL TI4.HNILAL LIHRARY

I RAND CnRPORATIUN / I RAWD CoRPURATIUN ATTN: LIUMANy U

I FEERAi AVIATION AUMINISIRATION ATTN: CAMI LIBRARY ACC-44•[1 ( I NAFEC I IHARY, ANA-64

I (;,R)NINGER LIHRARY AITN: ATZFi-WSL BLLb 1313

I CE'4TER FUN NAVAL ANALYSIS

4o

, . ,u s,
I NAVAL HEALTH RSCH CEN LIdWARY

I NAVAL FLECTRONICS LAd ATTN: RF6~.ARCH LIBRARY

I NAVAL PERSONNEL H AND U CLN LIBRARAY ATTN: CODE 9201L

I AIR FORCE HUMAN RESOURCE.S LAW ATTNt AFHkL1VOT

I HO. FT. HUACHUL.A AtTN: TE.CH REF UIv

I USA ACADEMY OF HEALTH SCIENCES 5TIMSON LIBRARY (DOCUMENTS)

I SCHOOL OF SYSTLMS ANU LOGISTICS ATTN& AFIT/LSCH

I USAMERflC TECHNICAL LIBRARY

I DEPARTMENT OF ITHE NAVY IRAININba ANALYSIS AND EVALUATION OP~

I NATIONAL CENTER FOR HEALTH STATIlTICS /

1 USP4A nEPT 01' bFHAVIORAL hC4 ANu LEADERSHIP

1 US NAVY CNET SUPPORT HN.SLARCH LIbRARY

1OLo DOMINION UNIVERSITY PE.RFORMANCE ASSFS5MENT LABORATORY

I USA COMMAND AND bENERAL STAFF COLLEGE AtTN; LIBRARY

I USA IRANSPORIA1iONd SCHOOL USA INANSP TECH INFO AND RSCH CFN

I NASA HOI NMROC PROGRAM MASIAGER FOR HUMAN Fk.RFJHMANCE

I NAVAL MEDICAL R A~40 0 COMMAND 4'*0

I USA AUMINCEN ItCHNICAL RESEARCH bRANCH L113RARY

e HQ0A IISA MED HSC-I AND ULVLL COMMAND

1 USA FIFLO ARTY Hu

I NAT CLFAHINGHOUSE FOR MENIAL HEALTH 1'dFO PARKLAWN BLDG

1 U oF TFXAS CEN F'OR COMMUNICATION RSCri

I INSTITUITE FOR IJEFENSE ANALYSES

I USA TRAINING SUPPORT CENTER~ UEVtL SYSTEmS TNU + OEVICFS DIRECTONATE

I AFHRL TECHNOLOGPY OFC (H)

I PURDUE UNIV DLPT OF PSYCMOLOGICAL SCIE.NCES

1 05A MOHILITY EWI,1PMENT w AND 0 LUMMANL) ATTNt URDME-1rG

IHQ. USA MDW ATTN; ANPE-vE

It D A US ARMY RETHAI'Jl1MG UtSb RESEARCH + EVALUATION DIRJ

IUSAF SCHOOL OF iEtROSPACI: MEDICINE AERUMEDICAL L.11RARY (TbK'A) I US MILITARY ACA0E4Y LIbRARY

1 USA INTELLIGENCE CI.N ANU SC..H U~t-P OF GROUND SENSORS

I MA41NE CORPS INSTITUTE

I NAVAL SAFETY CENTER/

I USAAVNr AND FT. kJC~tR AITNI AIZWUES

I US ARMY AWN TNU LIbRARY ATTN: C.HILF LIBRARIAN

I USAAVNC ATTNS AIlU-U

I US MILITARY ACAIIL4IY O1'C 01 MIt IIARY LLAOF1RSHIP

I US MILTTAHY ACAI)E4,Y DIRECTOR O1' INSTITUTIONAL FISCH

I USA AIP UEFENSL SCHO)L ATTN: Ai4A-Ct)-MS

A USAAUS-LI3RAIYiUOCUMh'NiS

I USA AIQ UEFENSt HDAkL) AIIN? FILLS REPOSITORY

I USA INFANTRY BO,%HD AT~vj- ATZO-Id-TS-ri

I LISA INTELLIGENLI, CEN \NO SUN EUUCATIONAL ADVISOR

I USA O~nNANCE CLN AND i~CH ATTN: ATSL-TEm-C

I USA ARMOR SCHOOL ATIN: ATbSb-UT-IP

I USA ARAOR CENTEN DIHIECTUHATL Ol, LUMHAT iDEVELOPMLNtS I NAVAL POSTGRADUATE SCH AIIN: D)UDLEY KNOX LWbRARY (COOF 14?4)

I UISA TRANSPORIATJU%4 SCHOOL DEPUTY ASSI. COMMANDANT EDIICA. TtCHNOLUt,Y

1 USA SIGNAL SCHOOL ANU FT. GORUON ATT4: ATZH-ET

I CHE ARMOR FNVA SCHOOL. ')C1UNADTG'SA EVAL 8RANCH, UIRECTJRATE Of' INSTRUCTION

I USASIbc, STAFF AND FACULTY DEV AND IN~i DlV

I HO ATC/'XFTo THAINING SYSIkMh [t.VELOP4LNT

I US ARMY ARMOR SCHOOL DIRECTORATE OF TRAININGý

I USA QUARTERMAST1'R SCHOOL DIRECIUHATE OF TRAININIG DEVELOPMENTS

I US COAST GUARD ACADEMY AuINI CADET CJUNSELOR (DICK SLTMArK,

I USA TRANSPORTA1II0J SCHOOL DIkp(.1OR OF TRAININ6

I USA INFANTRY SUtiOOL LIbRAHY/

41
I SARMY INFANTRY SCHOOL AT TNI AT~m-cj

I USA INFANTRY SCHUOL AlTNt AISi4.UOI

I USA INFANTRY SLtiOOL AITNI ATSm-t.V

I USA MligITAIýY PULICL SCt100L/TRAININb CENTFR ATTN: ATIN-PTS

I USa M ilTARY PULICk SCHOOI~THAtNINb CENTE.R 11H. C014BAT OEVLLOP1MENt

I USr% MhIrIARY POLICE SCHUULITHAIN4INb CENTFH UIR* TRAININU3 DELVLOPMENT

I US,% Mu IIAPY PUILICL SCHoOL/TkAINAN4; CENTFH ATTNI ATZN-ACt

I LJS LN'rdI1UTF Of AUMINISIHAT;ON ATTNI RfsIDLNT TRAINING MANAOk#4ENT

I USA ýIFLO AR71LLLRY SCHOOL MoRHiS SwETT LIBRARY

I USA INSTITUTE Oý AUMINISTHATION A(;Ai)EmIC LIBRARY

I USA WAil COLLEGL ATTNI LIdWA*4Y

I USA LNGINI.FH SCHOOL LIMNHAY ANU LLAW4ING, RLSOURCES CENTER

I USA ARMOR SCHOOL (USA~RMS) AiTNt LIBRARY

I OH~iANIIATIONAL hýFECTIVLNk6S 1Nt2 CLN * SCH- AITN9 LIBRARIAN

1 US ARMY INTELLtiEJCE CL~iTLH + SCHOOL ATTN: ATSI1TU

I US ARMY INIELLIiibNCE CENktH * SCHIOOL ATTN% ATSI-RM-M

1 US ARMY IN7ELLhiE'CE CLNILH + SCMOOL ATTNI ATSI-TOýLU

I US ARMY INTELLhINCE. CtNrLW * SLýMUOL ArINt ATSI-CU-CS-C

I US ARMY INTELLhIL'JCE CENNit + SCHOOL ATNIN ATSIODT-SF-IM

4 RRITISH LMHASSY -314TISH ULFL.Nck STAFF

2 CA'JAUIAN JOINT STAFF

I C~i-S (w) LIHRAHY

I FRENCH MILITARY ATTACHL

I AUSTRIAN EMHASbY MILITARY ANU AIR AITACHh.

A CANAUIAN UEFENCý LIAISON STAFI ATTN: COUNSELLORt UEFENCE P ANt) 1)

ROYAL NETHFRLANiUS EMbASSY MILITARY ATTACHE

I CA'JAUiIa' FORCE5 h-ASE CORNWALLIS ATTN$ PFRSONNEL SELECTION

e CANADIAN FORCES PENSONNEL APPL HSCH UN4IT

II AI4AY PFRSONNEL NkSEAHCH t!STAbLISI1MLNT

IAWAY PFRSONNEL RESEAHCH t.SAdLISIIMLNl AW SCIENTIFIC COORDINATION OIFICE FI NErHERI.ANUS EM8PSSY Oi-?ICL W1 IML AIR ATTACHE

6 LIHRARY OF CON(.?RSS EALAANGk. ANU bIFT DIV

I DEFENSF TLCHNI(.AL INFORMATION (3-1 ATINI o1c-TC

153 L14RANY OF CON(b;k,-M UNIT UOCUiL.NTS E PEDITIN0 PROJECT

1 EUTTOR. H AND U HA(JAZINL 'AN TN:UCDE-LN

I US GOVERNMENT P,41TINO UFi. LlHkAkYq PUbLIC DOCUMENTS DEPARTM4ENT

1 US GUVERNMFNY P14141ING Of(. LIONARY AW4 STATUTORY, L1IR DIV (SLL)

I THE ARM-Y LIHRAHY

ihi