~~~

r c

~

r

~

Research Repsrt 1206

LEVELc

BT33 TRAINING

~~

EFFECTiVENESS ANALYSIS

a

~

-br ~~

. Ij ~

f

~~~L Q 0. L Finley and H. C. Strasel ~ 4 W ~~1~~~ l~

!

U

API FIELD UNIT AT FORT BENNING, GEORGIA

C )

w

U. S. Army

Research Institu te for the Behavioral and Social Sciences

AprIl 1919

Aprovid for pubUc r&.ssi; distribution NnH.5d. 

~~, U 3 1)
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of St aff for Personnel

WILLI AM L. HAUSER t

JOSEPH ZFJDNER Colonel, US Army

Technical Director (

~

oinmander

I

i

NOTicES 7,

DIST~ISUTION: Primary distribution of this report hst bssn made by ARI. FilMs address eto~ ondenre

ocncsrnlng distribution of reports to: U. S. Army Nissoicli InstituqI for th. Ithosloisi and Social $cisncss.

ATTh: PERI.P~ 5001 Elianhostor ~~snus. Alaxandria, Virginia 22333.

FINAL 0l~~øIITI~~5: This report ipoy be dswsysd vdtsn It 1 no 10.151 maded. Please do not rstorn it to

the U. S. Anny Aans..ah Iresisuw for tbe IahsrL..l and Social $cianma.

~~~~~ The findings in di. report are not to be oons.nasd an an official Dup.ronsflI of tise Army position.

unION an dN n.Ssd by other sutherland documents.

1
______ _____________________________ ________

Unclassified

*E~~j

~~ TY CL.A5$IPICAIlON 0? THIS PAGE (

~~

.. Sm. &,S.~~~

REPORT DOCUMENTATiON PAGE ______________________

kf~~SOaT NUMSJR a. .ovt ACCESSION NO ). RECIPIEIIT $ CATALOG NUMSER

j~~~earch fiei~~~~~206 ____________________

,,

,.. .— 4. TITLE (d S.add .) S. TYPE OF REPORT S PEmOD COVERED

.

~T33 TMINtNG. EFFECTIV NESS_~ UlLYSIS . L- -- • 1. PERFORMING ORG. REPORT NUMSER

THOR(.) S. CONTRACT OR GRANT NUMOE*’W)

D. L./Finley ~~~ H. C4Strase~7

0. PERFORMING ORGANIZATION NAME AND ADDRESS ‘SQ. PROGRAM I.EMINT. PROJECT. TASK

Army Research Institute for the Behavioral an (J AREA S WORK UNIT wUq~

oENs

Social Sciences I 2Q76373lA77~., ~~ 7

It. 

5001 

CONTROLLING 

Eisenhower 

OFFICE NAME 

Avenue

AND 

, 

ADDRESS 

Alexandria, VA 2 33 

IL ___

63 

_____

4

_

/34 

__ 9773 U—

U.S. Army Field Artillery School A~r •79 -

Fort Sill, 0I( 73503 • ,.u .t.n OF PAGES

86

14. ISONIT N AGENCY NAME & ADDRSU(SI ~~tgei m 5 ~~~~~~~~~~~ OEflc ~

) IS. SECURiTY CLASS. (.1 OS. ,ipo,t )

-- / L/ ~~ ~~~~~~ 

Unclassified

15& g~~~kASs.FIcATIoM/ooWNORAoING

IS. OISTRISUTION STATEMEN T (.9 OS. RSpOI )

Approved for public releases distrthution unlimited. 1

HI

17. D~STRtSUTION STATEMENT (of 1. .bm,.cf .ir .r.d Sn Bt•ck 20, If ~~~~~~~ from R.po.f)

55. SUPPLEMENTARY NOTES

IC. KEY WONOS (ComiSnu. m~ ,rnv . .Sd. It n.os.omy m.d Id.ntIl~’ by blocS m &or)

Training effectiveness analysis Forward Observer training

Training devices Human engineering evaluation

Simulators Artillery train ing N Performance evaluation

20. A~~

STfl .jCT (Coml~~

i.. .. ,.omoc .I

~~ It A.oc.s.y mid SdmifSS~’ 

by block O~~~SOI)

BT33 is a Swedish-built Forward Observer (FO ) trainer for institutional use . This training effectiveness analysis of the BT33 was initiated

to validate the concept that FO training devices could yield training and

cost saving s benefits . It was a part of and contributed to the cost and

training effectiveness analysis being conducted on a P0 trainer under devel- _~ }7

opseent in the United States for the unit training environ ment . ~~~~~~ _—

(continued )

00 I j

~ $fl 1473 E0~

Y1ON OP S ~oy S$ IS 0000LITI unclas sified

SECURITY CLANOFICATIOW OP TileS PASt (do.. Dm. gms,.~~ ‘-4

---I -. ______________ ___________________
F” ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ TI —

~~~~~ ~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unclassified - 

SCUNTY CI.A$0.FICAflON OP TNI$ PAIS4~~ ~~~~

20 (continued)

~~Th. results of the study deter mined that the P0 training device coQcept

is valid. All the tested training programs employing the BT33 resulted in

training that appeared overall to be as good as or better than the current

training program.

Of general research interest is the methodology developed to acquire

student performance data on several indice s and condition3 and to evaluate

and integrate these data into relative training effectiveness indices.

• 

L

r

ACCES3~

-’~ f

~

.

• •- p4 rp~ ‘~~~;

~~ ~

.

~~‘JCfI

• DDC B f So~:io~ A ’ J ~ ~

4C~D

J

~

J$ ~~~~~~~~~~~~~

BY

C~ES

~

I1~

11 

flI~

:.

I 

~ 

Unclassified

SECURITY CLAUIPICATION OP tHIS P501(0511. Sm. ~~~~~~~

______ ~~~~~~
______ • ..

~~ -— --~~~~~

-. 

- i• _ ,

~

--.,-,

_ -

~ 

~~~~~~~~~~~~~ ~~~~~ ~~~~~~~~~~~~~~~~~~~~~ ~~~~~

Research Report 1206

BT33 TRAINING

EFFECTIVENESS ANALYSIS

D. L. Finley and H. C. Strasel

Submitted by:

H. C. Strasel, Chief

ARI FIELD UNIT AT FORT BENNING, GEORGIA

Appr oved By:

A. H. Birnbaum , Acting Director

ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

Joseph Zeidner, TECHNICAL DIRECTOR

U.S. Army Research Institute for

the Behavioral and Social Sciences

U.S. ARMY RESEARCH INSTITUTE FOR ThE BE~

1AVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

April 1979

Army Project Number Systems Dsvslopment and

20763731 A773 Training Research

~~~iwed for pu~ ie r

~..ss; muelbnlon unlimited.

L

~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
__ 

~~~~~~~~~~~~ I TTI

~~

1

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon completion of a major phase of the task , formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

p 

V

I 

J ’ ~~~

.

k f i b’

1 .

~~~~~ ‘~~~~~~~ Lk~~~555~ .

~~ ... .. ~~~~ .—- -.- - .

~~I-. .- - ____
.,.- ~~~~~~ ~~ •

.

~ 

— •- •— ~

- — —i.—- ‘-i .- ~

- ••‘ — — —~ _,.‘•_ 

~~~~~~~~~~~~~~~~ .•—.~ -

~ 

.,•-•

F ’

FOREWORD

The research reported here was perfo rmed by the Army Research

Institute ’s Fort Benning Field Unit . It is part of an ongoing program

of research directed toward development of cost effective methods for

individual and collective training . This program includes research on

aultiple aspects of the design , development , evaluation , and integra-

• tion of cost and training effective training systems for the U.S. Army .

This report presents an analysis of the relative training effectiveness of the BT33, a Swedish-developed Forward Observer (FO ) training device. The analysis was designed in response to a req uest by the

USA Field Artillery School (USAFAS) and seconded by USA Training Sup~ort

Cente r Training Devices Directorate (ATSC TDD) . The study was to validate the concept that P0 training devices could yield effective tra ining

with potential cost savings . Re sults of the study were to be used as

input to the cost and training effectiveness analysis (CTEA) being conducted on the Observed Fire Trainer under development by the Army .

The research was executed at USAFAS using the FA Basic Officer

Course as the vehicle. Close coordination between ARI and USAFAS , together with dedicated test supervision by the ZJSAFAS Study Director ,

LTC Lyle Butler , Jr. , and the Project Officer , CPT Spencer A. Fisher ,

resulted in a successful test of BT33 training effectiveness.

ART research in training systems development is conducted as an • -

~

inhouse effort augmented by contracts with organizations selected as

having unique capabilities for research in the area . This stud y was

performed by ART personnel from the Fort Benning Field Unit . The project was conducted as part of Army Project 2Q76373 1A773 , FY 76 Work

Program , and Army Project 2Q763743A773 , FY 77. It was directly responsive to the requirei €SOthFtJSAPAS and ATSC TDD.

-

6JSEPH LINER ‘

~~ chnical Director

~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~ •

~ -. -r~

.r--. ~~~ ~~~~_

~

•_ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- —

I

p

5T33 TRAINING EFFECTIVENESS ANALYSIS

1

Requirement:

• To validate the concept of Forward Observer (P0) training device

cost and training benefits. This is in supp ort of a program to develop

a unit P0 training device for the U.S. Army .

Procedure :

- • A Swedish institutiona l P0 training device , the BT33, was obtained

for testing at the U.S. Army Field Artillery School (USAFAS) . It is

similar in overall simulation capability to the Observe d Fire Trainer

(Or r) being developed in the United States for a more rugged unit train- r • • ing environment. P0 performance resulting from the current P0 train -

ing course (which employs costly live fire exercises) was compared to

P0 performance s resu lting from seven variations of the course using

the BT33.

Findings :

The findings indicate that P0 training devices like the BT33 can

be effective additi ons to P0 training programs . Specifically :

• The P0 training device concept is valid . All the tested

train ing programs employing the 3T33 resulted in training

that appeared overal l to be as good as or better than the

current training program.

• Most of the problems noted in the BT33 (e.g. , poor target ~~•

displays , poor maintainability design) are being dealt with

in the OFP development .

• The most serious apparent limitation of the BT33, the two—

4 dimensional display , as opposed to a three-d imensional dieplay , is not a problem as long as it is realized that acme

time must still be allotted for additional visua l skills

development in the field environment . •1 H:

~

1

____ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ____________

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -

H
• •

Utilization of Findings: •;

The results of this BT33 training effectiveness analysis (TEA ) •

are being integrated into the OFT cost and training effectiveness • 

• •

analysis (CTEA) and OT II test plans. These results are also being

used by USAPAS , in conjunction with other data , to evaluate the poe- -

I • sibility of retaining the 3T33 at USAPAS for institut ional training . -. 

•

I

• • J

1 .:’

tLi.

~

1

~~~~

:( .

1: j
i

~~~~~~T iiT~~~~~ 

-

~

3T33 TRAINING EFFECTIVENESS ANALYSIS

~O~~ENTS

Page

• OB.~

ECTIVES . . . a . 1 I

• I’

~~~’PR0D . . . 1

SUMMARY CONCLUSIONS AND RECOMMENDATIONS 2

TECBNICAL SIJPPLEMENT 3

BACKGROUND . . . . . . . . . . . . . 3

General 3

Study Purposes . . . 3

• System Descriptions and Potential Benefits 4

I’~ ’1’R0D0LOGY 5

Test Design . 5

I Analysis Procedures 9

RESUl TS 13

• 

• 

Performance Data 13

Questionnaire Data 31

• DETAILED CONCLUSTONS 41

APPENDIXA. QUESTIONNAIRES 45

• • B. IMMEDIATE SUPPRESSION-GRID COORDINATES DATA

SUBGROUP 71

C. EEijk VALUES . 77

• DISTRIBUTION 85

~~~

LIST OP TABLES

Table 1. A s~~~~ry of significant findings resulting from ANOVAs

on the adjust fire—grid coordinates data 14
- ~~~~~~~~~~~ - • -

~• • • • • 

~

_

• • .

~~ .w_ _~

_ • - •

~“~ •• 

~~~~~~~~~~~~~~~~~~~~~

- -- • • -- • 

~

--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

••

CONTENTS (Continued )

Page

Table 2. A sununary of significant findin gs resulting from

ANCOVAs on the adjust fire-grid coordinates data . . . . 15

3. Time to locate target (seconds ) raw data means by

event and condition (adjust fire-grid coordinates • • 

•

data subgroup) 17 V

4. Total mission time (seconds) raw data means by event

• and condition (adjust fire-gri d coordinates data • •

subgroup ) 18

5. Accuracy of initial targe t location (radial error , ~~~•

meters ) raw data means by event and conditi on - •

• (adjust fire—grid coordinates data subgroup) 19 • 

•

6 • Number of rounds fired raw data means by event and

condition (adjust fire-gr id coordinates data

subgroup) 20

7. Procedural errors cut score raw data means by event

and condition (adjust fire-grid coordinates data

subgroup) 21

8. Total cuts score raw data means by event and • 

-

: 4 condition (adjust fire-g rid coordinates data

subgroup) 22

9. Time to locate target (seconds ) apti tude covariate •

1

• 

• adjusted data means by event and condition (adjust

fire-g rid coordinates data subgroup) 23 • 

•

10. Accuracy of initial target location (radial error ,

• meters) aptitude covariate adjusted data means by •

• event and condition (adjust fire—grid coordinates • •

data subgroup) 24

11. Total mission time (seconds) aptitude covariate ~ •

adjusted data means by event and condition (adjust ‘ 

~

fire—grid coordinates data subgroup) 25 •

~~

• 12. Procedural errors cut score aptitude covariate

adjusted data means by event and conditi on (adj ust

fire—grid coordinates data subgroup) . • . • • . . • . • 26

___________ 

) - 

•

;

• ~~~~~~~~~~~ ~~~~~~~
-‘ ~

•r- ,

~~~~~ ~~ ~

—- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- 

~~~~~~~~~ —“.— ~~~~-

~ ,i~

_-•.- -

~~

• _-

~~ -•• 

~~

.

~~~~r-- — ____________

CONTENTS (Continued)

P~ge

Table 13. Total cuts score aptitude covariate adjusted data

means by event and condition (adjust fire-grid

coordinates data subgroup) 27

14. RE. , RE., and RE. values 30

V

IS. FAOBC Student Evaluation Questionnaire grand means - . 32

16. Mean FAOBC question responses as a function of

experimental condition 33

17. FAOAC Student Evaluation Questionnaire mean responses . 36

18. Instructor Evaluation Questionnaire mean responses . . 38

19. Operator Evaluation Questionnaire mean responses - . . 40

B1. A sw~

m~ary of raw data subject Ns and mean scores on

the time to locate target (seconds) measure for the

immediate suppression-grid coordinates data subgroup . 71

B2. A snnvn~ry of raw data subject Ns and mean scores on

the total mission time (seconds) measure for the

immediate suppression-grid coordinates data subgroup . 72

B3. A su~~~~ry of raw data subject Ns and mean scores on

the accuracy of initial target location (radial error,

meters) measure for the immediate suppression—grid

• coordinates data subgroup 73

B4. A summary of raw data subject Ms and mean scores on

the number of rounds fired measure for the immediate •

suppression—grid coordinates data subgroup . .. .. • 74

B5. A s

~~~ ary of raw data subject Ns and mean scores on • •

the procedural errors cut score measure for the im—

mediate suppression-grid coordinates data subgroup • . 75

*

B6. A su~~~ ry of raw data subject Ms and mean scores on

the total cuts score measure for the Inmediate 

suppression—grid coordinates data subgroup 76

Cl. REj ik, REjk , and REk values for experimental condition

Nua~ er One (SXX ) 77

030

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • . • 

• • — • • — —— • —---—---~~~~~~ •• •• •• • ‘ • • • ______________________

— —f ______ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ________
•.-• •~~~~• ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- - • •

CONTENTS (Continued)

H Page

Table C2 . REijk. REj k, and REk value s for experimental

condition number two (XSX) 78

C3. REjik, REjk1 and REk values for experimental

condition number three (XXS) 79

C4. REJik, RE)k, and REk values for experimental

condition number four (SSX) 80

C5. REilk, REjk, and PEk values for experimental

condition number five (SXS) 81

C6. REj ik, RE]k, and REk values for experimental

condition number six (XSS) 82

• C7. REijk~ RE)k, and REk values for experimental a condition number seven (SSS) 83

LIST OF FIGURES

Figure 1. Mean scores obtained on the eight experimental

conditions across the three criterion events for

the time to locate target (seconds) performance

• measure (adjust fire-grid coordinates data

subgroup) 28

• 

•

L. ~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ —~~~~~~~ -- ~~~~~~ —~---~~~ -- ~~~- ~~~~~~~~~~ —~~ ~~~~~~~~~ _1

~

__l__j
‘

~~~ ‘~~~ W~ -

~~

-— ~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~ -

~ ~~~~~~~~~~~~~~ ‘

BT33 TRAINING EFFECTIVENESS ANALYSIS

Given the problems of increased ammunition costs and reduced

training manpower and budget and the need for improved effectiveness,

• the USAFAS would like to develop these options: (a) achieve the

present level of Forward Observer (P0) capability with much less manpower and budget; (b) achieve higher levels of P0 capability within - current manpower and budget limits; and (c) achieve some improvements

• in £0 capability with somewhat less manpower and budget. Innovative

• training techniques which hold promise of providing these options are

two very similar training devices: the Swedish SAAB-Scania Fire Control Simulator BT33 and the U.S.-developed Observed Fire Trainer (OFT).

The BT33 is built for the institutional training environment and is

currently available for analysis and evaluation . The OFT is now under

development and will be suitable for both institutional and the more

rugged unit environments. The question is, can these devices be integrated into unit and institutional P0 training programs so as to

provide the above options? This study is a part of the OFT Cost and

Training Effectiveness Analysis (CTEA ) program being conducted to

answer this question .

OBJECTIVES

This Training Effectiveness Analysis (TEA) of the BT33 has been

conducted to:

. Validate the concept of FO training device utility.

• Determine how such a device can be employed in the institutional environment so as to realize cost and training

benefits.

• Provide data needed by the OFT CTEA study.

• Provide information useful to finalization of the OFT design.

• METHOD

Quantitative data were obtained on the criterion performance of

students trained under each of seven different experimental training

• programs employing the BT33. These data were compared to data on the

performance of students trained under the current training program. S

- 1

~ The seven experimental training programs varied in the extent to which

• • the BT33 was used in place of live fire and the particular event(s)

for which it was substituted. The collectioff and comparison of these

data provided a quantitative basis for evaluating the relative training

1 •

___________________ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
— — ~~~~~~~~~~~~~~~~~~~~~~~~~~~ — — _________________________________

effectiveness of the BT33, used in alternative ways, as compared to • 

• 

the current training program.

Questionnaire data were also obtained from students, instructors,

and BT33 operators. These data provided qualitative information regarding good and poor aspects of the BT33 and the acceptability of it

to students and instructors for training and evaluation purposes.

SUMMARY CONCLUSIONS AND RECOMMENDATIONS - •

The results of this BT33 TEA study establish the follow ng :

• • The FO training device concept is valid. More specifically ,

all the tested training programs employing the BT33 resulted

in training that appeared overall to be as good as or better ) •

‘

~~ than the current training program.

• Most of the problems noted in the BT33 (e.g. , poor target

displays, poor maintainability design) are being dealt with

in the OFT development.

• The most serious apparent limitation of the BT33, the twodimensional display , as opposed to a three-dimensional display, is not a problem as long as it is realized that some

time must be allotted for additional visual skills develop—

inent in the field environment .

These conclusions indicate that £0 training devices (like the BT33)

can be effective additions to FO training programs. The following

recommendations are made :

• USAPAS should study the detailed results carefully for implir cations for maximizing the training effectiveness of such devices in training programs. :

A

• When the 8T33/OFT is used to a maximum extent, with simulation of several live—fire events, performance on the first

- • actual 

shoot and 

live-fire 

evaluated 

exercise 

accordingly. 

should be judged as a transitional •

I

~

;•

~~

_

I

1 2

• ~~~~~~~~~~~~~~ ~r

~~ s~~~ A - 

~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~ -~~~~-
— — ~~~~~ 

— 

* W~~~~~~~~~~~ ’ ________ • • - • • • - •

1 ~

• - 

I

TECHNICAL SUPPLEMENT

:~

-

~ (

BACI(GRC~

JND

General

- The Forward Observer (P0) is responsible for acquiring targets and

• directing artillery fire to suppress or destroy the target. The basic

skills training of Artillery FOs is accomplished by both classroom in—

struction and live fire exercises. Currently, the major training devices

(simulators) used in the U .S. Army Field Artillery School (USAPAS) 13-

week Gunnery training programs are the “Puffboard” and the M-31 14.5mm

. subcaliber trainer. The “Puffboard” is a device which allows a puff of

smoke to burst from a terrain mockup in response to a simulated round .

The M-31 is a subcaliber device that allows FO training on 1/10 scale

ranges. All other FO skills training takes place in the field and requires expensive ammunition and firing battery support.

Given the problems of reduced training manpower and budget , on the

one hand, and the need for improved effectiveness, on the other hand ,

the USAFAS would like to develop these options: (a) achieve the present

level of £0 capability with much less manpower and budget, (b) achieve

- I higher levels of P0 capability within current manpower and budget limits, -• and (c) achieve some improvements in £0 capability with somewhat less

manpower and budget. Innovative training techniques which hold promise

• of providing these options are two very similar training devices: the

Swedish SAAB-Scania Fire Control Simulator BT33 and the U.S.-developed

Observed Fire Trainer (OFT). The BT33 is built for institutional training environments (like USAFAS) and is currently available for analysis

and evaluation. The OFT is now under development and will be suitable

for both institutional and unit environments. The question is, can

• these devices be integrated into unit and institutional FO training programs so as to provide the above options? This study is a part of the

OFT CTEA program being conducted to answer this question.

Study Purposes

This BT33 TEA was designed to (a) validate the utility of the FO

P training device concept; (b) assess the relative effectiveness of the

alternative ways the BT33 can be used in £0 training at USAFAS; Cc) pro-

• vide preliminary data for input to the CTEA for the OFT; and Cd) provide

• design and performance data on the BT33 device for consideration in development of the OFT. The study objectives and methodology developed

to satisfy these purposes are described in the next major section . •

3
____ •-

‘--

~~~~ •• • • • ______ • ~~~~~~

k System Descriptions and Potential Benefits

The BT33 fire control simulator consists of the following main

parts: target area screen , central unit, display and burst projection

units, and target display unit. The BT33 simulator is designed to be

sited indoors. The battlefield panorama , including the target area ,

is projected on a screen by means of a slide projector. The observers

can see the target area from their places in the lecture room as from

observation posts. The instructor indicates the target or activities

in the target area by means of the target display unit . The students

observe the zone before them and use normal R1’O communication procedures

to make input to the device operator who simulates the Fire Direction

Center (FDC). The device operator feeds data into the control panel of

the central unit as necessary to implement student calls for fire. The

positions of the bursts in the target area are calculated by the central

unit. Burst positions simulated by symbols are projected in the target

area by the burst projectors. The size and type of symbols are deter- • 

-

mined by the central unit to correspond to the specific call for fire

of the student.

The OFT is quite similar in overall configuration and operation to

the BT33. The main configuration difference (other than unitary con-

• struction and “ruggedizing” of the design) is that the students and the

• projection equipment are closer to the OFT target area display screen

and , consequently, fewer students can be accommodated at one time . The

principal operation difference is that certain OFT fire control functions have been automated , enabling the OFT instructor to perform both

instructional and device operation roles (in the BT33 , two persons are

required to perform these roles) . The principal display difference is

• that the OF~ provides more realistic burst and target representations.

• Potential training device benefits include the following :

• A better overall training effect by variations in artillery

firing: impact burst, air burst, illumination missions, smoke,

and firing against advancing tanks and ships. Variations are

possible with respect to burst duration , angle of impact, firing direction , terrain scenes, dispersion , target movement and

type, and number of firing guns. Experience gained from live • S.

firing at USAFAS is limited in number of guns used (often one

platoon , two guns), the resemblance of terrain as compared to . ( •

actual combat arenas, and the exclusive use of nonmoving

4 targets.

fr~ !

• A shorter training time to achieve a fixed performance level.

• • A work environment that is stimulating.

• Ability to continue training during bad weather.

• _!I • n.,, ., r

~~ • - - • • • • 

__-~~~~~S.

~~~ S . _~~~~ J5 • • • ~~_‘, .- , ~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ S
~~- _____

- • 

I __

• Economic gains by saving artillery ammunition, having fewer

• artillery and FDC support requirements , and having reduced

instructor requirements for conducting live fire training.

) • Added capability for training in night observation and fire • 

• 

adjustment.

- METHODOLOGY

This study requires diagnostic as well as status information. In :1

other words, the need is not only to determine if the BT33 is training

• effective but also to identify the conditions under which it is most

effective and the factors determining effectiveness variation. This

additional information can be used to maximize OFT effectiveness through

inputs to system design and to training system procedures. Given this

need for detailed information about both causes and effects, it was

necessary to (a) obtain both student performance and subjective opinion

• data on a comprehensive measure set and (b) obtain criterion data on I student performances resulting from alternative BT33/live fire mixes.

The following two sections discuss the test design and analysis procedures used in the study.

Test Design

Design Considerations. The study design resulted from three con—

siderations: (a) what are the alternative ways in which the BT33 could

H be integrated into the £0 training program; (b) what constitutes the

criterion conditions needed to demonstrate training effectiveness ; and

Cc) how can the constitution of the various experimental groups be controlled with respect to aptitude levels? With regard to the first

• • question , the BT33 and , hence, the OFT could be utilized in £0 training

at USAFAS for the following groups: Field Artillery Officer Basic

• • Course (FAOBC) students, Field Artillery Officer Advanced Course (FAOAC)

• students, Marine Scout Officer Course (MASOC’ students, Advanced m di—

vidual Training (AlT) students, and III Corps Artillery personnel. The •

FAOBC program provides the largest and most controllable population of

• beginning students and will yield the highest payoff if the training 1’

devices are proven useful. Therefore , the FAOBC program was selected

I - as the test bed for the BT33 training effectiveness evaluation. The • USAFAS Gunnery Department evaluated the £0 portion of their FAOBC • training program in terms of apparent BT33 capabilities and identified

three points in the program at which the BT33 might well be substituted

for current training procedures. These training events are

• The Puffboard Exercise--during Gunnery Week 2 the Puffboard

• is used to introduce the students to £0 procedures and to pro- •

H111 vide the first opportunity for a few members of the class to

F ~~~~ practice. This device was described earlier. •

5

• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •
• - ‘

~~~~‘ 

~~~~~~~~~~~~~~~~~~~~~~ ~ ~~~ _

-- _________ S. -

• Daly Hill--a live-fire adjustment exercise taking place on

Daly Hill, Fort Sill, during Gunnery Week 5. All students

perform at least one mission.

• Arbuckle Hill--a live-fire exercise taking place on Arbuckle

• Hill , Port Sill, during Gunnery Week 7. All students perform

at least one mission.

Substitution of the BT33. for one or more of these events, plus the progra m as it is currently run , constitutes eight exper imental conditions

• for the BT33 effectiveness evaluation. Experimental conditions 1

through 8 were as follows:

• 

. Training Experimental conditions

events 1 2 3 4 5 6 7 8

Puffboard 8* x x • s s x s x

Daly Rill x a x s x s s x

Arbuckle Hill x x s x s s s x

*9 = BT33 substitution.

- 4

• With regard to the second question , the criterion events are the

three live fire graded exercises which follow the Arbuckle Hill training

event .

• Mobile Shoot One--takes place during the 7th to 9th weeks of

training and requi res the students to walk or ride from shoot

• location to shoot location , using multiple tar get areas . All

• students perform at least one mission .

• Bunker Shoot--take s place during the 8th to 10th weeks of training and requires the students to work from a bunker and uses one

target area. All students perform at least one mission.

• Mobile Shoot Two--takes place during the 9th to 11th weeks of . 

•

training and , again, requires the students to walk or ride from

shoot location to shoot location , using multiple targe t areas.

All students per form at least one mission .

Performance data on these graded events , if comparable, would permit

evaluation of the relative effectiveness of the alternative BT33 utili—

zation plana.

H

• 6

- ~~~~~~~~~~~~~~~~~~~~~~ ~~~~ . . 

~~~~~j~

-: 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~
•___________ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

— • ~~~~~~~~~~~~~~~~~~~~~~~~~ 

-

TT~~~~~ ~~~~ • -- ___

A question that required examination was whether or not the data

would be ccmparable. That is, would the sample of experiences of one

experimental group on, for example, Mobile Shoot One , be reasonably

similar to the sample of experiences encountered by other experimental

groups, so that performance differences between the groups would be

primarily a function of exper imental condition differen ces. There

were two factors affecting the comparability of the data . One factor

was that no two students shoot at exactly the same target/terrain cornbination or from the same perspective. The other factor was that the - mobile shoots take place alternately on two ranges which differ greatly

• in the difficulty of range estimation . With regard to the first factor , • it was determined through discussions with the USAPAS Gunner y Department and a tour of range locations that, although the variety of perceptual problems that could be encountered was considerable , the sample

• of problems faced by any one experimental group would be reasonabl y

• equivalent to that faced by other groups . Further , the assignment of

terrain /target situations to individuals could be considered to be es- •

sentia lly random. With regard to the second factor , the difference

• P between the two ranges , it was determined that estimates of performance

difference due to range difference could be computed and , hence , statistically accounted for .

With regard to the third question, control of group composition

with respect to aptitude , USAPAS currently uses the STEP score 1 for

this purpose . Their objective is to match student capabilities and

• requirements to instructional techniq ues. They do this by separating

each class into four groups based on STEP scores--high , medium high,

medium low, and low——which then receive separate instruction . For the

8T33 effectiveness study , however , it was desired to compose groups of

r •

~~

• students with similar capability for each of the experimental conditions. The problem was to develop a way of classifying students which

would both provide the instructors with easily managed groups of students in terms of instructional needs and still satisfy the req uirements of the study. After much discussion a compromis e was effected.

• Class groups wer e assigned to experimental conditions so that an equivalent sample from each of the four STEP—score categories was obtained for

each experimental condition over all classes. Each instructor had representative s from only two of the categor ies (high-low or medium highmedium low) in each group . As a safety precaution , STEP scor es were • • obtained for all students so that a covariance analysis could be con-

• 

• 

ducted if necessary.

1

The STEP , or Sequential Tests of Educational Progress, score provides

~J r an overall index of spatial , mathe matical , reading, and reasoning

skills. The concern for group equivalence on this measure resulted

• ~ from the potential for interaction between apt itude and trainin g

prog rams.

________ __________ ~~~~~~~~~ 

7
-: -•

~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~

_

~~~~ ;~~~~~~~• • • -~~~

___•_

• 

• 

The study was designed in two parts: a pilot phase , using two

L

classes, in which the training, control , and data collection methods

were checked out ; and a test phase, using seven FAOBC classes, during

which the principal data reported here were collected. Test classes,

• excluding the pilot phase , were assigned to the experimental conditions as follows:

(Subject Experimental conditions

Class N) 1 2 3 4 5 6 7 8

9—76 (107) x x x x

11—76 (84) x x x x

12—76 (59) x x

13—76 (100) x x x x

14—76 (86) x x ) - • l—7T (138) x x x x

2—7T (138) x x x x • •

.

Data Collection. Three types of data were collected: performance

data , questionnaire data , and situation descrip tive data. USAFAS instructors collected all of the data . The performance data were collected

on students as they performed their missions during the Daly and Arbuckle

Hill training events and during each of the three criterion events. The

performance data collected during the training events were used to assess

• learning behaviors which might explain performance differences in the

criterion events . The performance data collected during the criterion

events were used to assess the relative training effectiveness of each

experimental training condition. The performance data collected were

1. Accuracy of initial target location .

2. Accuracy of final fire-for-effect rounds.

• 3 • Time to locate target.

4. Total mission time.

5 • Number of rounds fired. •

6 • Procedural errors cut score.

7. Total cuts score.

• •

Data collected were separated into four subgroups, defined by mission type and location method: adjust fire mission—grid coordinates

method ; adjust fire mission—polar plot method; inmiediate suppression

• mission—grid coordinates method; and immediate suppression mission-p olar

• 

- plot method . This separation was necessary due to performance differences between these subgroups as a function of mission criteria and procedural differences.

8

~~ i ~~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~~~~~ -

~~~~~

-

~~~~ ~~~~ • ~~~~~~~~~~~~~~~ -~~~~~~ -

~•-

• • •-• • • •

---—- -— -—
• —‘-•- — • 

~~~SQ~7 ~ , 

-~~s-. ,,

~~~~~ T •~~~r’~~ ~~~~~~ 

- •

The above performance measures 1 through 7 assess the performance

of both the individual student and the system, and of both performance

during a mission phase and in the overall mission. Items 1 and 3 above

are the purest measures of student performance per se. They assess

those individual skills applied during the first phase of the indirect

fire control mission. Items 2 , 4 , and 5 are measures of the total mis—

• sion performance effectiveness by the total indirect fire control sys-

• tern. These measures partially reflect the effectiveness of the P0.

But , they also include the effects of variations among the FDC and fir—

• ing battery components of the system, as well as other conditions——e.g. ,

• weather. Variation in these factors can be assumed to be random varia—

tions, not systematically affected by the training conditions. To the

• extent that this is true, these three items are the most important

measures from an operational point of view. The major problem in using

i t these measures is that the random variation of these factors may operate

to conceal differences in FO effectiveness actually existing as an ef-

• fect of the training conditions. This is a statistical and empirical

question to be answered through analysis. Items 6 and 7 above are instructor judgments of procedural errors (pr imarily as observed in erroneous communications to the FDC) and of wasted time or rounds of

ammunition. They are direct P0 performance measures which assess procedural skills. Data on these two measures were expected to vary as a

• function of instructor differences as well as student skill differences.

Multiple instructors were randomly assigned to each condition to minimize the effect of instructor differences.

Questionnaire data were collected from both students and instructors

• in FAOBC and FAOAC. Students were asked to judge the effectiveness of

the BT33 in training different aspects of P0 performance (e.g., communication procedures vs. range estimation), how well it was being utilized,

and what might be done to improve its effectiveness. Instructors j udged

apparent device effectiveness, how well it facilitated the instructional

process, and any problems and problem solutions. Opinions were also obtained from the device operators concerning operability , maintainability,

operator training requirements, and design improvements. The question— -•

naires are presented in appendix A.

Situation descriptive data were collected to allow an assessment

of whether or not bias was introduced from uncontrolled factors and to

• enable statistical control where necessary. The principal data collected

were instructor names, event dates, event weather conditions, and instructor backgrounds and experience .

Analysis Procedures • 

• -

• J The Questions. The purposes of the study can be translated into • -

questions concerning training effectiveness, learning behaviors, device

design , and device usage: ••

• What is the demonstrated relative training effectiveness of the

• BT33 in training for live-fire FO missions?

9

- ~~~~~~~~~~~
• — • ••• • •

- 

- ‘1

• What is the judged relative training effectiveness of the

-• • BT33 in training for live-fire P0 missions?

• Are the training programs differentially effective for students of different aptitude levels?

• How well is the BT33 designed from a human factors standpoint?

• 

• • How well does the BT33 facilitate instruction as compared to

training events consisting of live-fire?

• To what extent do students and instructors want to use the - BT33 for training purposes?

• • Can the BT33 be useful in student evaluation?

The Data Sources and Analyses. Each of the above questions will

be discussed in the following paragraphs with regard to the analysis

• requirements and the data sources.

a. What is the demonstrated relative training effectiveness of

the BT33 in training for live—fire FO missions? S

The data sources for this question were the performance data from

the three criterion events. The principal data subgroup was the adjust

fire mission—grid coordinates method , as this is the most frequent mission and method combination employed. Answers to this question will

consist of seven REk values, one for each of the seven experimental

conditions. REk is a measure of the relative training effectiveness of

the experimental condition as compared to the current training condition. A value of REj~ 1.0 implies that the kth experimental condition

trains as effectively, but no more so, as the current training program.

The REk values are an average of the elements REij k where:

REiik L

~~~~ E

ij k 4

• *~~ should be noted that the relative effectiveness measure suggested by •

TRADOC (TR74DOC Pam 71—8 , Analyzing Training Effectiveness, Headquarters,

• U.S. Army Training and Doctrine Command , Port Monroe , Va., 1975) is E -~~~ divided by C, which is appropriate when the measures of E and C are such

- i that a larger value means better performance. In this study, however,

• all seven of the performance measures are defined such that a smaller

value is better. This being the case , C divided by E was selected as

- • the relative effectiveness measure so that the effectiveness ratio would

• yield values consistent with usual expectations (less than 1.0 is a

worse condition , greater than 1.0 is a better condition) .

10

~~~ -

~~~~~~~~i~ __~~~ •• • • •- 

-

::::: 

~~~~~~~. :‘

~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

--

• i-_
F 

—

where: C ~ the mean criterion performance score resulting from the

standard method of P0 training, experimental condition

no. 8;

• f B — the mean criterion performance score resulting from any

4 one of the 3T33 training conditions, experimental conditions no. 1—7;

and i = performance measure i of measure nos. 1—7 ;

j = criterion event j, with Mobile Shoot One = 1, Bunker

Shoot = 2 , and Mobile Shoot Two = 3; and

k = experimental condition no. k of conditions nos. 1-7.

The REjik values combine across i and j to form the seven REk

L values as follows:

3 7

= j=l i~1

’

~~~j k

b. What is the judged relative training effectiveness of the BT33

in training for live—fire P0 missions?

J The data were responses to Student Questionnaire items 1, 2a, 2b,

3, and 4; and Instructor Questionnaire items 1 and 4c—4f. The analysis

principally consisted of an examination of the means and standard devi- •

ations of the rating responses and an evaluation of the written responses.

Also two—way 4x7 least squares ANOVAs were run : the four aptitude levels

by the seven experimental conditions wherein the students trained on thp

3T33. However, due to missing data , not all experimental conditions had

• samples from the four aptitude categories. The five or six missing cells

appeared random in nature. The SPSS computer program option chosen for

• testing the main effects consisted of fitting the joint effects of both

factors and then removing any effect which might be attributable to the •

other factor. The results of these analyses were used to address both

this question and the following one .

c. Are the training programs differentially effective for students

• of different aptitude levels?

The data included all performance data on criterion events and • • I questionnaire rating response data. The ANOVA5 conducted on these -

•

• /. data were described under the questions above.

- -

_

~~ - ••

~

•

~~

•

~~~~~

• 

11 

- 

~~

--____

IràL~4 ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
• 

- 

•-

•

~~ ‘ 

•- • -• ~

— — ‘F•-’~ “- -

~

-

~ -

- -

~ - - ‘~~~~~-~~ • • • ••4 •- ,• • •~~~~~~~~~~~~~~~~ -

~~~~~~~~~ - -~

--“ • • • -•••

~

•-

• •

~

-

~~~

• 

—7- • 

•— - • • ••-- __________ -

~ ~~~~~~~~~~~ 

~~ —- —-

~

•,•-

~

—.

-_ _— - - -. ~ I

d. How well is the BT33 designed from a human factors

standpoint?

The data are responses to Instructor Questionnaire items 8 and

• 9; and Operator Questionnaire items 1—8. The means and standard dcviations of the rating responses were examined .

e. How well does the BT33 facilitate instruction as compared to

training events consisting of live fire?

• The data are performance data on measure 4 , Total Mission Time ,

and the Instructor Evaluation Questionnaire , items 6 and 7 • The total

mission times for training events taking place in the BT33 were cornpared to training event total mission times on the range to determine

if a time difference existed . The means and standard deviations of the

rating responses were examined.

• f. To what extent do students and instructors want to use the

BT33 for training purposes?

• The data sources are the Instructor Evaluation Questionnaire,

• item 2, and the Student Evaluation Questionnaire, item 6. The means

and standard deviations of the rating responses were examined . -P

g. Can the BT33 be useful in student evaluation?

The data sources are the Instructor Evaluation Questionnaire ,

item 5, and the Student Evaluation Questionnaire, item 4d. The written

responses were tabulated and categorized .

• Special Data Adjustments. It was noted in an earlier section that

performance scores on the Mobile Shoots did differ as a function of

range difficulty and that the difference between ranges would be controlled statistically. Data obtained for individuals performing either

• • Mobile Shoot on the East range were adjusted to conform to that obtained

for individuals actually performing the same Mobile Shoot on the West

range . The adjustment was made separately for Adjust Fire Missions and -

‘

for Immediate Suppression missions, and for each shoot. The adjustment -

•

process consisted of two steps: (1) identification of actual differ—

• ences in performance through examination of means and standard devia—

tions of scores (for all variables) obtained by students on each range

• for each shoot by mission type; and (2) transforming all East range

I scores (for all variables) to the equivalent of a West range score

through a modified z-score transformation process. The formula is:

• 

~~~~— ~~~- 

) - / ~ ijk ~~~ijk ~~~jk W

ik wik

12 

5

E.k

I

~ L~~d ~~~~~~~~~~~~~ ~~~A~ - ~~~~~~~ ~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
r ~

T - ~~~~~ -

~~~--~

- • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • ~

—-- - • - • •

Where: X

E 

individual score on East range

ijk

= translated (West) equivalent score

ijk

x = mean

s = standard deviation

and i = individual score

• j = performance measure

k = mission type.

) RESULTS

- Performance Data

Data availability was analyzed for the four procedural conditions,

based on mission type and location method, identified in the Design

section. It was found that only two combinations were used consistently

enough to provide N’s sufficiently large for analysis. Approximately 60

• percent of the data was obtained for the Adjust Fire mission using Grid

• Coordinates for location. This data set was fully usable for analysis

among experimental conditions and only these data are reported here. - Data obtained for the Immediate Suppression mission using the Grid Coordinates for location provided a somewhat less adequate sample. Several

experimental conditions had no data obtained under this combination.

Summary results of this data subgroup are presented in appendix B to

this report. The other two procedural combinations occurred so rarely

in the test situation that data for these conditions could not be $ analyzed.

The computer raw data printouts on all of the measures were examined

and found to be reasonable except for the Accuracy of Final Fire for Ef-

• fect Rounds measure. The data on this measure had been reduced improp—

erly. Hence this measure was dropped from the analyses.

• 

- . ANOVA and ANCOVA Results. The results of the ANOVAs are listed

in Table 1. Seeing that aptitude was a significant factor in seven of

- the analyses and the interaction between aptitude and experimental con-

• dition was significant in six, ANCOVAS, using the actual STEP score as

• j the covariate, were performed. The results of these analyses appear in

If Table 2.

• In reviewing Table 2, it can be seen that aptitude is significantly

• 

related to the performance measures as follows:

I

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ilIEi .

~~ , —3~ -- ___________
‘

~~

- s

~ 

-- 

~~~~~~ ‘ ~~~~~~~~ - -

am m ~~~~~~~ 0

0 III I IOIIO O I I IIIII O

111 11 .11 . .I 111111 .

9

8 Iii lOt 1 0 111111111 0 1 1 1 1. 1 1 . 1 ~~~~~~~~i ~~~~~ jp .

a_I Ci) I

____ ____ ___ ___ ___ ___ ____•

0) 1111111111 •toi loll S llIIIlIlIIII . l l .l l

0 ) 0 I I I I I I I II IIIIIIIIII I-I 04 0 •_4 .4)

U U

____ 

<

~~ 

‘

~~;~~

<: ;:~~~~~~:

~~ ..

~~ a_i .1 I I I 10000 I I 0 100 I 000 • 0 4) tIll.. . . 11 . 1 . . l

‘-I ‘

~~~~~~~ .1) lIll I I

~~~~~ I~~~~~~~~ I~~~ II~ R

04 4) S .rI I- U

a_i .

~~

ii I 

____ ____ ___ ___ ___ ___ ____ 

I -

;

a_i rI i—I C I IOOIIIII O IIIIII I .14

41 14 II I •.l 1111 . 1111111

‘ L~~L’ ‘1’ ‘ L

• I

~~.

I 41 E ’ O 4) 0 ~~~4.

’ • a_I 14 5

ES) U

14 0) ~~~0) ~~~4J 0J 5 •,g E4 14 4J ,-4 UI 14 E4 ..i t~

L... ~ ~~~~~~~

a __ __ __ __ ___

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~ ~~~~~~~ 

- 

~~~~~~~~~~~~~~~~~ • -

• ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •
i

U)

N

0 II 01 II II II 0 1

L ’

~ ____ ___ ___ ___ ___ ___ ___

UI 4) 4) U) 0 N 00 m I 0) 0 C I 0 0 II II II 1 0

I

. .

~~ • I .. II I I II I . U Cl) ~~~I LL II II II IL

•1•4

o _ _ _ _ _ _ _ _ _ _

S U • N m N 0)

• U II 1 0 0 1 II 0 1 II 4)

• 

ii H H

• 

• II ~~~~~ 

Il ~~~~ H I

I 8

~~~~ ___ ___ ___ 

8k

~ 

8

~~ 

1.1

_____ ____ ___ ___ ___ ___ ____ 0

me ~0 ,-4 ‘.4 04

~~

‘ C 00 0 • .14 i-I 0* 00 II tO II II 14

0 4) 0 i-I .1 .. II I • II II 0

U S II ‘L II II o

4

p .

~~~~~~~ A • 

~~~~~~ 3M 3 M

4)

I

- .1 . E4~~~ a_l.~~~4i ‘4 1 4 C) ‘

~~‘~~ 0 I 14 4) 00)

0) 5 .14 5 4) I 14 r4 4) rl 0) 14 .14 - 14 ‘40) V Q S

I

L.. ~~~ °‘~ ‘~~

~~~~~~~~ 

9 ~ b 8

i

~ 

blil 1

~ ___ 

H

~~~~~~ •~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

• 

~4., •

~~~~ 

_

~

,_ z _ • _~~~~~~~~~~~ _

-

.•
__ 

I •

Covaries on four events: Total Mission Time.

Covaries on two events: Time to Locate Target and Procedural

Errors Cut Score .

• Covaries on one event: Accuracy of Initial Target Location and

Total Cuts Score .

• Covaries on no events: Number of Rounds Fired.

This means that for these events the individual performance measure

scores were partially dependent upon the aptitude level (the STEP score)

of the student. On further examination of the relationship, however ,

no linear trends were observed. On some events and for some measures

students with high STEP scores did better than those with low scores.

On other events and other measures the reverse was true. Also, in some

cases, the performance of students scoring in the mid ranges was better

than that of either high- or low-scoring students. In short, there

were no definitive trends in the data. There is no indication of consistent differential effectiveness of these training programs as a function of student aptitude.

The most important occasions where experimental condition was a

significant factor are the criterion events. This is for two reasons.

• One , because differences between conditions in training events were

sometimes primarily a function of field vs. simulator situational differences (as in total mission time). And, two, because the principal

matter of concern here is whether the differences between the experimental groups, and hence the REJc value differences, are significant.

Looking, therefore, at only the criterion event results in Table 2 , we

see that performances varied significantly as a function of experimental

condition in only 3 cases out of 18: twice on the Total Mission Time

measure and once on the Total Cuts Score.

Mean Scores and RE Values. There are two possible sets of mean

performance scores per measure : (1) means based on the raw data and

(2) means that have been adjusted for the effect of the covariate, aptitude . Tables 3 through 8 present the subject Ns and mean scores based

on the raw data for the Adjust Fire Mission-Grid Coordinates method data

subgroup. Tables 9 through 13 present the covariate adjusted means for

those measures and event combinations where the covariate was signifi—

• cant at the .10 level or better.

So as to provide the reader with at least one graphic view of the

~~! results, Figure 1 was developed for the Time to Locate Target measure .

I 

- The raw data means or the adjusted means were used as appropriate, using -

•

Tables 3 and 9. In viewing this graph , the overall learning trend is

• apparent between Mobile Shoot One and Two, as is the comparative ease

of the Bunker Shoot. One thing that can be observed in this graph, and

also occurred on some other measures, is change in relative position of

the students who had experimental condition 7 (all BT33 substitution)

16
___ - ~- -

~~~~~~~~-—_ -__~•~~~~— •----—-

r M~~!V ~~~~~~~~~~ —I - -,- • •

~ 

- 

~

• - • - • -

0 0 N N N 1,5 if) 0 m N m m N N N m

- ‘.4

.‘.I 4J

- if ~~~~~~~ C~ N (‘1 N m 0 - - f 1 J U) m N i—I m t’~

5. __ __ __ __ __ __ __ __ __

4)

0 in N ‘.4 m It) in 1’- N

0 > 0 N m N m N N N N

I - • I .14 0) I)

- t 0 Ii .

~~ C) - . U a_I 0) 0’. 0) N N ‘4 4) ~ N - ‘4 m 0

- .14 ~~

I

’ -

~ 4)04 a_I N N 0 0) ‘.4 0) U) if) m m m N m m m ~~~ - . -

~~ 1 0 ) 0 0 ) 0 4’ 1 ‘-I 4)

I •14 4) 0) — 

~~ - 8 ~~~~ -~ U) in in in an m 0

U) m .14 N N m N U) ‘4 50 .4 )

4) 0)

‘.4 I UI

j OW 4)

‘I .:

~ i

t

U)

~~~~~~~~~~~~~ __ __ __ __ __ __ __ __ 

I

C~~~ I .

~~

- 10 C .14 14 .14 C 0) 14 .4 U) N U) U) N 0 0. ~

fl, I .

~ 

m m in in m ~

I

p . ‘

~~ 0) N ~~ ‘ N U) 0 N N • • - 0_I ~~ N .—4 ‘—4 ‘—4 N

— ___ ___ ___ ___ ___ ___ ___ __ ___

-,

:

~ 17

~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ 

~~~-~~-r~~~~ ~~i: ____
— •- - — - 

;—.-

~ ----

• 2.c in ‘

.0 in it) 0) N N

>1 ti) m . N N m N U)

= ___ ___ ___ ___ ___ ___ ___ ___ ___

‘4 U) 0 ‘.4 f’ . 0 N It) 0

4) ‘.4 ‘.4 1.5 0 0 1,5 U) U) U)

0) 010 ‘4 N N N U) m m N ‘.0

‘.4 ‘-I

5 4I

• 4 01. .-I ‘

.0 N in m V 14 N ‘4 N N 1.0

I .0)

•1~.

OW C 0) C

14 .14 U) 1.0 in m N N N

f4 .4 ‘.0 1.0 U) ‘

.0 m N In

0 0) 01. 0) 01. In m N

4)4) ‘4

.

~~1 >1 b

~~l E4~~r’. i-I C

N 0) 1,0 U) 0 N 01 ri

— 

(1 p-I i-I ‘4 N

. 4 , j.’

4 ’ ’

18

_________________ ;

~~J ~~~~ -

~~ 

-
-•—-- -—-———~~~~~~~~~ • .. - •

~~~~~~.••— _•-•-——.——~~~— - •- — - -

L

~~~ : ~~ :

~~~ :

~~~~~ :

~~~~ 

:

~~~~~ :

~~~ :

~ :

~~~ :

~

I) 01. It)

C

0r’. 10 C

0~~~ 14 ‘.4 N in N N N 0) 0) N

i-I N U) N U) ‘.0 m 01. N — m ‘.4 m N It) 0) 4’) in

4’

>1

• • I.I’4 i-i C

0) 4 1 10 N 0) N Cl 0 N 01 N

• $ 14.4 ~~ N ‘4 ‘4 ‘4 N

‘-I — _ _ _ _ _ _ _ _ _

.

~~

1W •g g 0 4) 5

0)

10 8

19

• • •: ~ - ~~~

- • .•• 

.

• • .

.

.. • -

~~~~T~~~~~~~ ’- • • -

~~~~~~~~~~~I • ~

-;— __ 

•

—

~

;:—

•
-

~~~~ - 

~~~~~~~~~~~~~~~~~ p— ~~~~~~~~~~~~~~~~~~~~~~ ~r~~~- 4~— ~~~~ • • _ • • 

•~~~~~~ • -

• • - ____ -

-

t 

____ ____ ____ ____ ____ ____ ____ ____ ____

01. 01. 0) m 0 N m m U) C’) N ~ I m s.D

~ 

Cl N N 0) U) 0 N

4’) 0) in 0) 0) 4) 0) •~~ 0) U)

0 $4 ‘4 .

N in 0) 0 in ‘.4 4’) 4) ‘4 ‘.4 0) N 0)

.

~~~~~

‘

.0 ‘.0 if) N 01. 0) It) 0)

m ‘4 N N C’) ri U) • -

~

= ___ ___ ___ ___ ___ ___ ___ ___ ___

)., 0) i-I i-I Cl ‘.4 N 0) 0 0 N . . ‘.0 .14 in 0) 0) in ‘.0 U) C’) It)

55 ‘.4 0) ~~~4I :1 I

~ 0) -:

- j 4’ p-I ~~~~~~~~~~~~~~~ Ii 0) 0 ~~~~~~ 

4 01. 01 u-I Cl 01. ‘0 0) U) N ‘4 It) u-I N N N

‘

5 

__ __ __ _ _ _ _ _ _ 

01.4 

ii

u-I N (‘1. 5. in ‘.0 N U)

* * * * * ——~~.— * * CC

I ”

I.j — 

~~~~~~~~~~~~~~~~~ ~~~~~~~ —~ -•

~

.

-~~ .•.—-- •--

~

•-.-.-‘--- _ ‘-~~~-.-••~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~
— ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •

~~~ -

•. •,7,• .

~~ ~~

__ -

_

~~

--

~

_ -

11— 

—_•___— ________________________

2 0) Cl U) C” 0 0 N in

p-I 01 01 0 ‘.0 01. It U) u-I ‘4 p-I

a_I ~ N in 0) 0 in 01 .-i a’) 4) u-I i-I N 0)

H 

~~~~~ 

~~: 

~

: ~~~:

~

: 

Cl 

~~~~~ ~: ~

:

- 4 Cl Cs. 0 01 ‘.0 C’s. U)

W’0 $4 N if) N N N N

$4.14

0 14 ___ ___ ___ ___ ___ ___ ___ ___ ___ .14

1 4) .14 C 4J $4 Id C u-I 0 in 411. N If) ‘.0 .4 C’)

U~~~. 14 p— f . . . . .14 0 N 0% It) U) 0) C’s. m

54 1 u-I ‘.4 ‘.4 u-I

i-I C

10 N 0) N 0% N N 01 N Cal i

~ 4,4 ‘.4 u-I ‘.4 N

— _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5 4; • U $4

.2 8

•

1 g u-I N 1’) 0) it) ‘0 

~

N U) N * * * * * * * * C - -

~

21

- - ~~.
- - - ~~~ . • - ~~~~~~~~~~~~~ ~~ ~~ ,

~ _. - • _~

_~ _______________ -. • .-•• • •..

~~~~~~~~~~~~~~~~~ 

: 

: : :

~~~ :

p

0) ‘0 u-I N 0’. 0) 0) Os.

2 01 Cl u-I 0 0% ‘0 4’) U) N ‘4 It) N N CM N

‘4 ‘4 C

10 C if) i-I 0) 4’) if) C’s. ‘.4 J4 ,-•• U) . S I • 14 u-I • C’s. 0 0) 0) 0) Cl 0% 41 .14 ‘.0 ‘4 IV) N C’s. 4’) i-I N N

0~~4 >5 C) u-I C U) 10 CM 0) N Cl CM CM 0% CM CM ‘.4 i-I ‘.4 N

• 4) — ___ ___ ___ ___ ___ ___ ___ ___ ___

8

u-I

10 I) 41 4’ 14 .9 0 . 0 ‘4 U

j :

~ 

1 •

‘it __ 1 

~ 

I 

~

I -

~

22

• -

- .

• • . • •

~~~•~~~•, ~~~~~~~~

- - 

~~~~~~

_ 

•

~~~~~~~~~~~~~~~

•

I ~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~ -~~~
- — — -_ •

- ~~~~~

- -: - ~~~~~~~~~~~~~~~~~~~~~~~

— - ___,e•-r--

~~

—_-•_— _-•- •,

___ ___ ___ ___• ___ ___ ___

- - 

‘

.0 N N 0) N ‘.0 0) ‘.0 b N CM CM 4’) u-I N CM CM

- 

~

l 4I.~j

0 1 4

Os. a’s. as.

u-I S I

.0 1

[ 4)?

-

‘4 .

C

.I.4 _ -~~~

C

f ti U

ii _ !..

~~ 

0) U) U) 0’. 0

41 5 4) i_i $4

0 A • .14 0

4) 5

4,

I -‘ - - -‘ d u-I (‘4 4’) 0) in ‘.0 N U) • Cal * * * * * * * *_ C C

23

~~~~ ____________ _______

L 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ___
- ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~ ~~~ - ~~~~~ ~~~~~ - •-‘.•• - —.. - .1 -

____ -T~~~ S ~‘.•~

•.

•‘

II ~~~~W 

I

I

’

.g

~ E•i

0 0 I

‘0

-

-

.

0) 11) 41

0 ‘4 ~

14 Cd) 4)

I. §

~~~~~

l

~

4LJ 

__ ___ ___ __ __ 

* * * * * 

: : * C:

1

’t

24

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ _________
• - • • —.

--- - 

~~~~~~~~~~~~~ •__•~_ ,

~

. — .— —.— - ~..—d— ~~~~~~~~~~~~~~~~~~~~~~

—---

1 _ _ _ _ _ _ _ _ _

01. U) N m in I’- IV) N 15 IV) CM u-I IV) IV) It)

4) N 0 U) 0) u-I 0) ‘0 Cl 0) 0) 0 u-I 0 01. N 0’. ‘0 u-f IV)

> 0 N N (‘4 0) CM ‘0 0) 0)

0)

‘4 0) $4 • 004 0) ~ N IV) 0 0’ IV) Cl IV) (‘1 4) ‘.4 ‘.4 C’s.

0 ‘4

~~~U)

u0

~~~ I

UI

• 

• .1_s .

‘

0 4) — ___ ___ ___ ___ ___ ___ ___ ___ ___

• W .,4

4~~’0

: 1 ~~~~~~~~ ‘4 U) Cl in U) CM 0) Cl U)

1 40 u-I 0 0) 0 Cl CM N U) U)

u-I (dO .14 N CM N N IV) a’s. CM ‘0 u-I >. u-I u-I

0) 5

u-I 14 41 u-I

~ ~?

14 ‘

~~~ W >. • 4 )14 -W ~~ 01. Cl ‘.5 N 5. 5. C’s. N • 8484 N u-I C”) u-I N CM ‘.0

.14 C Si_ s 10 C 1010 1.4 u-I r-I Cl IV) N 0 in C’s. ‘.0

14 u-I 0) 0) 0) IV) u-I ‘.0 u-I in

-14 u-I 0) Cl an 0 in C’) N • • U u-I ‘.4

>5

u-I C

4) 0) CM 0) ‘.0 U) U) N Cl 0 Q CM u-I u-I N

— __ __ __ __ __ __ __ __ __

u-I N C’) 0) It) ‘.0 N U) 10) * * * * * * * * C

- 

• 

25

4’ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -&
~~~~~~~~~~~~~ ~. -:. ~~~~~~~~ W T ~~~~~~~~~~~~~~~~ ~~ - • :

it

0) U) U) 0 u-I 0 N CM 0 4) ~~ • . • . •

• (dU) 0) 0 U) 0 U) N ‘.0 0’ U) u-I u-f u-f

0 .1 41

‘ 8 8 (‘4 C’s. U) ‘.0 ‘.0 U) 0) ‘.0 S U) C”) u-I r-I N CM N N

-I-S W

.

~~ 

u-~

4 CV) u-I N 0) N U) U) .5. • Ii N u-I u-f u-I u-I in 4) 14 0’ • .5.4 84 .14

~~~4J -

~~ -

,

5 0)

I 8.2

4,

~~~ ~~~~L.. 

~~ -

‘ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~ .l..

~~~~~ •
—- ~~~~~~~~ ~~~~~~~~~~~ ~~~~~~•_• • --

0 C I I u-I Cl 0% ‘.0 ‘.0 U) 0_Il-I • I • . . . u-I u-I 0 IV) in ‘

~~ a’s. -s-I 41 CM I CM u-I I u-f u-I N

‘8 8 C I I

~~~ U) U) I- in U) 0)

U) C”) I I 4’) N C’s.

U’

4’ ___ ___ ___ ___ ___ ___ ___ ___ ___ 0

-i-I

4)

-s-I .1)

I 

_ _ _ _ _ _ _ _ _

_ H__ __

• C’s. 41(d u-I

4) dl I

I; a __ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _

-s-I ll

‘04, I

-rI -rs.

0)

‘4 5 0 0 4) -‘-I C) .0 dl

U -I-I

10 0)

u-I U

• 1 8

u-I

- 1 J• -~~

14

,4

~ 1 I i a fi I I I I it ‘4 (‘4 4’) 0) It) ‘.0 N U)

* * * * * * * * C

27

I
~

iI T

~~ *T __ -5u---~~ 

)

/ 

0)

‘U)

C”) u-I ’.O CMU’)N 8 U

I L

/\

~~~\

\

\ ‘ 111

I

) 

‘‘ 8 14 4110 5.. 0)41

~~

4 IdS4,

1 I • 14

I 1 .0 ~~ I I

/ J

I I W - -

/ 1 4) I I

I - I

I I

I I 04 ) f

~ I I

a,

/1; / I • i-l

N N U)’. 0 u-4 I’1 1(5 0)

i

p Th’ W1.4~~~ ZOO

0 0 0 0 0 0 t ,

.

in 0) C’) N u-I

1— 41

• f-

~’. (sptXO3eS) ~e5.z~j

~ ~~~~~O’I o~ eur~

28

____ —— •• • .. .

~~~~~~~~~ T . - -— - • • -

L ~A ;:Ii

~~~~~r~

.

~- ~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .
r-r ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~- •

~~~~~~~~~~~-~~~~~ -,--- • •~~~~~~~— s1, 

*— —_— •—---- ____________________________

across criterion events. On Mobile Shoot One they were the poorest

performing group, on the Bunker Shoot they were intermediate, and on

the final Mobile Shoot Two they were the best performing group. This

same trend did not occur for all measures or for all experimental

conditions. Examination of Tables 3 through 13 shows that Time to

Locate Target and Accuracy of Initial Target Location did decrease

from Mobile Shoot One to Mobile Shoot Two. However, Total Mission - - Time (Table 4) showed an increase over these same shoots; while the

Number of Rounds Fired (Table 6), Procedural Errors (Table 7), and

Total Cuts Scores (Table 8) showed mixed changes by condition with the

- scores being closely similar overall from the first to the second Mo-

. bile Shoot.

It was hypothesized that one advantage of the BT33 over the livefire situation may be a reduction in time needed per mission.. The

live-fire shoot was thought to take longer due to the delayed response

from the FDC. If this is true then either more students could shoot

more missions per training period in the BT33, and thus receive more

practice , or the training time could be reduced, resulting in a savings.

The mean total mission times per BT33 versus live—fire sessions were

computed for the training events and it was found that the anticipated

time saving for the BT33 missions was indeed realized: 8.5 minutes

versus 15.2 minutes, a difference of 6.7 minutes or 44%.

REj

~j~ values were calculated using either raw data means or co-

• variate adjusted means according to the rule: If the covariate was

significant at the .100 level or better then the adjusted means were

• used ; otherwise the raw data means were used. These REijk values are

presented in appendix C. The overall REikI RE), and REk values are

sunmtarized in Table 14. Given that experimental condition generally

did not account for a significant amount of the performance variance ,

small differences between these values are not important .

Review of Table 14 brings out some overall trends of interest with

j respect to each of the RE indices. A close examination of the REik -

• • values discloses that the relative position of conditions changes in

many cases across events. For example, experimental conditions 3 and 4

(XXS and sSX) have the highest ratings on Mobile Shoot One and lower

relative positions on Mobile Shoot Two. Conditions 5 and 6 (SXS and - 

- XSS) , on the other hand , occupy lower relative positions on Mobile Shoot -- -

. One and higher relative positions on Mobile Shoot Two. Which simply

• says that the condition which re~ ~ in the best initial live-fire

j • graded event performance is not ~ ily the one which results in

the best ultimate performance. Iii - the ~~ va •es , it can

be noted that, overall, exper1ment~ i ons -or~ -

~

. . ating the BT33

result in performance equivalent t ~rent ~ . gi tm 3 - bile Shoot • 

~~~~

• One (REj a .98) and better than the current progra ir - nker Shoot

and Mobile Shoot Two (REj — 1.20, 1.18). with r~qa ‘Ek values, -

~~~

the overall order of the experimental conditions w.~ ~ t to train—

ing effectiveness is:

29

~~~~~~~~~~~~ ~~~~~~~~~~~~ ~ ~~~ 

k

~

&~L ~.. -

~~~~~ i

4

~~~~~~~~~~~~~~~~~~~ • - ~~~~~~~~ ~~~
__

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- 1

1- -

~ cperimental conditions

- 

#4 (SSX) 1.20

#5 (sXS) 1.19

• #7 (SSs) 1.12

• #1 (SXX) 1.11

- 

• • #6 (XSS) 1.10

#3 (XXS) 1.08

• #8 (XXX) 1.00

#2 (XSX) .98

It can be seen that the REk values fall into three groups, with conditions 4 and 5 being the best , and conditions 2 and 8 being the poorest.

Table 14

REjk~ RE., and REk Values

Events

Experimental Mobile Bunker Mobile R~~ condition Shoot One Shoot Shoot Two values

#1 (SXX) .98 1.43 1.03 1.11

#2 (XSX) .86 .94 1.13 .98

#3 (XXS) 1.05 1.22 .97 1.08

#4 (SSx) 1.06 1.33 1.22 1.20

#5 (SXS) .98 1.21 1.39 1.19

#6 (XSS) .93 1.21 1.17 1.10

#7 (SSS) 1.01 1.04 1.32 1.12

#8 (Xxx) 1.00 1.00 1.00 1.00

RE~ values .98 1.20 1.18

It is again pointed out that the ANCOVAS determined that the dif-

• ferences between performance as a function of experimental condition

I - 

were significant in only 3 cases out of 18. Hence , although the above . -

REj and REk values are of interest, the differences between them are of

negligible importance. Given the lack of statistical significance in - - • - the performance differences, the cost savings to be realized from use of

~~ 

~~~~~~ P0 training devices is of greater importance .

30

,
—.• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •, —~- y ~~~ - --~~ ~~~~~~~~~~~~~~~~ , 

- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- •

-

• ~

I- —• - _________ _______ _______

• Questionnaire Data

• FAOBC Student Evaluation. This questionnaire is contained in

appendix A. Student responses are discussed in the following paragraphs. Mean responses to rating questions are listed in Table 15.

The data on each questionnaire item were analyzed (ANOVA) to determine • if either experimental condition or aptitude determined the responses.

Where either a main effect or interaction influence was significant ,

this will be discussed.

a. FAOBC Questionnaire items 1 and 2.

- The first four rating questions, contained in questionnaire items

1 and 2 , asked the student to evaluate the effectiveness of the BT33—-

both overall and for specific clusters of skills. It is clear fr-cm

the mean ratings that while the device is rated overall as moderately

effective (

~ = 3.2) , it is seen to be most effective as a procedures

and communications skills trainer (

~ = 3.9 , 3.7) and somewhat less ef—

fective as a visual/perceptual skills trainer (1 = 2.4) .

The basis for these ratings is spelled out in the responses to

item 2b concerning differences between performance in the BT33 vs. on

the range. Nineteen percent indicated procedural differences exist and

13% indicated communications differences; but 71% felt that visual!

perceptual differences exist.

If the FAOBC students indicated that a difference did exist , they

were to describe the difference , the duration of its impact on range •

performance, and suggestions for improvement . The differences noted

primarily revolved around visual/perceptual problems with the BT33.

- - ( The problems included difficulty in range estimation , lateral shift

estimation, height of burst (HOB) estimation, and rounds spotting in

the BT33 with a short term impact on range performance. The suggestions

included improvement of the film quality, the inclusion of pictures of

Fort Sill terrain (as opposed to the shots of European terrain supplied

• by SAAB-Scania) , and the possibility of 3-D projection.

Although few students noted any differences with respect to corniminications skills, some who did felt that the instructors were not - •

being sufficiently strict in the BT33 and were allowing incomplete

• • transmissions in the BT33 classroom environment. Others suggested the - addition of radios or phones to the BT33 simulation.

ANOVAs on the four rating questions in items 1 and 2 determined

that experimental condition was a significant response determining

factor on three of these. The individual means per condition for these

three questions are presented in Table 16.

- 

- 31.

($

1 ___ 

- ~~~ -4

~~--- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -

~~~
• • ~~~~~~~~~~~~~~~~~~~~~~ -—- - - --—-.--~~ - • _________ 

- ¶ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~

Table 15

FAOBC Student Evaluation Questionnaire :

~ 

-

Grand Means (N’s = 138—278) L. ~

.

I

~~ 

•

-:

~

Mean - Questionnaire item rating

1. Overall BT33 training effectiveness - 3.2* - - •

2. BT33 effectiveness: Procedural skills training 3.9

BT33 effectiveness: Communication skills training 3.7

BT33 effectiveness: Visual/perceptual skills training 2.4

3. Puffboard vs. live—fire effectiveness: Overall 2.0

BT33 vs. live—fire effectiveness: Overall 2.5

4. BT33 vs. live—fire effectiveness: Puffboard 3.0 -

BT33 vs. live—fire effectiveness: Daly Hill 2.5

BT33 vs. live—fire effectiveness: Arbuckle Hill 2.3

8T33 vs. live—fire effectiveness: Bunker Shoot 2.2

BT33 vs. live—fire effectiveness: Mobile Shoot One 2.0

BT33 vs. live—fire effectiveness: Mobile Shoot Two 2.0

6. BT33 availability for practice desirability 3.5

*IteIns rated on a 5-point scale with 1.0 — very low and 5.0 — very high.

- 

•

• • I I -

y

32

(

~~ i ______

~~~t ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _________
-

Table 16

Mean FAOBC Question Responses as a Function of -

Experimental Condition (N ’s a 19-72)

Experimental conditions

1 2 3 4 5 6 7

FAOBC questions (items 1 and 2) SXX XSX XXS SSX SXS XSS SSS

Overall effectiveness* 3.0 3.2 3.5 3.1 3.1 3.0 3.4

Procedura l skills effective** 37 42 4.3 3.8 3.7 3.9 4.3

Communication skills

effectiveness*** 3.6 4.0 3.8 3.5 3.7 3.5 4.2

*Experiinental condition p = .046 , Aptitude Interaction p — .027.

**Experjmental condition p = .019 .

***Experimental condition p = .042. -

In examining the above , certain overall trends can be noted . The

students exposed to the BT33 only once (experimental condition No. 1)

tended to give low ratings to all three questions. Students exposed to

the BT33 the maximum amount (condition No. 7) tended to give the highest ratings on BT33 effectiveness. Students exposed to a mix of Puffboard , BT33 , and live-fire training exercises (conditions 2 and 3)

tended to give moderately higher ratings. Finally, students exposed

to the 5T33 twice , with either the Puffboard or live fire on the other

training event , tended to give moderately lower ratings.

The interaction effect was uninterpretable due to empty cells.

It can only be noted that on the fir-st question, concerning overall

BT33 effectiveness, the responses across the four aptitude categories j

did interact with the experimental conditions. • 

- - -

b. FAOBC Questionnaire item 3.

The two rating questions contained in this item compared the

a training effectiveness of the Puffboard and the BT33 to that of a

live-fire exercise . Although both were judged to be somewhat less

effective than a live-fire exercise , the BT33 received a higher rat- -

‘

m g than did the Puffboard Cx — 2.5 vs. 2.0) .

H

33

~~~ •~~~~~~~~~~~~~~~~~ ~

—~~-

~~~~~~~~~ ‘~~~~~~~~~~ ‘-
- ~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~ - 

~ “~~TT ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

c. FAOBC Questionnaire item 4.

Item 4 contained six rating questions, one yes/no question and

a verbal response question. The rating questions (see Table 15) were

- more specific versions of item 3 , where the student was asked to judge

• the comparative effectiveness of the BT33 in place of each of the cur-

¶ rent training and criterion events. The responses were as might be — expected-—highest for the first training event Cx — 3.0) and gradually

declining to smaller values on the final criterion events Ii — 2.2 ,

- 2.0) . Data on three of the six rating questions (item 4b) were influenced significantly (p — .040 , .038 , and .002) by an interaction between the aptitude factor levels and the experimental conditions. Due, however, to empty cells the impact is uninterpretable .

The yes/no question under item 4d concerned the possible use of

the BT33 for student evaluation purposes : “Do you think a realistic - evaluation of your skills could be made in the BT33 rather than at

Bunker Hill?” Seventy—seven percent of the students responded “no” • 

-

to this question .

The verbal response question asked: “Are there any other FADBC

field exercises where the BT33 might be an effective substitute?” Responses pointed to the following:

• The terrain association class, a dry—fire shack shoot, held

at Apache Ridge ;

• Whenever an event is weathered out;

e Could be used in conjunction with the FDC (FAOAC) exercise;

• Could be a means of later technique polishing and to maintain • proficiency; and

• Should not be used as a substitute, but could be a valuable

• addition , especially for procedural and communications skills

development. - -

d. FAOBC Questionnaire item 5.

• Item 5 asked for student recommendations for improvement of the

BT33. The first area covered (item 5a) was terrain, target, and burst

• presentations. Here the students made the following recommendations, with the first two appearing most frequently:

j • Make the burst presentations longer, clearer, and with greater

- 4 realism. When burst is in woods, display a puff of smoke.

- 

• Use displays of different types of terrain, including shots of

difficult Fort Sill terrain, and change them more frequently - during training. - •

34

J -

~

• -

iIl

~

_

~~~~~ ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~
•

~~~~~! 

- - - - ~~~~~~~~~~~~~~~~~~~~~~~~ -

~~~ ~~

_ •~~~..~~~~~~~~ 4~~~~ • • •- -

• Improve the sharpness of the terrain display, but provide *

fewer terrain features.

• Simulate a mobile shoot by using different shots of the same

( target area.

• Close targets need to be clearer and continuously displayed.

Target size and clarity change would aid depth perception.

The second area covered (item Sb) was the manner in which the

BT33 was used by instructors. The most frequent responses here were: - (a) the instructors did a good job, (b) smaller classes would be very

desirable so that more missions could be run for students, and Cc) which - students had performed a mission needed to be monitored more carefully - in that not all students received an opportunity to perform during each

event. Some students also urged that greater emphasis be placed on - I developing specific procedures, communications, and map reading skills.

The final question under this item (5c) was an open-ended one,

asking, “Anything else?” The majority of the comments made here have

— already been presented under the above items 1 through 4. The only

- additional ideas were: (a) the BT33 should be made available for student use during evening hours and weekends and (b) the BT33 was useful

• in that it allowed the instructor to train students how to handle contingency situations like lost rounds and FDC errors.

• 

• 

e. FAOBC Questionnaire item 6.

Item 6 asked the student to indicate the extent to which he would

- like to have the BT33 available for additional practice outside the - presently scheduled classes. The responses were generally moderately

favorable, with a mean response of 3.5.

- These data were influenced significantly by an aptitude X experimental condition interaction (p = .026). Due, however, to empty cells

the results are uninterpretable.

FAOAC Student Evaluation. The FAOAC, MASOC, AlT, and Zil Corps

Artillery questionnaire is presented in appendix A. It was responded

• to only by FAOAC students. The mean FAOAC student responses to rating - 

I - questions are summarized in Table 17 and discussed in the following

• f paragraph.

• I -

~~ a. FAOAC Questionnaire items 1 and 2.

- 

~

. The FAOAC students gave_the BT33 fairly high ratings on training • 4

effectiveness both overall Cx — 3.5) and for each of the three sets

• ~~~~ 

of skills (x — 3.1 to 4.0). The ratings given by.- the FAOAC students

were higher across the board than those given by the FAOBC students

(compare Tables 15 and 17), but in agreement that the BT33 is most

- 

-&. effective as a procedural and communications skills trainer and least

(tI _______

______ 

_______________ 

~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~ ~~~~ -

~

-

~~~~~ -~~~• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• -•- - •tl~

• - ~~~~~~ ~~~~~~ 

- - • - • • - - - - - 

~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- — —

- • • -

effective as a visual skills trainer. These judgments of differential

eff ectiveness per skill type can again be explained by perceived differences between visual performance in the BT33 versus on the range .

Only 12% perceived procedural differences to exist and only 16% perceived coi,a~

in~icationg differences to exist; wherea~ 61% perceived -

•

- visual/perceptual differences.

FAOAC Student Evaluation Questionnaire Mean Responses

4

Mean

4 Questionnaire item rating - -

1. Overall BT33 training effectiveness (N = 142) 3.5

2. 8T33 effectiveness: Procedural skills training (N = 142) 4.0

BT33 effectiveness: Communications skills training

• - (N = l42) 3.5

I BT33 effectiveness: Visual/Perceptual skills training

(N = 142) 3

•

.l

3. Puffboard vs. live—fire effectiveness: Overall (N = 48) 2.4 - 

• I 

BT33 vs. live—fire effectiveness: Overall (N = 98) 3.1

• 4. BT33 vs. live—fire effectiveness: FAOAC program (N — 127) - 2.9

-

~ 6. 8T33 availability for practice: Desirability (N = 111) 2.8

*Iteme rated on a 5-point scale with 1 0 = very low and 5 0 = very high

• - 

I If the FAOAC students indicated that a difference did exist, they

were to describe the difference, the duration of its impact on range

• performance, and suggestions for improvement. The main differences

noted were the lack of radios and 3-D displays. The principal problems

noted revolved around the difficulty in visually estimating ranges and

- ROBs on a 2-D display where neither burst nor target sizes changed as

a function of distance. The problem was apparently most acute for

students seated close to the screen. The students generally felt the

- 1

~ - impact of this perceptual problem on range performance was short-lived.

• They recommended improved film quality, a greater variety of pictures,

and 3—D proj ection.

36

_______ ~~~~ — -~~~~~~~~~~~~~~

-

~~~~-• • - • - •- ~~~~~~~~~~~~~~~~~~~~~~~~ ~
• ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ •___••__•i_.__ _••

• •__ • ~~~~~~~ -~~ - - -

b. FAOAC questionnaire items 3 and 4.

The FAOAC students also gave higher ratings to the two questions

contained in item 3 than did the FAOBC students (compare Tables 15 and

17) . They judged the Puffboard to be somewhat less effective than live

fire (x = 2.4) , while the BT33 was judged to be as effective overall

as live fire (i

~ 3.1) .

In judging the effectiveness of the BT33 in contrast with live

fire for their specific FAOAC training program (question 4a) , the

students againjudged the BT33 to be just about as training effective

as live fire (x = 2.9). The responses to question 4b, “How often and

• • in what manner do you think the BT33 might be best used in the FAOAC?,”

were grouped into the following categories:

Response category % of total response

(N = 145)

• The BT33 could be used entirely,

eliminating live fire 4%

• The BT33 could be used “extensively, ”

“f requently, ” or “equally” 9%

• The BT33 could be used once or

twice for refresher training 78%

• • The BT33 should not be used at all 9%

c. The FAOAC questionnaire item 5.

Questionnaire item 5 asked for recommendations for improvement of

the 8T33. The first question in this item concerned terrain, target ,

• and burst presentations. The recommendations were

Terrain : provide more scenes

Targets: make more realistic

Bursts: color them; scale them, making the close in ones

larger than the current ones; and display for a

longer time .

- I The second question concerned the manner in which the BT33 is

-• :- used by the instructors. The comments made here and elsewhere can best

— • be represented by one of the responses: “Instructors must be sure that

target location and orientation on the BT33 are accurately set up. Poor

data inputs defeat the purpose with respect to student attitude .”

M Granted, the 0T33 can be used to introduce the students to the fallibilities of the FDC and gunnery. But the matter of how and when to do

this apparently needs to be reconsidered .

44 ~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -
- - -

~~~~~~ ~~~~~~

ii.

The third question was a catchall , “Anything else?” All the ; - ideas presented here have already been discussed elsewhere.

d. FAOAC questionnaire item 6.

Whereas the FAOAC students gave higher ratings than did the FAOBC

students on all the preceding questions, they reversed the trend on

- 

this item. While the FAOBC students had indicated a definite desire - to have the BT33 available for practice (7 = 3.5), the more experienced

FAOAC students did not feel a need to quite the same extent (

~~ — 2.8).

• Instructor Evaluation. The Instructor Evaluation Questionnaire is presented in appendix A. The mean instructor responses are su~~~rized in Table 18 and discussed below.

-

• Tablel8

- Instructor Evaluation Questionnaire Mean Responses

• Mean

Questionnaire item rating

1. Overall BT33 training effectiveness (N = 19) 3.4

I 4. BT33 vs. range training environments: — - How well the student learns (N = 19) 2.5 • • How quickly the student learns (N = 19) 3.1

For FAOBC training (N = 9) 2.8

H For FAOAC training (N = 5) 1.6

For AlT training (N = 4) 3.1

For MASOC training (N = 3) 3.3

For field FO training (N 19) 2 .2

6. BT33 facilitation of instructor-student interaction

• 

-

~~ (N — l9) 3.0

7. BT33 facilitation of student learning problems I diagnosis (N — 19) 2.9

- -

~~ 8. Adequacy of BT33 presentation management (N — 19) 3 4

I;

~~~~~~~~~~ rated on a 5-point scale with 1.0 — very low and 5.0 = very high.

_________—

~ 

___ ___ 

-

~ 3$ 

- 

— --• -•

~~

-- 

~

_•_

~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~ • . ~~~----- -~~- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ---
— 

~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • . •

~~~ 

~~~~~~~~~ ~~~~~~~~~~~~~ ~~ - -

~~ — • •

(

a. Instructor Questionnaire items 1, 2 , and 4.

The instructors rated the BT33 as fairly effective overall (item

1, 5

~ 3.4) . The FAOBC instructors (N = 9) indicated that 22% of

FAOBC training could take place in the BT33 while the FAOAC instructors

(N = 5) indicated 20% for FAOAC training (item 2).

Item 4 asked the instructors to make judgments regarding student

learning behavior and the effectiveness of the BT33, as opposed to live

* fire, for specific training programs. Two of the items in Table 18,

( BT33 effectiveness for FAOAC training and for field FO training, were

given rather low ratings and hence warrant comment. The low rating

given by five instructors of BT33 effectiveness for FAOAC students

(x = 1.6) seems to be in conflict with the moderately high rating given

by the FAOAC students themselves (Table 17, 7 = 2.9)--but it may not be.

The majority of the FAOAC students also indicated that the BT33 would

be most useful for one or two FAOAC sessions but not much beyond that.

This may be just what the instructors were saying as well. • -

.-

The rating given with regard to field FO training, 1 = 2.2, is a • - -

sore real rating in that it is based on a larger N of 19. It is also

more of a problem in that the major objective of the OFT now under de-

• velopment is to provide this unit, or field, FO training. It may be

• that the instructors were actually rating the 5T33 with respect to its

lack of suitability for the more rugged unit environment rather than I -

with respect to its training value. On the other hand, the ratings may

indicate a real doubt regarding the value of a FO unit training device .

If so, then the OFT CTEA may need to be extended to the unit environment

with the LRIP models prior to initiating full rate production of the

OFT. Certainly the opinions of the instructors regarding the value of

a FO trainer in the unit environment need to be reexamined.

b . Instructor Questionnaire items 5 through 8.

Items 5 through 8 dealt with questions related to student management in the BT33 environment. The first of these, item 5, concerned

the issue of whether student performance could be evaluated in the

BT33. Forty—seven percent of the instructors felt that the answer was

y’. . These instructors generally felt that this evaluation should

• take place early in training to evaluate procedural learning and

progress. - 

- -4

I 

-

• With regard to instructor capabilities to interact with students

and diagnose their learning problems, the instructors seemed to feel

that the BT33 and range environments were about the same (x = 3.0 and IF. 2.9) . ~

Finally, with regard to instructor capability to manage 8T33

display presentations, the instructors on the whole felt fairly well

~~~~~~~ 

-

~~ in control. The mean response to question 8 was 3.4, with 90%~ .ndi—

• I cating that no specific problems exist. -

~

—— —-—-—__~~~~ -—•~~~~~-——- ~ -.~~~~~~~~ —-• • • 

-•
., 

4~~~ UI

~ I~~I~~ ~~~~~~~~ — ~~~~ I

~’~ ~~~~~~~~~~~~~~~~ ,

~~~-.r- -— • I

• Operator Evaluation. This questionnaire is presented in appendix A. Operator responses are discussed in the following paragraphs.

a. Questionnaire items 1 through 4.

I The first four items asked the operator for ratings, yes/no responses, and opinions regarding the operability of the BT33. The mean

ratings for items 1 through 3 are given in Table 19. In discussing

the issue of how easy it is to damage the equipment, it was pointed

out that the target and sound track tapes were very easy to damage or

tear, and that projector fan belts and firing battery buttons came

apart easily.

Table 19

- Operator Evaluation Questionnaire

Mean Responses (N = 3)

[ I

Mean

Questionnaire item rating

Equipment operation:

1. Overall ease of operation 4.0*

2. Performance error avoidance 3.9

3. Equipment damage avoidance 2.4

Operator background: 4

5. Electronic background required 1. 7**

• 6 • Forward observer background required • 2.7

• 

• 

*~~ t~~~~~~ 1 through 3 rated on a 5—point scale with 1.0 = poorest rating

and 5.0 — best rating possible .

- 

I ~1 **It~~~ 5 and 6 rated on a 5-point scale with 1.0 = very little and

5.0 — a great deal .

• ~ In responding to item 4, “Are there any other specific problems

in the layout or operation of the device?,” two of the three operators

• answered yea . The problems mentioned were that the unit should be 1 more compact and that the projectors were difficult to service and

align due to location.

40

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~• •J•.
• • -

~ - 

~~~~~ ~

1 V T -T • --------~~~ •• -- •

~~~‘~~ -~~~~~~~~~ -.

~~~ - 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • - .

• _______ -

b. Questionnaire items 5 and 6.

Items 5 and 6 concerned the amount of electronic equipment and TO

background required to become a 5T33 operator. As can be seen from

• Table 19, the operators felt that little electronic background was

needed, but that a moderate TO procedures and map reading background

would be helpful. Comments were to the effect that a 6 months’ background in the 13E MOS was desirable.

c. Questionnaire items 7 and 8.

These items address the questions of how much training and practice is needed to become an operator and then maintain proficiency.

The operators indicated t~

-idt_2 to 5 days (7 = 3) classroom training

was needed and 1 to 5 days (x = 4) OJT. Two of the operators thought

that once a month operation was sufficient to maintain proficiency ,

while the third operator felt that, given 2 months’ experience on the

• equipment first, once a week was necessary.

DETAILED CONCLUSIONS

The conclusions of the study are discussed in terms of the ques—

tions presented in the Analysis Procedures section.

a. What is the demonstrated relative training effectiveness of

the BT33 for live—fire TO missions? a

The differences in performance on the criterion events did not,

except in 3 cases out of 18, significantly differ as a function of experimental condition (see Table 2). Given that, it can be said that,

overall, the seven experimental training programs with the BT33 are as

good as the current training program. Experimental conditions 4 and 5

(SSX and SXS) tend to produce slightly better training overall (REk =

1.20 and 1.19, while experimental conditions 2 and 8 (XSX and XXX, the

current program) tend to produce the poorest training overall (REk =

.98 and 1.00) (see Table 14). Programs 6 and 7 (XSS and SSS) tend to

be intermediately effective (REj~ = 1.10 and L.l2) and are the most

cost effective due to the reduction of live—fire and ammunition costs.

b. What is the judged relative training effectiveness of the

• • BT33 for live—fire TO missions?

The FAOBC and FAOAC students gave the BT33 fairly high ratings

• for early procedures and communications skills training (see Tables 15

and 17). The BT33 was rated as somewhat less than moderately effec—

tive for visual/perceptual skills training, however , and short-term

problems were noted upon transition to the field environment. This

- • ‘i

~~~ visual problem may be the reason for the lower RE value obtained on

Mobile Shoot One (see Table 14) . The fact that the RE values for the

final two graded events were higher substantiates the judgments that

41

_ ___ - • •~~~~~~~~~~ -• ~• • . - • •

~ i~

A• 

‘d~~~~~~—.J ~~~~~~~~~~~~ 1 _

~~~~~~~~~ 5L2 ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~ ________________________
~~~~~~~~~~-r- 

~~~~~~~~~~~~~~~~~~~~~~~~~ — - 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- -~~~~~~~ ~~~~~~~~~~~~~~~~~~~~

-__-

the transition problems were indeed short-lived and negates the ianportance of the problem given some field training in live fire. •

I - The instructors judged the BT33 to be moderately effective overall and for three groups of students: FAOBC, AlT, and MASOC (see

Table 18). They judged it as less than moderately effective for the

FAOAC and unit students. The implications of the latter two exceptions

for OFT are not clear for two reasons:

1. The FAOAC students, in contrast to their instructors, gave

• the BT33 even higher ratings than did the FAOBC students.

- • • This difference of opinion, based on a very small N in the

case of the instructors (N = 5), may give greater weight to -

- • the higher rating given by the students.

2. The BT33 is not designed for the field environment, whereas

OFT will be.

It is not known whether the instructors were rating the BT33 in terms

of its field environment suitability or in terms of its likely unit

training effectiveness.

The conclusions to be derived here are two. One, the BT33 is

judged to be effective in the institutional training environment, so

long as the need for adequate live—fire training continues to be recognized. Two, the instructors should be assessed again regarding their

views concerning the value of the OFT in the unit environment. If

doubts really exist then it might be well to extend the OFT CTEA evaluation to the unit environment when the LRIP OFT models are received.

• c. Are the training programs differentially effective for stu—

dents of different aptitude levels?

Results of the ANOVAs indicate that the eight experimental programs were differentially effective based on aptitude differences for

7 out of 30 data points. The interaction between experimental condition and aptitude was significant on three out of five events for the • H

Total Cuts Score , two events for Total Mission Time , and one event for

Accuracy of Initial Target Location (see Table 1). The significant -

~~~~

interactions show that the individual programs produced different performance results for subjects of high versus low aptitudes, but the *

relationships between the STEP score and performance were not consistent.

A more detailed examination was also conducted to determine if

the BT33 is itself differentially effective for different aptitudes as . 

*

I compared to the current training environments. The relationships be- • 

—

tween STEP score and performance, as assessed by the correlation coef—

1— ficient, were examined for each individual training event x experimental

condition situation. Again , no consistent relationships were found.

4

42 

-

~~~ -• • -~~--~~~~ - -~ -~~~- - • - • ,*.~ - • - - * -- ----—___

~~ •j • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

I
--•-—-—--. -- —~

- • • 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~~~— -—-~ —~ :— -—- - - i—~~i

~~~~~~~~~~ - - • - ——- —-- c— — - - —

d. How well is the BT33 designed from a human factors standpoint?

• The operator and instructor judgments indicated that the BT33 is

designed fairly well from an operability standpoint but not so well

from a maintainability standpoint (see Tables 18 and. 19). Specific

- problems are discussed on pp. 39-41.

- - e. How well does the BT33 facilitate instruction as compared to - training events consisting of live fire?

The BT33 appears to facilitate instruction at least moderately - well in terms of student learning speed, instructor-student interac-

• tions, and diagnosis of student learning problems (see Table 18) . Fur-

• ther, it was determined that a BT33 mission takes an average of 8.5

minutes, as compared to 15.2 minutes for a live—fire mission; i.e., it 4 takes 44% less time to shoot a mission in the BT33 than in the field.

• Hence either students can shoot sore missions, thus receiving more

practice, or the class time could be cut. The latter option should

not be employed however without careful testing to insure that adequate

training would still be received.

f. To what extent do students and instructors want to use the

BT33 for training purposes?

Based on their coms~

ents and evaluations regarding training ef— I fectiveness, plus the desire evidenced by both the FAOBC and FAOAC

students for additional time on the BT33 outside of class (see Tables

— 15 and 17) , it appears that the majority of students do want to use the

8T33 to at least some extent. The instructors indicated that 20% to

22% of the TO training could well take place in the BT33.

Overall, it appears that most students will accept the BT33 at

- least to the extent that the instructors do. The principal concern

of the students is obtaining a high score in the graded events. If • - • they are reassured that their overall grade average will be as high if -

• they use the BT33, using the findings of this study as evidence , even

more students will accept the BT33 positively.

- 

• 

g. Can the BT33 be useful in student evaluation?

- Almost half of the instructors felt that student performance in

I the BT33 could be validly evaluated, primarily with regard to procedural - ‘

• skills.

4

—‘4
rr ~ T ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- - -

.

~~

•

~~~~~~~ --— -~~~~~~~~~~~~

- •

~~

---- ~~~—•—~~~~ -~~ :—

- - ___ 

~~~~~~~~~~~~~ , 

ñi

~~~~ -

APPENDIX A

QUESTIONNAIRES

• USAFAS BT33 TRAINING EFFECTIVENESS STUDY

STUDENT EVALUATION -

FAOBC CLASSES

- FOR ADF

USE ONLY

The USAFAS , with the assistance of the Army Research 1. _____

Institute, is evaluating the utility of the BT33 Fire Con- 2. _____

• trol Simulator for Forward Observer training. It is a 3. - Swedish training device, built by Saab-Scania. It is com- 4. _____

• parable in many ways to the Army ’ s own Observed Fire 5. _____

- Trainer, which is now under development. 6. _____

7. _____

As an aid in evaluating the impact of this training 8. _____

device on the student and how it might best be used, 9. _____

I please fill out this cover sheet end the attached question— 10. _____

naire as completely as you can. Your answers will be used 11.

for research purposes only and will not become a part of 12.

your record. When identifiers (name and Social Security 13. _____

~~ Account Number) are requested they are to be used for ad- 14. _____

ministrative and statistical control purposes only. Full 15.

I confidentiality of the responses will be maintained in 16. _____ 

-

the processing of these data. 17. _____

18. _____

Your participation in this research is strictly 19. _____ - •

voluntary. Individuals are encouraged to provide complete 20. _____

and accurate information in the interests of this equip- 21. _____

• ment evaluation, but there will be no effect on m di— 22. _____

- viduals for not providing all or any part of the infor- 23. _____

mation. In keeping with the Privacy Act of 1974, we 24. _____

• request that you sign your name at the bottom of the 25. _____

cover sheet indicating that you have not objected to 26. _____

• completing this form. 27. _____

- 28. _____

• CLASS (e.g. , FAOBC 9—76) _______ 

SECTION _______ 29. _____

30.

- DATE __________________ 31. _____

• 32. ____

NAME _______________________________________________ 33. _____

Last First MI 34. _____

• 35. _____

1. Time as a commissioned officer (months) ____________ 36. _____

2. Source of commission (check one): BIOCC ______ 

USZ4A ______

• SIGNATURE

(~•i 45

4- -c’__________ __ 

___

~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-, 

— 

~~

*

~~~

- — — 

-.-~------ -- ~~~~~~~~ -

~~~~~~~~~~~~ . — — — -- — ‘-* ~~~~-,-.~

---

~~ ~~~~~~~~~~~~~~~

*

I .

In several of the questions you will ~e asked to evaluate some - characteristic of the BT33 through use of a rating scale. Please

regard these scales as being continuous. Place an UXN at whatever location on the scale best describes your opinion. - i

-i

•

l. Overall, how effecti-

~

-e do you feel the BT33 is as a training

device?

I I —I

Very Moderately Very

Ineffective Effective Effective

- 2 • Performance of the Forward Observer’ s job involves the use of

procedural, co~~ inications, and visual/perceptual skills. In

• answering the following questions, think about what you learned

and the manner of performance of each of these in the BT33 training environment vs. out on the range during live firing exercises.

- 

a. How effective do you feel the BT33 is for training each of

- 

these skills:

Procedural?

• I I I I

Very Moderately Very

Ineffective Effective Effective

• Comesinications?

I I -I

Very Moderately Very -

•

Ineffective Effective Effective - -

• Visual/Perceptual? *

I I -I

Very Moderately Very

Ineffective Effective Effective

1*

46

_____________________ 

~~—
• •-

~~

- -

~r - - -

~ _____________ _________ 

-

-

1

~~~~

- • b • Were there any differences in the manner of your performance

• in the BI~

33 vs. range environments which caused you difficulty when performing on the range?

Procedural differences? Yes _____

- If the above answer was “Yes,” please describe the differences and I

- indicate whether the difficulty you experienced was very short-

• term or lasted for some longer period of time

• Communications differences? Yes - -

If the above answer was “Yes,” please describe the differences and •

I indicate whether the difficulty you experienced was very shortterm or lasted for some longer period of time.

Visual/Perceptual differences? Yes _____

H No __

If the above answer was “Yes,” please describe the differences and - 

I

~~ 

indicate whether the difficulty you experienced was very shortL term or lasted for some longer period of time.

1

‘ 1

-

~~~~~~~~~~~ 

- I

•1~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
-~~~~ 

- 

~~~~~~~~~

-

~~~

- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ U-- 

-

______

! i - i

c. If any of the answers in 2b above were “Yes ,” please discuss

what changes might be made in the use of the BT33 so as to

i -. resolve the problem.

- 

Procedural differences:

.•1

Communications differences:

Visual/Perceptual differences:

• ~

!-. 48

-

~~~~~~~~;-~~~;•~.: 

~~~~~~~~~~~ ~~~~~-.- I

~~M~~~
_____ —*-

~~~~~~~~.-.• ‘.—-- ~--- -* ~~~• ~~~• • ~~~~~~~~~ “

~~~~~~~~~~~~ I -•

- - - - ______________

— -•- -- •-

~

• • ••. • . •

~~~~~~

— -~~ —

( 3. Overall, how do the BT33 and Puff Board training devices compare

to a live fire training exercise for training effectiveness?

(If you were not exposed to the Puff Board please skip that part

• of the evaluation.)

Puff Board :

I I —

~

Much Less Same More More

Effective Effective

BT33 Training Device:

I

Much Less Same Much More

Effective Effective

~

4. Now more specifically, how does the BT33 compare to live fire cx—

ercises for training effectiveness at different points in training?

a. The very first practice session (known as the “Puffboard ” exer— 

cisc). How does the BT33 compare to a live fire exercise for

training effectiveness on this session? 

I I -I

Much Less Same Much More

Effective Effective

b. Daly Hill exercise . (Compare the BT33 to the Daly Hill situation for training-effectiveness.)

I -I— I I I

Much Less Same Much More

Eff ective Effective

• c. Arbuckle Hill exercise.

I I

. I I

Much Less Same Much More

• Effective Effective

.i

~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -

• ~~~~~~
-

~

• d. Bunker Hill Shoot. j

• I I

Much Less Same Much More

Effective Eff.ctiv .

• Do you think a realistic evaluation of your skills could be made -~ 

- in the BT33 rather than at Bunker Hill? Yes ____

No _

e. Mobile Shoot ~ ie.

I I I

Much Less Same Much More

Effective Effective

- 

- 

f. Mobile Shoot Two.

I

Much Less Same Much More

Effective Effective

g. • Are there any other FAOBC field exercises where the BT33

might be an effective substitute?

- I 5 • Do you have any recommendations for improvement of BT33 training

• effectiveness with respect to any of the following?

a. Terrain, target, or burst presentations?

• 

I

Ii

1

I

50

V

I 

~ — - --•—--~ •~~~ • , ; ‘

~

- • • • • ,_~ • •ç —~ -

I ~~~~
____• ~~~~~~~~~~~~~~~~~~~~~~~ 

- - 

• c

~~~~-•-.~ ~~~~~~~~~~~~~~~~~~~~~~~

- 

b . The manner in which the BT33 is used by the instructors?

H

• c. Anything else?

.1 •

6. Would you like to have the BT33 available to you for additional

practice outside of the presently scheduled class training • •

sessions?

I I I I I

I Would Moderately Would

• Dislike Like Like

- Very VPry

Much -

~i’h

I

H

- 51

~

t

~i 

L _________ ________________________

~I —-- — 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
— 

~~~ 

~~~~~~~~~~~~~~~~~~~~ ~~~- —

USAPAS BT33 TRAINING EFFECTIVENESS STUDY

STUDENT EVALUATION -

FAOAC, MASOC, kIT AND III CORPS AI

~ ILLERY -

VOR ADP (

USE ONLY

The USAPAS , with the assistance of the Army Research

• Institute, is evaluating the- utility of the BT33 Fire Con- 1. _____

trol Simulator for Forward Observer training. It is a 2. _____

• Swedish training device, built by Saab-Scania . It is corn- 3.

• parable in many ways to the Army ’s own Observed Fire 4. ____

Trainer, which is now under development. 5. _____

6. _____

As an aid in evaluating the impact of this training de- 7. ____

vice on the student and how it might best be used, please 8. _____ j -

fill out this cover sheet and the attached questionnaire as 9. _____

completely as you can. Your answers will be used for re- 10. _____

search purposes only and will not become a part of your 11. _____

record. When identifiers (name and Social Security Account 12. _____

Number) are requested they are to be used for administrative 13. _____ 1 •t

and statistical control purposes only. Full confidentiality 14. _____

of the responses will be maintained in the processing of 15. _____

these data. 16. _____

17. _____

Your participation in this research is strictly volun— 18. _____

tary. Individuals are encouraged to provide complete and 19. _____

accurate information in the interests of this equipment 20. _____

evaluation, but there will be no effect on individuals for 21. _____

not providing all or any part of the information. In keep- 22. _____

ing with the Privacy Act of 1974, we request that you sign 23. _____

your name at the bottom of the second cover sheet indicating 24. _____

that you have not objected to completing this form. 25. _____

26. _____ ~~ :1

27.

• CLASS (e.g., FAOBC 9 76) ____ 

SECTION ____ 

28. ____

29. _____

DATE __________ 30. ____

31. _____

NAME _______________________________ 

SSAN ________________ 

32. •

Last First MI 33. _____

34. -

35• —

36. ___

14

,

52

• --•- •-

~ • ,- • - ‘ I - ~~~~~ — *~~~~~~—--~~ • - --,--• -

~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~
,‘ ~

__c~~~~ . 

~~~~~~~~~~~~~~~~~~~~~~ — ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ‘-~r-•-—~~ - - -

~~~

- —_ _--—

1. Officer __________ Enlisted __________ 

(check one) • ~~~~~~~ - - —

- 2. Rank • —

-- • 3. Time in the military _____months

-- - 4. Prior enlisted service ______mont hs

5. Source of commission (check one if applicable): BIOCC ____ 

USM? ____

ROTC ____

6. Have you ever served as an FO? Yas _ No _

How long ______months In combat ? Yes_ No —

- 

- 7. How many missions have you fired since OBC/OCS? (estimate) _________

- 

• 8. Which program are you currently enro1l

~d in? (check one) 

•

FAOAC ______ MASOC ______ 

kIT ______ 

III Corps Artillery ______

NOTE : PLEASE RESPOND TO THIS QUESTIONNAIRE REGARDING THE BT33 ONLY

IN TEPIAS OF ITS POTENTIAL USE WITHIN THE PROGRAM YOU ARE I - -

CURRENTLY IN.

SIGNATURE ___________________________________

!4 . 

- 

-

______ -• 

— 

53 

— 

—
____ ~~~~~~~~~~~~~~~~~~~ ~ T i

~ ~~~~~~~~~~~~~~~~ ~~~~~“

~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~

-

~~ 

— t - In several of the questions you Will be asked to evaluate some

characteristic of the BT33 through use of a rating scale. Please • - regard these scales as being continuous. Place an N XSS at what- ~~~ 

•

-

‘ ever location on the scale best describes your opinion. H

• • -

1. Overall, how effective do you feel the BT33 is as a training - - device?

— 

I I I I -

Very Moderately Very

Ineffective Effective Effective • 

-

- 2. Performance of the Forward Observer ’s Job involves the use of pro—

- - cedural, c

~~~ inications, and visual/perceptual skills. In answer- -

~~~~~

ing the following questions , think about what you learned and the

manner of performance of each of these in the 8T33 training environment vs. out on the range during live firing exercises.

a. How effective do you feel the BT33 is for traiz4ng each of I these skills?

Procedural?

I I I I

1 Very Moderately Very

• I Ineffective Effective Effective

- co unications?

I I I I - 

Very Moderately Very • _ Ineffective Effective Effective

I Visual/Perceptual?

Very Moderately Very - - Ineffective Effective Effective

!

~ ~~~ • 

-

: 4 :

:

;

~ ~ i.~’,

it 

~ ~ ~

%f2~

- ~

r ~~~~~~ ~~~ Y~ ~~~~ ‘

~ 

“ Z ~~~ ~~~ ~ -- - -

~~~~~~~. ~~~~~~

- -. 

~

r L ~~-

~~~~~~~~JJt~~- . ~
_____________________________________________________________________________ - 

~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~

• 

~~~~~~

• ,

~~~~~I~~__.___ —

b. Were there any differences in the manner of your performance

in the BT33 vs. range environments which caused you difficulty

when performing on the range?

Procedural differences? Yes ____

No __

If the above answer was “Yes ,” please describe the differences and

indicate whether the difficulty you experienced was very shortterm or lasted for some longer period of time.

• Conununications differences? Yes ____

No ___

If the above answer was “Yes,” please describe the differences and

indicate whether the difficulty you experienced was very shortterm or lasted for some longer period of time. -

J

Visual/Perceptual differences? Yes

No _

If the above answer was “Yes,” please describe the differences and

• indicate whether the difficulty you experienced was very short- - •

term or lasted for some longer period of time.

L I -

55

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —
- • 

- -

~~~~

- -- ~~~

- 

~~~~~~~~~~~

‘ .- -

c. If any of the answers in 2b above were “Yes,” please discuss

what changes might be made in the use of the BT33 so as to

resolve the problem.

Procedural differences:

communications differences:

Visual/Perceptual differences:

:

56

L.

~ ~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~ • • - • • , ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • ‘

~ _1_~

I-

~

=-

~ _ -~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~
~~~~- ~~~~~~~ ___________ - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ • -~~~~~~~~-,

~~~~~.~~~~~--•---‘----,- - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -~~~

3. Overall, how do the BT33 and Puff Board training devicea compare

to a live fire training exercise for training effectiveness?

(If you were not exposed to the Puff Board please skip that part

of the evaluation.)

Puff Board:

I - I -- —I -I

Much Less Same Much More

Effective Effective

ET33 Training Device:

I— — I

Much Less Same Much More

• Effective - Effective

4. Now more specifically, how does the BT33 compare to live fire cxercises for training effectiveness at different points in training?

I And in different training proqrams (e.g., FAOAC vs. kIT)? Make

evaluations for all of those programs in which you have actually

participated as well as the one you are currently in.

PAOAC a. How does the BT33 compare to live fire exercises for

training effectiveness within the FAOAC program?

(Answer only if you are or have been in the FAOAC.)

4 14

I I I I —I

- i

~

- Much Less Same Much More

Effective Effective

57

• .

~~— ~~~ --~ - - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
-n - • • - -

~~~ —,~

- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —‘~~

FAOAC b. How often and in what manner do you think the BT33

might be best used in the FAOAC?

kIT c. How does the BT33 compare to live fire exercises for

training effectiveness in the AlT program? (Answer .

only if you are or have been in kIT.)

I I I I

Much Less Same Much More

Effective Effective

kIT d. How often and in what manner do you think the BT33 ( - might be best used in the kIT program?

-‘ ~ k

MASOC e. How does the BT33 compare to live fire exercises for

training effectiveness in the MASOC program? (Answer

only if you are in MASOC.)

I I I I I

Much Less Same Much More

Effective Effective

MASOC f. How often and in what manner do you think the BT33

might be best used in the MASOC program?

•

58

1• - 

-

~~~ . :. ~~~~~~~~~~~ —. ~~~~~~~
_

~~ _,._~,,.S_.__•_~s • - - - . ~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -~ •

III Corps —

- Artillery g. How does the BT33 compare to live fire exercises

for refresher training effectiveness? (Answer

• only if you are a member of the III Corps

Artillery.)

- 

I I 1

Much Less Same Much More

- • Effective Effective

- III Corps - Artillery h. How often and in what manner do you think the

BT33 might be best used for refresher training?

5. Do you have any recommendations for improvement of 8T33 training

effectiveness with respect to any of the following?

a. Terrain , target , or burst presentations?

I

~

H

b. The manner in which the BT33 is used by the instructors? - 

-

- i 59

t

~

;) ___ 

_____________ 

- -3 —- 

~~~~~~~~~ ~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

_~~~~ • -

~~-- ~~~~~~~~~~~~~~~~~ -
-

~~-~~~—~~~ 

~~~~ ~~~ ~~~~~~~ T~~~W~~ ~‘

~~~‘~

I

___ _____ 

____ —- -- - - — -— --—

____- - -• - ______ -- 

I

c. Anything else?

6. Would you like to have the 8T33 available to you for additional

— practice outside of the presently scheduled class training sessions? . 

• 

-

I I I - Wou]

~

1 Moderately Would f •

Dislike Like Like ‘

1 ~~ Very Very - 

- Much Much :~ 

60

~a ~~~~~~~~~~~~~~~~~ 

-

~~~~~~__

:_ . __ -~~~.- ~-~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I

~~~~~~k
- ~

.T • 

- - 

~

r - • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -r -

~

-• -—- • -----•

~~~ 

— -- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- — - - ----• - • - ----—--.—-—--—-•-- - - --- --—•-—--- --- --- •—-

~~~

—--•- -—-•- -“-• • - •

-i - USAFAS BT33 TRAINING EFFECTIV~~ESS STUDY

INSTRUCTOR EVALUATION

FOR ADP

USE ONLY

The USAFAS, with the assistance of the Army Research 1.. _____

Institute (ARI), is evaluating the utility of the BT33 Fire 2. _____

• Control Simulator for Forward Observer training. It is a 3. _____

( - Swedish training device , buij .t by SAAB-Scania . It is com- 4. _____

parable in many ways to the Army’s own Observed Fire 5. _____

Trainer, which is now under development. 6. _____

7.

As an aid in evaluating the characteristics and utility 8.

of this training device as an instructional tool , please 9. _____ 

- -

fill out this cover sheet and the attached questionnaire as 1.0. _____

completely as you can. Your answers will be used for re- 11. _____

r search purposes only and will not become a part of your 12. _____

record. When identifiers (name and Social Security Pccount 13. _____

Number) are requested they are to be used for adinini -tra— 14. _____

tive and statistical control purposes only. FuLl 15. _____

dentiality of the responses will be maintained in ti~ 16. _____

processing of these data. 1.7. _____

18. __

Your participation in this research is strictly v an- 19. _____

tary. Individuals are encouraged to provide complete and 20. _____

accurate information in the interests of this equipment 21. _____

evaluation , but there will be no effect on individuals 22. _____

- - for not providing all or any part of the information. In 23. _____

keeping with the Privacy Act of 1974, we request that you 24. _____

sign your name at the bottom of the second cover sheet 25. _____

indicating that you have not objected to completing this 26. _____

form. 27. _____

- 28. _____

29. _____

NAME ______________

Last First MI

DATE ___________________ 

SSAN __________________

4 - - 

INSTRUCTOR POOL MEMBERSHIP : ____ 

FAOBC ____ 

FAOAC

NOTE: Please respond to this questionnaire only in terms ~~~

- .

of your own current teaching background (i.e.,

FAOBC or FAOAC).

61

f.SL —— - - - 

~~~~~~~~~ ;:

~~

-

~~~~~~-~~~~~

--

~ • • - - - • _
• ~~~~~~~~ _______ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •

~~~~~~~~~~~~~ ,

~~~- --,-~~ -~~~ ---- .- 

~~~w —.~-,--.-, —V -

~~

w.’ w ~

-

~~~~.--.~~~~-,.-,-—----- —---

_____ ______ • ______- •-

~~

1. Time as a commissioned officer; _____Months

- 

- 2. Source of commission (check one) : BIOCC ______ 

USMA ______

ROTC __

3. Prior enlisted service? ______Months

4. Have you completed the FAOAC? _____Yes _____

No

5. Have you served asapO? - 

- •

How long? ______Months In combat? ______Yes ______No

6. How many missions have you fired since OBC/OCS (estimate)? _____

7. How long have you been an instructor? ______

Months

8. How many classes have you handled? ______

SIGNATURE _______________________________

:1:

4 V

62 

— n _____________

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~
— — -

~~~~~~~~~~~~~~~~

----

~

----

~~~~

-- 

~~~~~

In several of the questions you will be asked to evaluate some

- ! characteristic of the BT33 through use of a rating scale. Please

- 

‘- regard these scales as being continuous. P1a~e an “x” at what- — ever location on the scale best describes your opinion.

- - 1. Overall, how effective do you feel the BT33 is as a training device?

- 

I

Very Moderately Very

Ineffective Effective Effective

- 2. For FAOBC ____ 

(check one) training, what percent of the

FAOAC ____

- 

practice exercises do you think could take place in the BT33?

I I I a

0% 50% 100%

3. For FAOBC ____ 

(check one) training, when in the training schedule

-i FAOAC ___

• do you think the BT33 could best be used (e.g., for the first,

thi~~ and fifth practice exercises, or 

4. Compare the training effectiveness of the 8T33 environment vs. out - 

on the range. How would you evaluate the effectiveness of the

8T33, as compared to a live fire exercise, with respect to the :- . - -

following parameters and conditions? •

-

k

- ’ J a. How well the student learns his skills?

- —i

Much Less Same Much More

- ,

-

~ Effective Effective

63

- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ — —
-- - - 

:_~~~~~~

_

~ ‘-~~~~~~-

~~~~~ 

- -— - 

~~~~~~~~~~~~~~~~~ ~~~~~~~~~

- - - . - 

~~~

- -

~~~~-: ~~~~~~~~ -.

-

- b. How quickly th. student learns his skills?

I

I I I I

Much Less Same Much More

Effective Effective

c. For ____ 

FAOEC (check one) training? - FAOAC

I I I I

:

~ Much Less Same Much More

Effective Effective

d. For Al? training? (Respond only if you have instructed an AlT

class.)

1 

- 

I I I

Much Less Same Much More

I Effective Effective

- e. For MASOC training? (Respond only if you have instructed an

MASOC class.) :

~

I— I

Much Less Same Much More

Effective Effective

--

• 

- 

-k’ f • For Forward Observer training in the field?

- H I I I

- - 

- • -

~~ Much Less Same Much More

Effective Effective

— 5 • Do you think any student performance evaluation could be done in

the BT33? Yes ____ No ____

If your answer is “Yes,” when do you think this should take place?

64

- t, ~~

-,
• -- - - —~~~--— — •w~~•

~~~• ~~~~~~~~~~~~~~~~~~~~~~ -

-~~ 

-

~~ 

~~~~~~~~~~~~

6. As compared to the live fire exercise situation, how well does

the BT33 environment facilitate instructor-student interaction?

I I I— I I

Very Same Very - Poorly Well

7. As compared to the live fire exercise situation, how well does

the BT33 environment enable you to identify the nature of student

performance problems and misunderstandings?

Very Same Very

- 

- 

Poorly Well —

8. How adequately can you manage the BT33 display presentations?

I —i

- - 1 - Very Moderately Very

Inadequately Well Adequately

Do you have any specific problems in managing the presentat ions?

Yes ____ 

No

If your answer was “Yes,” please describe the problems:

1’

65

- 

~~~~~~~~~~~ ~~~~~~~~~~~~ ~.— 4-.-., ~~~~~~~~~~~~~~~~~~~~ ~~~~—.—.- .——--.— -- —-

— . 

~~~~~~~ 

- —

~~~~~~

-=~~

-

~~~

-~
______ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~-

‘

~~~~ 

~ 

-

~~

-‘

~~~~

‘ 

~~~~~~~~~~~~~~~~~~~ ~~~

‘

~~~~~~~~

‘

______________________________ —~~ — —— •

~• • •!

• -

~~

- U::

9. Consider the fidelity of BT33 display presentations and the van - - ety of situations you can create. Please make a judgment of - - device adequacy and identify any needs for improvement, which,

- 

if made , would improve the device’s training effectiveness.

DISPLAY ITEM ADEQUATE INADEQUATE NEED FOR IMP1~

)VEME~~ - 

TARGETS ______ _______ ________________

- (Check one)

~~~~~~~~~~~~~~~~~~1

-j

TERRAIN ______ _______

B~RST~ ____ ______ _____________

OTHER: ______ _____ _______

~ 

4 - 

10. Do you have any other recommendations for improvement of either the

BT33 itself or the manner in which it is used?

--

I

• : f -

46

______ ——
- 

- 

- 

I — - -— 

- - 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - • -

,

- I

USAFAS BT33 TRAINING EFFECTIVENESS STUDY

OPERATOR EVALUATION

The USAFAS, with the assistance of the Army Research Institute

(ABI) , is evaluating the utility of the BT33 Fire Control Simulator

for Forward Observer training; It is a Swedish training device,

beilt by Saab—Scania. It is comparable in many ways to the Army’s

own Observed Fire Trainer, which is now under development.

As an aid in evaluating the characteristics and utility of this

training device as an instructional tool, please fill out this cover

sheet and the attached questionnaire as completely as you can. Your

answers will, be used for research purposes only and will not become

a part of your record. When identifiers (name and Social Security

Account Number) are requested they are to be used for administrative

and statistical control purposes only. Full conf identiality of the

responses will be maintained in the processing of these data.

Your participation in this research is strictly voluntary. Individuals are encouraged to provide complete and accurate information

in the interests of this equipment evaluation, but there will be no

effect on individuals for not providing all or any part of the information. In keeping with the Privacy Act of 1974, we request that you

sign your name at the bottom of the cover sheet indicating that you

have not objected to completing this form.

NAME

Last First MI

DATE ____________________ SSAN _____________________

MOS ______________ 

TIME IN THIS MOS ____________Months

GRADE ____________ 

TIME IN SERVICE _____________Months

SIGNATURE _________________________________

~

- 

- 

- - 

- 

- 

~~~ 67 

~~~~~~~~~~~~~~~~~~~~~~~~~ ~ -

~~

-- -~~ ~~~~~~~ -- - --- ~~~~~~—-~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- - - 

~~
-

~~~ 

- ~~~~~~~~~~~~~ • 1

~~~~F W~~~~~~~W~- -•:---- - —~~~~—-- -

~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ _

~— w--~~ ~~~~~~~~~~~~~~~~~~~~~~~~ •

. ~~~~~~~~~~

• 

-—-—-—— -—- — — •-- -— — •— .— ••

• • •• --•— • — • • • • • — •- • — • - • - -• - • —

Many of the questions will ask that you rate the BT33 on some - characteristic through use of a rating scale. Please consider

- the scales to be continuous ones. Place an “x” on the scale

at whatever location best describes your opinion.

- - 1. Compared to other similar devices you have worked with, how easy - is the BT33 to operate overall?

I I I I

- Very Moderate Very - •

Difficult Easy

- 

2. Is it easy or difficult to make errors in the operation of the

BT33? I.e., how precise and careful do you always have to be

- - - • - in order to avoid making errors?

- I I I I -

‘ - Very Moderate Very

Easy to Difficult to - 

• Make Errors Make Errors

-

• I If any operational errors can be easily made, would you please

describe what these are?

I • - 3. Is it easy or difficult to avoid damage to the equipment when - 

• operating the BT33?

I I I 1

Very Moderate Very

- 

Difficult Easy

- If any items of BT33 equipment can be rather easily damaged dur- - I ing operations, would you please indicate what these are?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ 

~

• 4 ~-

~~~~• • ~~~~~~~~~~~~~~ ~
I—1—~~~~~ .-

~~~~-- ~~~~~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~

4. Are there any other specific problems in the layout or operation

of the device? - 

- 

-•

Yes _____

No

-

~~ 

- 

If your answer was “Yes,” please describe these problems.

II

• 5. In the selection of personnel to become BT33 operators , how much

electronic equipment background experience do you think they

should have?

H I I

Very Moderate A Great

Little Deal

Would you please describe the desirable background for a 5T33

operator in terms of MOS, amount of MOS experience, grade, etc.?

6. In the selection of personnel to become BT33 operators, how much

forward observer background experience do you think they should

have?

I I I I

Very Moderate A Great

Little Deal

Would you please describe the desirable TO background in terms of

• programs (e.g., Al?) and operational time?

69

__________ — ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ : ~~~~ ____
-~~ 

~~~~~~~ ~~~~~ ~ -

~~~~ w-’

~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

-, 

~~~~~~~~~~~~~~ 

I

______________ • •-~--- - • - -• - ---- - -— -•-- - ----— -— - - ---- - 

I

- I

~~~~

-

7. How many days of classroom and on-the—job training do you feel a

person would have to have before he would be a qualified BT33

operator?

- 

Classroom __________days

OJT _______________days

8. How frequently do you feel a person would have to perform as a

BT33 operator to maintain his operator proficiency?

F— I I

Once a Once a Every

Month Week Day

- Please describe the above frequency in words (e.g., one week out

- 

of four , once a week, etc.).

H

9. From an operator’s standpoint, do you have any reconunendations for

the improvement of the BT33?

F

I

-

~~~ Ii • -~

- 

I -

~

-

~ - g~~~j 70

~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-

~~~~ I’~ - 

-

~ ~~~~ - • -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ — ~~~~~~~ ~~~~~~~~~~~~~~~~~ - ~ - ~~~~~~~~~~~~~~~~~~~~~~~~~ U~~~~ iL )W

-— - - - - - ‘f l - ~~~~~~~~~~~~~~~~~~~~ - -- -

APPENDIX B

IMMEDIATE SUPPRESSION-GRID COORDINATES DATA SUBGROUP

0 In N U) U) 0 C’I e In

C) 3 N i-I CM N i—I CM N N

O 0 14

U) ri

E4 4) ~~‘ ‘-I ~~‘ 01 U) 0 U) 0

0 (i) 0 0 N C’) N r.4 r-I C’) r4 N

0 ~i .rC C)

O s 1.1 .)4

4) 0 C) ~~ N N ~‘ N r-4 ~~‘ CM In

41 4)

00 •rI ~ 

C) C’) U) U) C’) In r- ‘

.0

0 0 CM N N C’) N ~~‘ N N

00

4)0 ‘-4

‘-‘ 4) — .08 o’~ ~

:

.c 0 U) N 0 C”) In ~

D ‘

.0

p4 (1) N p4 p’4 p4 C’)

WI ___

4) Q -’.4 = _____ 

p4

= = = -

= =• = = = i-I U ) C) r-I 0” C’) In 0 ‘0 U) i-I 0 .0 0) C’) CM CM ~~ N C’)

.d ~~~0 Ei 0 14 0)

a) ‘0 (i) ~

. C)

N C’) N ~‘ CM ~~‘ C’) In 4) ~ 14 ‘-4 0)0 4

Z’d ‘

~~.

‘0 C V 4) 4) —4 C Os a) ‘-4 0 N I ‘

.0 i-I ~~‘ 0

4 ) 5 1-I p4 ~‘ p’4 C’) 4’ ‘0 C’) 14 -.4 I I

.0

44 C 4)14 In p4 I CM I p4 CM In

‘ 40 —

0114 •

30 4)

0) ‘.4 11444 4) 0 4) ‘-I

Z ‘0 4)

a . C)

‘-I .

~~~ ‘4

+ ‘C I 

—--- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - 1 —-.±~~~ ~~~ ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~ -

~~~~~ 

- , . 

-
- — - ~~~~~~~—~~~~~ •

------- 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

-- - 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~

- ~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ L

• ~~~~ 0 C’) . ‘0 CM p4 U) ‘.0 i-I (1) p4 - N

* 0) —

• 0 4)

C) 4)

0 0 0 0 CM CM C’) p4 i-I CM 0 (I) ~ 0 ‘0 0 ‘0 N 4’ CM N N 4) C’) C’) C’) N N ~~‘ p4 p4

4

0)

) 6 

14 0 14 . H

.,4 al —4 4) 14.0 14 .

~~

-~~~ ~ N p4 C’) CM p4 ~~‘ CM In

o ‘.4 1-- “

444 1-4

0 ) 41 C: — —

0 In 0 CM 0 ~~‘ ~~ ~~‘ p4 01 g ‘0 N ‘.0 CM U) ‘-4 ‘0 ‘0 p 4 4) ~

)0 CM CM CM If) CM CM N N

U) C’) In ‘.0

=_

N C’) N ~~‘ CM 4’ C’) CM

~~

• 1 4~ p4

14 ‘.4 4)

0 44 ‘-I N 0 I p4 0 CM C’) ~) 0)4 ) 14 p4 C’) N U) U) 0 i-I U) 14 ‘i-I CM p4 I p4 ~~‘ C’) CM N - - • ‘.4 p4 4 1’Ø 00 >1 00) ‘-4 I

C

_T T 1 T TT1

~~~~~~~~~~~~

‘C 

_ 

4

72

_____ - - - ---~~~~ -- - --~~~~~~~~~~~~

b. a ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~ L ~~~~~~~~~~ 

- ‘•

~~~~~~~~~ •• - - -.
—

I

__ •-- • --

~~~~

•-------

~~~~~~~~~~~~~~~~~

- -_

- 0 0’. 0 It) p4 0 CM U) an

- 3 0) U) C’) ~‘ 0 p-I 0 U)

0 14 p4 ~~‘ CM N I’) 4’) N

‘—4

0 4’) ‘.0 CM ‘-I U) ‘0 p4

p4 p-I C’I

-- Id

• / -i-I I 4) 0’ _____ —

— - I • -f-I 4~

J

U) In 0’. 0 0 ‘.0 r’4 U)

4) > 0 ‘-4 In ~~‘ In 0 U) N N

9444 0 ~~ p-I N C’) ~ I CM CM .—I - 0 -ri

:)hI

’g ~~ Ii

- 0

~~ -rI 4)

I ~~~~ ~

i ‘

~~ N N ~~ CM p4 ~~ N If)

- -I g

~~~~~~~~~~~ _—————_— - -

4) CM ~~‘ U) N U) ~‘ C”) C”)

CI N N in C” ) 0’. 0) - -

•

00 C”) N CM C’) CM C’) ,-4 p4

u—I -

j ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

— ___ _____ 

___ =

4’) 0 )4 )

In N 4’ ‘—I ~~ ‘.0 N N

C’) 0 U) ‘0 It) 0 ‘.0 0’)

4) Z4 1 0 .-I in 4’ CM ‘.0 i-I 0 CM

p4 N • 1 .0 ‘0 01 0 Id 4) ‘-I

14 ‘414

Z

• 0 44 ____ 

44 r4 N 0 1 0 0 ~ I ~ 4 C’)

C))~~~~ 0) ~~I 0’. 0 0 N N 0

~~~0 I .-I It) ‘0

It) ~ I I CM ~4 ~ l CM it) -

-

O 4 ~ 

— 

0)

- ‘C 8 -

~~~~~~~~

A’ I g

~~~

uu

~ 

it

u-I CM 4’) 4’ In ‘.0 N U) h - ~

4 * * * * * * * * C

i 

_ _ 

73

— -: 

- 

‘

~~~~~~~~~~~~ - ‘~~~~~~~~~~~ J

~~~~~~~~~ _. -~~~~~~~~
• - ~~ - —-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~

- ‘ — -

- — 

~~~~~~~~~~~~~~~~~~ ----, *r--~~

r •- -

____

9 ‘.0 C”) It) It) 0 i-I ‘.0 0 S S • • • S • S

4)14 p4 p4 CM N m CM CM CM p4

,

~ 0 4’) ‘

.0 CM .-I U) ‘.0 p4 U) p4 CM

0) ___ 

4)

— — — 14 

~~ 8 0 ~I C’)) v-I CM CM CM CM CM

—I

N CM ~ ‘ CM p4 ~~‘ CM In

0104 4) p4 U) C’) U) 0 4) ‘0 CM CM 0. 5 . . S S S • 4 ) 0 CM p4 (9 CM CM CM N CM

4’) O’a N ‘-I C’) In ‘.0 F4’ 

L 

‘0 —

~~~H 14 u--C 01 4

~ 4) 14 ‘-4 0’. 

. 

m CM 0 0 N m a’.

4) • • . • • . • ~~~~~~~ p-I CM 4’) 4’) ~‘ C’) in p4

0 04 ) ~ -~~ 4) 114 14 4) C)

_

0 14 p4 0 0 ) 0 0 0 0 ‘.0

-

~~~~~~ -u Ii ~~ -~~~~~~~

* 

‘ ~

1’ - 74

- -- — ---“ ~~~~~~~~~~_ _

—

~~ I ______________ - 

~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- ---- ----.-— -

-- ------.~~-

-- ~~~~ 

~~~~~~~~~~~~~ I

~~~~~~ I~aI •-

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I: - L - — ~~~~~~~~~~~~~~~~~~~ -- — ~

- - —. - —

9 ~‘ C’) N 0 0 In N In

3 . S S S • S S

01. p4 CM N 0 N 0 ‘.0 p4 p4 p4

‘ I 0 C’) ‘.0 N u--I U) ‘0 p4 (4) p4 N

• 4) 01 ____ 

- —

p4 ~~ 4)

14 ~ g ‘0 0 0 ~~‘ 0 0 it) U)

,

~~~~~~~ 4)~~~~~~

0 ) -u-i 4)

~ g N CM ~~‘ N p4 ~~ CM in -p4 4)

4) 14) -u-I

-

~~~~~ U C) O rnCM0)o ~~~~~ 01C~ 08 ~ ~ .4 0 u-I p4 p4

114) 04 ‘14)

8 - (1)-u-I 

~~ C’) 0) N p4 4’) In ‘.0 N

‘0 U) CM p4 p4 p4 C’)

~~ = __ ===~~~~~~~

==== ~~ ~~~~~~~~~ p4 N N It) ~~

‘ 0 0 0 p4

4)~~~~~~~~ 4 J I ~ -

~~~~~~~ .-4~~~~~~~~~~N f - 0’. r n r n N

p4 CM p4

.0 0)140 44) 0

Id Z O O) 4) u-I 14 

4J ~ 

.14

N C’) ~~ It) C’) ~~ C’) N

-

~~ ~~~ 

— 

4)4) 14 u-I 0 0 I 0 In 0 0 0

14 1) 1.4 p4 5 5 5 • S S - -

00 I-’ -u-I ~ 0J 0 U) In ‘.0 U) CM p4 p4

.1-) )‘

~ I

Q In u-I IN CM u-I CM it)

01 ______

P 

— ___ — — 

‘C I

~~~~~ _~~~~~~~~~~ .~~~_ _--- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .a —~~~~~~ 

~~~~~~~~~~~~~~~~~~~~~~~
-

~~~~~~ 

~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~ ~~~~~~~~~ :

~~~~~~~~~-

_ _ - -

~__ - -

- --- -

~~

- --

-1

In CM In In 0 N 0) 0’.

d .4 

~ ~~ 0 N 0 .4 p4 p4 N CM p4 ~I N p4

- - 0 I~

) ‘.0 CM p4 U) ‘.0 ‘-4 tO N

• l 

0 In U) It) 0 In 0 ~~‘ 5—4 ~~ 0 . 5 5 • S S 5 5

~

P

J 

4) ~‘ U) U) 1’ In CM 0 CM

p4 p4

(4)L00 -.4 4 3 -• . 00 -

~~~~~~~ • CM 0’) N p4 4’) Il’) ‘.0 N 4) -u-I Cl) CM u-I p4 ,—I C”)

=_

-‘I • S • - . - -

4) Z -.-4~~~ F— 6) CM 0 N 0”. 0 CM p4 4 ) $ ~~U) CM CM N CM p4 p4 ‘-4

H 

~ 

Z 14 

~1~~g - ~ ~~ 

.

~~

-

~~ 0) ‘4 p4 CM 0 I 0 It) 0 0 CM

3 03 1.) p4 5 S — • S — •

1441 14 -‘-4 In 0 1 0 CM In ‘.0 0’.

‘-4 CM CM p4

Il-I >1 0 i-4 I

14 4)

~~ In p4 I CM CM p4 N It)

H ‘C

-1 u

~ - 0 I iwu i

~~ CM CM ‘0 N 

n

U)

~~ * * * * * * * * 4) 4) 

4)

- j

76

—~~~~~ ~~~~~~~~~~~~~~~~~

-
~~~~

.- 

~ y - -

~~~

- 

~~~~~~ ~~~~~~~~~~~~~~~~ -

~~~z~~--- - :=, —:~

T~.~

w! -- • -w• — - 

~ —~~~~~-—7~~~: -

-__ 

- -- _____ - ___--

I - c

APP~~IDIX C

VALUES

- Table C—1

ESijk l ESik~ and ESk Values for Experimental

~

!oriditicn Number One (SXX)

- Performance ________________ 

Events _________________

- : measures Mobile Shoot One Bunker Shoot Mobile Shoot Two

Time to Locate 1.09 1.00 1.00 - -

Target

-

• 

Total Mission 94 .61 .96 Time

Accuracy of Initial .92 .93 1.27 - - Target Location

Number of 1.02 1.21 1.06 1 Rounds Fired

4; I Procedural Errors .91 2.12 .75 r -j Cut Score

~~~~~ 

j 

Total Cuts 1.01 2.69 1.13

I Score

T _________________ _______________ ____________ ________________

- ESjk~ k— ] .98 1.43 1.03

~~~~I 

~~ REM , k—i 1.11

-

- 

‘e-

-

I — 

~

I - -

:

~-~- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~ _______ _________ ___ - — a -
_____________ - - 

~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

- 

-, -‘---.-—~

,.-•----.-v -’, , w ~

_—,

~~~~

Table C-2

~~ijk ’ ~~j k ’ and REk Values for Experimental

Condition Number Two (XSX)

Performance _________________ 

Events _________________

measures Mobile Shoot One Bunker Shoot Mobile Shoot Two .

Time to Locate .95 .96 1.11 — • • -

Target

Total Mission 1.02 2.20 1.13 

- 

-

-

Accuracy of Initial 83 35 1 01 Target Location

Number of .90 .76 1.36 Rounds Fired

Procedural Errors 74 57 86 Cut Score

I •

:

~

Total Cuts .71 .80 1.28 Score

REjk C k=2 .86 .94 1.13

RE,~, k— 2 98

a

- f — - , 78

:

~~t~ ___ ____ 

—

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (1

~~~ ~~~~~~~~~ H1H~~ ~~~ --~~~

- ~~~~~~~ ~~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ‘ ~~~~~~~~~~~~
_____ —‘-- -- .-

~~~~~ -

~

-

~ -

~~~ ~~~~~~~~~ ~~~~~- - — —

Table C-3

PEj ik. REjk . and REk Values for Experimental

Condition Number Three (XXS)

- Performance ________________ Events ________________

a measures • 

Mobile Shoot One Bunker Shoot Mobile Shoot Two

Time to Locate 1.17 1.18 .94 Target

Total Mission 1.33 1.47 .80

Accuracy of Initial 1 01 1 34 1 05

• I Target Location S

-

- ~

- 

- 

Number of - .96 1.27 1.04 Rounds Fired

- Procedural Errors 1.00 .67 .87 - - Cut Score

• 

- 

i Total Cuts 84 1.40 1.14 - Score

REjk~ k=3 1.05 1.22 97

- REk ? k3 1.08

- I

~ 

I -

~~~~

I. - -‘

~~~~ 79

- tt~~~

-i

~k1 -

~~~~~~ ~~~~~~~~~~~ ~~ I

~~~• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~
~~‘

~ 1’-r ’~~’~ ~~~~~ 

- 

-,

~

-, — — — — — ‘I-

-1!

- Table C—4

REj ik u- REIk, and REk Values for Experimental

• 6ndition Number Four (SSX)

Performance ________________ Events ________________

- - measures Mobile Shoot One Bunker Shoot Mobile Shoot Two S

Time to Locate 1.03 .76 .94 Target

Total Mission - - —

- - Time .9 .93 1.

_____________________________ _________________________ ___________________ _________________________

Accuracy of Initial 1.21 1 14 1.64 Target Location

N~~~er of ~ 1 07 1 10 • Rounds Fired

Procedural Errors 1.13 1.85 .83 Cut Score

• - Total Cuts 

• .98 2.24 1.71 Score

- REj k~ k—4 1.06 1.33 1.22

REk~ k—4 1.20

S

4.

___ 

- 

80 

- _______________________ I _____________________________________________ ~~~~~~~~ 

~~~~~~~~~~~~~~~~~~~~

I
‘

~~~~ ~~~~~1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~

Table C-S

REjik, RE4k, and REk Values for Experimental

Condition Number Five (SXS)

Performance ________________ 

Events ________________

measures Mobile Shoot One Bunker Shoot Mobile Shoot TWO

Time to Locate 1.67 1.53 1.15

Target

( Total Mission 1.58 1.51 1.12 - 

-

Time

•

~

•

~ 

I

Accuracy of Initial 52 1 78 1 61

Target Location •

1

Number of - .84 .8]. 1.06 -

Rounds Fired

Procedural Errors 57 .84 1.42

Cut Score

Tota l Cuts .71 .77 1.99 Score -

• 

- 

REjk~ k—5 .98 1.21 1.39 •

REk~ k—S 1.19

1

81

_______________________ 

H

- ~~~~~~~ ~~~~~~~~~~~~
- 

- 

______ ~~TT T ITT TT~

;

~~~~ ~~~~~~~~~~~~~~ ~~~~

- 

- Table C—6 ft

REjik, REik, and REk Values for Experimental 

- 

-

Condition Number Six (XSS)

- 

- 

Performance ________________ Events ________________

- measures Mobile Shoot One Bunker Shoot Mobile Shoot Two

r - - ________________ _____________ __________ _____________

Time to Locate 1.03 1.00 1.03 - 

-

Target

Total Missiofl .83 .66 1.16 Time

Accuracy of Initial 87 54 1 21 Target Location -

Number of 

Fired .84 81 1.14

Procedural Errors 1.19 • 

3.60 .94 Cut Score

Total Cuts .84 .66 1.53 Score

RE. , k=6 93 1.21 1.17

RE.,~~ k=6 110

:

1. : -

H82

____________________________________ *ti

~

;L11~

_ __

_

~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~
__ ‘

~

T

~ ~~~~~~TTTI ii__

~~~~

WT

Table C-7

I 

REjik. REik? and RE.~ Values for Experimental

I Condition Number Seven (SSS)

Performance ________________ 

Events ________________

- 

measures Mobile Shoot One Bunker Shoot Mobile Shoot Two

- - 

Time to Locate 

92 08 2 Target - 1.0

I Total Mission .9]. 1.06 1.29

Accuracy of Initial 1.33 .86 1.21

Number of

Rounds Fired - - - 8

- 1 a

Procedural Errors 

87 1 00 1 49 Cut Score -

- 

Total Cuts .98 83 1.74 1 Score

-

- - k=7 1.01 1.04 1.32

REk I k=7 1.12

a

- .5 - ii -

~~~~~~~~~

83

• - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

____
— ~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ -~~~~~~~——--‘———- __________ — ________

c4

~~~~~ 

— 

~~~~~~~~~ 

-

I ~~~C~flI)~~ ~~~~ ~~~~~~~~~~~ y

~~j~~~ ‘

~~ 11

DISTRIBUTION • - 

I

- 

ARI DhVtbutJun L4,t -

4 OASO (M&RA) 2 IIQUSACOEC. Ftard. ATTN: Li& y -

2 HQUA 4DAMI-CSZ) I I4OUSACDEC Ft Ord, ATTN: ATEC-~

-EX-E--I4um F.c~~s

1 HODA (DAPE-P8R - - - 2 USAEEC, Ft Ssni4)nin t

~~

r4i..n ATm: U~ wy

I HUQA (DAMA.AR) - I USAPACUC. Ft BenluilIn Huivlucn, A~

TTP4~ ATCP-44 11

• I HODA (DAPe-HRE-P0) - - - I USA Conwn—E$sctSdi, Ft Maum~

ou

~ ’

, 

.ATTN: ATSN—EA

I HonA tSGRD-ID) - 1 USAEC. Ft Manmoutfi. ATYN: AMS~L—CT --NDP

I HQUA (OAMI-OOT-C) - 1 USAEC. Ft Manmouth,, ATTN: MISEL-PA-P

I HUQA (DAPC-PMz-A) - - I USAEC. Ft Monmauth, ATTN: AMSEL-$I-C8

I HUQA (DACH-PPZ-A) - - - I LSAEC. Ft Monmoulfl, ATTN: C,, F.d Div B~ I HODA (DAPE-HRE) - I USA M.Iri Sys AnulAgay, Ab.ithan, Aim: AMXSY-P

I HODA (DAPS-MPO-C) I Edgewood Anenal, Absu-duen, ATTN: WEA-RL--44

i ..oo* ( P ~ -0W) - I USA Cr6 Cit & Sit’. Aberdsen,M15 ATSL—TEM-C -

1 HODA (D*PE-HRL) • 2 USA Hum Iny Lab. Aberdusn. AflN: Library/Dir

1 HQUA IDAfl-~~S) . - - - I USA Chmbiut Arms Yng Sd. Ft ~unnln~ Aim: Ad Supervisor

I P400* (DAFD-MFA) - 

- I USA Infantry Hum Rich Unit. Ft Bs’.vul.’.g. ATTN: Chief

-~ I HQUA (DAllD-ARS~ ) - - - I USA Infantry Sd, Ft Sunning, Ails: STEBC—TE--T -

• I HOOA (DS?C-PA&*l 1 USASMA. Ft Dli.., ArrN: Af

~~—LAC

I I40~~ (DUSA4 ~

f) . • • I USA Alt D.f Sth. Ft Bliiu5 ATTh: AUA—CTD-ME -

• - I HODA (DAMO-ROR) - • I USA Air D.l Sd’., Ft B$h.. AT1N: Tidi Lib -

I HQOA (DASGI - - I USA Alt 0.1 Bd. Ft Bliss, ATffi: FILES

- 1 HQUA (DA1D~t) . 

- I USA Au Del Sd, Ft Slits. MTh: STEBO—PO -

1 Chief. Conadt Div (DA-OUG). MuIØ’.I. MD I USA Cmd & General Stt Call,.. Ft L.. ~ uth . ATTN: Ub

r I 511 Ann. Hum Re.. 000R&E. 0*0 ~

E&LS) 1 USA Cind & General Sit College, FtL.........Snnh . Aim: ATSW-SE-L C 1 HO USARAL, APO S.~ tl,,, ATTN: ARAGP-R 1 USA Ciod & General Sit College, Ft L.e.enworth, ATTN : Ed Advisor

I HO Ficit Army. ATTN: AFK#0l-Tt - 1 USA Combined Arms Onhi Div Act. Ft La..e....twffi , ATTh: DepCdr

2-HO Fifth Army, Ft Sen. Ploumsue • I USA Combined Anne Cmbt Dee Act. Ft Leavenworth. ATffi: CCS

1 Dir. Asmy Sit Stu~ es Oft, ATm: OAV~~A ~D~~) - I USA Combined Arm. Cmbt Des Act. Ft Leavenworth. ATTN: ATCASA

I O~c Ch~e

~ 

o

~ 

Stf - Sttdle, Ott - 

~ USA ~~~~~~ Armi Cmbt 0ev Act. Ft l.

ctvrw’.orth, *7Th: ATCACO—E

1 DCSPER, *7Th: PS~~ F I USA Comb ned Arr’ s Cmtt Dee Mt. Ft Leavenworth. AiiM: ATCACC—Cl

I The Army Ub, F..li,.,,,, Aim: RSB Chief - I USAECOM. Night Vi.uon Lab,- Ft 8.1.0k , Aim: AMSEL—NV---SD

I The Army Ub. Pentagon, Aim: *IIRAL 3 USA Computer Sy, C.nd, Ft 5.1.0k, AUN: Tacit Ubrery

I Oft, Atst Sect of the Army (R&D) I USAMERDC, Ft Selvoir, AiiM: STSFB-DO

1 Ted’. Supptst Oft, OJCS I USA Eng Sd’.. Ft Belvok, AiiM: Library

1 LSASA, Mington. Aim: lARD-i - 1 USA Tapoy~’.l.ic Lab. FtBeIvclr, Aim: 5Th-iD--S ‘1 1 USA Red’. Ofc, Duthem, Aim: Life Sd.ncu Dir I USA TOPOIr PhIC Lab, Ft Belvoir. ATTN: STINFO Center

2 USARIEM , Nitidi, ATTIt SGRD-UE-CA - 

- I USA Topographic Lab. Ft 5.1.0k, ATYN: ETL—GSI1 USATTC, Ft Dayton, ATYN: STETC-M0-A - 1 USA tntslligenct Cu & Sd.. Ft Huedsoci ATTh: CTD-MS

I USAIMA. Ft Bragg, Aim: ATSU-CTD-OM I USA Intelligence Cu & Sd’., Ft Hu.d’.uce, AT IN: ATS—C’ID-MS

1 USAIMA. Ft Bragg, AiiM: Ma~

quat Ub I USA Intelligence Co& Sd’., Ft 1-luaciwea, AiiM: AiSI-TE -

• 1 US WAC Ct, & Sd., Ft McClellan, AIiM: Lib I USA lnt.Illgenci Cit & Sd’. Ft (Iueituucs, AIiM: ATSI—TEX---GS

1 US WAC Cit & Sd.. Ft M~~

IeIIan, Aim: Tng Di 1 USA Intelligence Cu & Sd’., Ft Huediuce, ATTN: ATSI—CIS-OR

I USA Du.rt.. ,..s.a.. Sd’., Ft Lee, AiiM: ATSM-TE I USA IntliUgenc. ‘.

~

u & Sd’., Ft Huidiuc., Aim: ATSI—CTD-DT

1 Intelgence i M.tectal Dee Oft, EWL. Ft HolubIrd - 1 USA InteIIlg.nce Cu & Sd., Ft Huadusca, Aim: ATSl-CTD-~~ I USA SE Sigeel 5th, Ft Gordon, AiiM: ATSO-EA 1 USA Intelligence Cv & Sd’., Ft Huaduuca, ATflI: DASISRD

1 USA Chaplain Cit & Sd.. Ft Hamilton, AiiM: ATSC-TE-R0 - I USA Intelligence Cu & Sd’., Ft Huad.uce, AIiM: ATSI-TEM

1 USATSCH, Ft Eusda, AiiM: Educ Advisor - I USA Intelligence Cu & Sd’., Ft Huecituca, Aim: Library

1 USA War College, Carlisle Bamsdu ATTN: Lib I CDII, HO Ft Huad’.uca, AiiM: Ted’. Ref Dlv

2 WRAIR, Neuropsycfuid.y Div 2 COR, (

~~ E$ectn,nlc Preg Grd

~ 

Aim: STEEP—MT-S

1 DLI, BOA, Monterey I HQ, ICATVt, ATTNs Tesi. LIbra.y

1 USA Concept Anal AScy, Both.sdi, ATTN. MOCA’MR 1 H0 TCATA, ATfl t AT CAT-OP-Q, Ft Hood

• ‘ I USA Concept Anal AeCy, Bothuida, A TTN. MOCA-JF I USA Recruiting Cmd, Ft Shesidan, ATTN. USARCPM-P - 

-

-

I USA Arctic Y~it CU’, APO scettle. ATTN* SiEAC-PL.MI I Senior Army Adv. USRFAGODITAC, Lu l’. AF Aux Fil No,f

1 USA Arctic Test CU’, AFO Ssmttte, Arrilt AMSit-PL-TS I HQ USARPAC, DCSPER, APOSF HISS, AiiM, OPPE4E

1 USA Armament Cited, Rastone Arsenal, ATTN. ATSK-TEM 1 Stlmion Llb~ Academy of Health Sciences. Ft Sam Houston - 

-

1 USA Armament C,ód, Rock Island, Au Nt AMSAR-TOC I Madni-Corp. meL , ATTN. Dean-MCI

I FAA-NAFEC. Atlantic City, ATTN. Library - I HQ$JSMC, Commandant, A’TTNa Code MTMT

I FAA-NAPEC, Atlantic City, ATTN. Hum Engr Br I HQIJSMC, Commandant, AflNs Code UP$-20-U

I FAA Ae.-onautlcat CIt. Oklahoma City, ATTN. AAC44 D 2 USCG Academy, New London, ATTN. Admission

A 2- USA FM Arty Sd’. Ft SIll. ATTN. Ubra.y 2 USCG Academy5 IdewLOndosi, ATTN. Library

I USA Armor Sd’., Ft Knox, ATTN. Ubrary 1 USCO Training Cit. NY, ATTN. CO

1 USA Armor Sd., Pt Knox, ATTN. ATSB-Ol

’

~ I USC4) TrainIng Cir, NY, ATTN, Edit Svc Oft

1 USA Armor Sd., Ft Knox. A rrm4s ATSB-OT-T’ - I USCS. P*YthOI M’s 5., DC, ATTN. aPI/SI

a us* Armor Sm, Ft Knox, ATTN. AiSS-CO-AD I HQ MId Rai’.ge Br, MC D.t, QUISBCO. ATTN. P05 Div

85

— - — 

~

i’_ --5

~~~~:L:~~~~~

~~

r4_ ~~~~~~~~~~~~~~ — ,

~~~~~~~~~~~~~~~~~~~~~~~ ~~~j cii.
w~ fl

~~ ~~~~—,.. —,t~~ )

I US CD~s LIsI~cui Do, MIC. AJ*L~~15, AiTh: ~~~~~S-F I 

~~

i& CiwN hisici EIwIIO ~~~~~~~~~~~ Coe.d’

I (*ATR*OOC. Ft MciWos, Aim: ATRO-ID I MR CAUS. K i~

ip, , ATTN: Iais$~

o k

S USATRADOC. Pt Mis’s’. AiIM: ATPR—AO I MUI~ ;_Tks’*, ~~~~~~~

I USATRACOC. Ft M.s’os. AIM: i Afl$-EA I ~~Is’Y AtMdct, Pliad’ Esd’sNy, ATIW: Doe I.e

I USA FINSSOiI& Ft Mdh~~~m. ATTN LNiu,y I Msd’da OW. C.LA~ &-An.ad. TsulcadNmW Fs’ic.

2 USA Awluda n Tctt Sd. F! Nudm~. ATTN: STUS-lO ~ ~~~ IdiudNiOsi. fr~~

I His’ ~~ NidI DIV. M)PI Y

1 USA Ap.y far .folsda Iduty. Ft Ru~~~. ATTW: Ubcty ci D.1uiliS. t4iw Quid

I (RA Apey for Ai4ct1.n Idly. Ft Nudl. AiIM: 5d’~ ~~~~~~~ I Put Radi Ole U~ cty. AKA. lan d Duluiws Panic

I USA Avision Sd’. Ft Nick., AiIM: P0 Dos’rO I Mli*MIIi Vi, DUINOIS. 000PIK). AId $aduul

I I$0N*Astsdon 5w.. Dod. I Laid’, AiIM: AMMv-.zoR P~~~~~~u~i Zd’un, Tim K.~a, N.di.i1 igh

2 USA Ailudsiw Si. ~~~ Act.. E~~~d’ APP. ATIN: $AVTE—T

I USA Ak ci Sd’, Pt Ill ’~ ATm: A1IATMi

I USA Mr Mislay Id & civ Lth. Maifsct Fid, AiIM: SAVDL—AS

1 USA Adsdsn Ida, A.. Tag Mgi, Ft Nudl, AflN: ATS1—T—RTM

I USA AvludanSdb,00, Pt Rucki , AnN: A1IT-O-A

I HO. DARCOM. AIs’indds. Aim: AMXCD-TL

I HO. 

DARCOM. Alixufl*IL Aim: CON

1 US MNls’y Acidumy. ~~~ t Paint. ATTN lurid’ UnIt

I US MII1s’y As’d.my, Woe Polct~ Aim: Ofc ci Mill ládip

I US MUIoey Ac

~~.ny. Woe PoInt. Aim: MAOR

1 USA Soedidicudon Gp. UK, FF0 NY. Aim: MAlE-aC

I O(c of N.vuI Ridi. Mlnilan. ATTN: Cod’ 452

3 Ole ci Nctd Audi, Mhigion, ATVN: Cod’ 455

I Oic ci Nsvsl Rid’, Pdngion. ATTN: Cods 450

I Ole ci Nivd Audi, MinpInmi. Aim: Cods 441

I Nuvd Auevupc Mid A. L* Fuonii.Js, ATm: A.c.m Sd’ Dlv

I N..d Asroicc Mud Act Id’, Pu~~..Js. Aim: Cod’ 151

1 Noel Auroupe Mud A. Ld’. £u~uu.Js, Aim: Cods 1.5

I CliUf oiNuwPs,s,ATTN:Pcti.OR . 4

I NAVAIRSTA~ Noifolk. Aim: Ssfsty CD

1 His o.mun~~

agg~

, DC. Aim: Cod’ 6251. CiwaiRTudi

I cunoe.l NwslMd,ATTN:DocCD

I NctAkhi~~om, ATTN AIR-5213C

I A.SuMud.AiTh: 713

1 ~~~~~~~~~~~~~~~~~ 2. FF0 SF 55601

I AFHRLIFUWIIIdAFI

I AFHNL(TT) Lossy AIR

I AFHRL (AS) WPAFLOII

2 AFHRL (D0.~ ) S,ook. AIR

1 AFHRI. (DON) Laddici d All

I H~~RAF (INTRO)

I HCO~~~ WF%XA )

I AFVTG (Rb) Ruidoipli AIR

3 AMRL OlE) ~~ AFI. OH

2 AF l,

~Io( Teds. ~~AFL OH, Aim: ENEISL

1 ATC (XPTDI Rs’dalpIi APR

1 INAF MroM.d Ub. Broil. Aft (SUL-4). Aim: DOC SEC

1 AFOIR INU. Mlngsun

I Al I.ag0nd, 

~~4~’

ulIon APR, AiIM: ALCIDPCNS

I Air For., A~~~ mp, DO, AiiM: Dupt of i Son

I i Poe&DssCb,$onDlugo

2 A.y Mid Nmmr~~~dd.DIc Id, IMla. Sic, blip,

1 Nay aaao.~.. 

1* Son Dlsp,,ATfN: Ass Ld’

I N~ T,ngC.n, Sic, Diup,, Aim: Cods SCCO—Ub

I NuwPcsiGrddm, M,.L.,i , Aim: Cod. ISA.

I N.PauiG.uddi. M..i.,uy , Aim: Cods 2121

I N.vTmgEqulpCt, Odsids. Aim: Tidi Ub

I US Duptof Lak.r. DC. Aim: M.cpactur Adoin •

1 US Dept ci kides, DC, Aim: Onug Enfors’ ~~~*, I Nut So, ci Iiu..d.nb, DC, Aim: Couepui cr Info Sucdsa

I Hit Clung How. for NH—Info, Rodivils

I Durour Fedind Ci,, L

~~

_.. J, Aim: SIN

12 OsImuis D.s .LL~~ DoMi

4 ckP~ da,Ac..~N%I

~

.mmok~ culi.vs

1 Sdu.dfl~ Msw, MU SI, Aumy Ifo, Awul Ofes, Cisurru

I MU aid MrAtiss, Idutici C...lusy

I Domu ds A.J...ike Dpi Puectu,,, Nicumlisdo Is Qufonus

2 Cuusir MlntIWlW4iL.gi.a.

I WAir lull. Raydi C.uid.n Al, Aim: P.r. lid Nd Br

301.1. Do’~~~ i ci Audi lull. Aim: C~~

RDS55I

4 RiNd’ Psilull , Rims, ~~~~~~~~~~~~~~~

86

~~~~~~~~~~~~~~~~~~~~ 2

~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~