•^wBommmmn

U.S. Army Research Institute

for the Behavioral and Social Sciences

Research Report 1804

Using the Laser Marksmanship Training System

to Predict Rifle Marltsmanship Qualification

Monte D. Smith

L-3 Communications

Joseph D. Hagman

U.S. Army Researcii Institute

May 2003

Approved for public release; distribution is unlimited.

V_ 20030710 022
U.S. Army Research Institute

for the Behavioral and Social Sciences

A Directorate of the U.S. Total Army Personnel Command

ZITA M. SIMUTiS

Director

Technical review by

Donald Riley, DCSOPS, 84"^ Division (Institutional Training)

Helmut Hein, U.S. Army Reserve Command

NOTICES

DISTRIBUTION: Primary distribution of this Research Report has been made by

ARI. Please address correspondence concerning distribution of reports to: U.S.

Army Research Institute for the Behavioral and Social Sciences, Attn: TAPC-ARIPO, 5001 Eisenhower Ave., Alexandria, VA 22333-5600.

FINAL DISPOSITION: This Research Report may be destroyed when it is no

longer needed. Please do not return it to the U.S. Army Research Institute for

the Behavioral and Social Sciences.

NOTE: The findings in this Research Report are not to be construed as an

official Department of the Army position, unless so designated by other

authorized documents.
REPORT DOCUMENTATION PAGE

1. REPORT DATE (dd-mm-yy) 2. REPORT TYPE

Final

3. DATES COVERED (from. . . to)

June 2002-April 2003

4. TITLE AND SUBTITLE

Using the Laser Marksmanship Training System to Predict Rifle

Marksmanship Qualification

5a. CONTRACT OR GRANT NUMBER

DASW01-99-D-0012

5b. PROGRAM ELEMENT NUMBER

633007

6. AUTHOR(S)

Monte D. Smith (L-3 Communications) and Joseph D. Hagman (U.S.

Army Research Institute)

5c. PROJECT NUMBER

A792

5d. TASK NUMBER

219

5e. WORK UNIT NUMBER

COS

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

L-3 Communications

Link Simulation & Training Division (LSTD)

13665 Dulles Technology Dr., Suite 350

Hemdon, VA 20171

8. PERFORMING ORGANIZATION REPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue

ATTN: TAPC-ARI-ID

Alexandria, VA 22333-5600

10. MONITOR ACRONYM

ARI

11. MONITOR REPORT NUMBER

Research Report 1804

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES

Contracting Officer's Representative: Joseph D. Hagman

14. ABSTRACT (Maximum 200 words): To determine the relation between simulation- (Laser Marksmanship Training System

[LMTS]) and live-fire-based rifle marksmanship performance, 186 Reserve Component (RC) soldiers from Idaho and Oregon fired

for qualification on a scaled LMTS version and live-fire version ofthe Army's standard pop-up target qualification course. LMTS

was fired under either a dry-fire mode or a Blazer (i.e., sound/recoil replicator) mode. Statistically significant positive linear

relations were found (and then validated) between first-run live-fire scores and both LMTS dry-fire- (r = .50) and Blazer-based

(r = .55) scores. These relations were ofsufficient strength to permit development of easy-to-use tools for accurately predicting

soldier chances offirst-run, live-fire qualification. With these tools, RC marksmanship frainers can implement a competencybased training program where soldiers most m need ofremedial training (i.e., poor shooters) can be quickly identified, and the

point at which sufficient training has been provided (i.e., when first-run live-fire quaUfication is likely) easily determined. These

tools also provide RC unit commanders with empirically derived live-fire performance standards needed to support use of

LMTS m place of live-fire for rifle marksmanship proficiency validation purposes when standard pop-up target course range

facilities are not readily available. Although both tools will serve these purposes, that based on LMTS dry-fire is recommended

because ofthe added expense offiring with Blazer without an accompanying statistically significant increased predictive

benefit.

15. SUBJECT TERMS

Rifle Marksmanship

Army Reserve

rrrsTTTrmmm?

Simulation Laser Marksmanship Training System (LMTS)

Reserve Component

Army National Guard

':^ECURITY CLASSIFICAl

16. REPORT

Unclassified

17. ABSTRACT

Unclassified

18. THIS PAGE

Unclassified

19. LIMITATION OF

ABSTRACT

Unclassified

20. NUMBER

OF PAGES

22

21. RESPONSIBLE PERSON

(Name and Telephone Number)

Joseph D. Hagman

208-334-9390
Research Report 1804

Using the Laser Marksmanship Training System to

Predict Rifle iVIarl^smanship Qualification

Monte D. Smith

L-3 Communications

Joseph D. Hagman

U.S. Army Research Institute

Reserve Component Training Research Unit

Lincea Ruth, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

May 2003

Army Project Number Personnel, Performance

2O633007A792 and Training

Approved for public release; distribution is unlimited.

Ill
FOREWORD

The United States Army Reserve (USAR) is looking for more effective and efficient

ways to train and evaluate rifle marksmanship through the use ofsimulation devices. To this end,

and at the request ofthe U.S. Army Reserve Command (USARC), the U.S. Army Research

Institute for the Behavioral and Social Sciences (ARI) has been working with the USAR's 84th

Institutional Training Division (DIVIT) and Small Arms Readiness Group (SARG) to develop

and evaluate a simulation-based (i.e., the Beamhit Laser Marksmanship Training System

[LMTS]) training and evaluation program for use at home station (i.e., reserve centers). The

common goal ofthis cooperative effort is to field a program that will produce marksmanship

proficiency levels that meet or exceed unit readiness requirements while minimizing the

resources needed to do so.

To date, a Program ofInstruction (POI) has been developed to cover the use ofLMTS in

training rifle marksmanship fundamentals (e.g., steady position, aiming, breath control, and

trigger squeeze). A necessary step in the implementation ofthis POI is to determine the relation

between LMTS- and live-fire-based marksmanship performance and, iffound to be ofsufficient

magnitude, develop tools for trainers to use in predicting the latter fi-om the former. This report

describes research conducted to assess this relation and then develop separate tools for predicting

Army standard pop-up target qualification course scores from LMTS dry vs. Blazer-based modes

offire.

This research was conducted by the ARI-Reserve Component Training Research Unit,

whose mission is to improve the effectiveness and efficiency ofReserve Component (RC)

training through use ofthe latest in training technology. This research is supported under Work

Package 219, "Strategies and Tools for Maximizing AC/RC Unit Performance," ofARI's Science

and Technology Program for Fiscal Year 2002.

This research was sponsored by USARC under a continuing Memorandum of

Understanding initially signed 12 Jvine 1985. Findings have been presented to the Weapons

Program Administrator, USARC; and Deputy Chief of Stafffor Operations, 84th DIVIT, USAR.

ITHLEEN A. QUINKERT

Acting Technical Director
USING THE LASER MARKSMANSHIP TRAINING SYSTEM TO PREDICT RIFLE

MARKSMANSHIP QUALMCATION

EXECUTIVE SUMMARY

Research Requirement:

To develop Laser Marksmanship Training System (LMTS) dry-fire- and Blazer-Based

(i.e., sound/recoil replicator) tools for predicting first-run rifle marksmanship performance on

the Army's standard, pop-up target qualification course.

Procedure:

A total of 186 Idaho and Oregon Reserve Component (RC) soldiers fired for qualification

on a scaled LMTS version and live-fire version ofthe Army's standard pop-up target

qualification course. LMTS was fired under either a dry-fire mode, where shooters had to

manually cock their rifles after firing each laser round, or a Blazer mode, where nonpolluting

blanks provided the recoil and noise associated with live rounds. Separate regression analyses

were performed to identify the relation between live-fire scores and those obtained under the

two LMTS firing modes. The identified relations were then used to develop separate LMTSbased tools for predicting the probability ofsoldier live-fire qualification.

Findings:

Statistically significant positive linear relations were found (and then validated) between

first-run live-fire scores and both LMTS dry-fire- (r = .50) and Blazer-based (r = .55) scores.

These relations were ofsufficient strength to permit development of easy-to-use tools for

predicting first-run, live-fire qualification with up to 90% confidence using the former and

with up to 80% confidence using the latter.

Use ofFindings:

With these tools, RC marksmanship trainers can implement a competency-based training

program where soldiers most in need ofremedial training (i.e., poor shooters) can be quickly

identified, and the point at which sufficient training has been provided (i.e., when first-run

live-fire qualification is likely) easily determined. These tools also provide RC unit

conmianders with empirically derived live-fire performance standards needed to support use

ofLMTS in place oflive-fire for rifle marksmanship proficiency validation purposes when

standard pop-up target course range facilities are not readily available. Although both tools

will serve these purposes, that based on LMTS dry-fire is recommended because ofthe added

expense offiring with Blazer without an accompanying statistically significant increased

predictive benefit.

vu
USING THE LASER MARKSMANSHIP TRAINING SYSTEM TO PREDICT RIFLE

MARKSMANSHIP QUALIFICATION

CONTENTS ___^_«.

Page

INTRODUCTION 1

EXPERIMENT! ...2

METHOD ; .2

Participants ,. ...2

Eqi%ment 2

Procedure .....3

Analytic Approach 4

RESULTS , ...4

Group 1 Data {n = 55) .4

Group 2 Data (« = 55) 4

Pooled Data {N= 110) 5

EXPERIMENT2 7

METHOD 7

Participants 7

Procedure 7

Analytic Approach .8

RESULTS .....8

Group 1 Data {n = 38) 8

Group 2 Data (« = 38) 9

Pooled Data iN= 76) 9

DISCUSSION AND CONCLUSIONS 11

Sunulation Methods: Dry-Fire vs. Blazer-Based Fire 12

LMTS Inplementation 12

REFERENCES 15

IX
CONTENTS (continued')

Page

LIST OF TABLES

Table L LMTS Dry-Fire-Based Predicted Chances ofFirst-Run Qualification

at Marksman (^3 hits). Sharpshooter (^30 hits), and Expert (> 36 hits)

Levels on a Standard Pop-Up Target Qualification Course 7

2. LMTS Blazer-Based Predicted Chances ofFirst-Run Qualification

at Marksman (>23 hits). Sharpshooter (^ 30 hits), and Expert (> 36 hits)

Levels on a Standard Pop-Up Target Qualification Course 11

LIST OF HGURES

Figure 1. LMTS con^uter/monitor, sanqile electronic target, and laser transmitter with

attached mandrel 3

2. LMTS Blazer 3

3. Soldier firing with the LMTS Blazer on the 15mmini-RETS range .3

4. Soldier firing with the M4 Carbine Blazer 8

5. Flowchart ofdelivery strategy 13
Using the Laser Marksmanship Training System to Predict

Rifle Marksmanship Qualification

Introduction

Tightening budgets, escalating ammunition costs, dwindling numbers ofcertified live-fire

ranges, and ever-present training time constraints have pron^ted the U.S. Army Reserve

(USAR) to search for alternative ways to train and evaluate rifle marksmanship through the use

ofsunulation. To this end, the U.S. Army Research Institute (ARI) has been working in

partnership with the U.S. Army Reserve Command's marksmanship executive agent, the 84*

Division (Institutional Training), to develop and evaluate a simulation-based rifle marksmanship

sustainment training program designed for use at home station. The goal ofthis effort is to

produce shooter proficiency levels that meet, or beat, unit readiness requirements A^4iile

minimizing the time and ammunition needed to do so (Plewes, 1997, November 24).

The envisioned program ofmarksmanship instruction (POI) calls for use ofthe Laser

Marksmanship Training System (LMTS) (BeamHit, 1999; Dunlin, 1999, January-February) to

support (a) the training ofmarksmanship fimdamentals (le., steady position, aiming, breatii

control, and trigger squeeze), shot grouping/weapon zeroing practice, and simulated record fire

evaluation, (b) conq)etency-based exercise delivery where LMTS-based pretesting is used to

predict which soldiers need training (i.e., unlikely live-fire qualifiers) and posttesting is used to

signal when enough such training has been proAdded (i.e., once live-fire qualification becomes

likely), and (c) use ofLMTS-based testing to validate live-fire qualification status when rai^e

facilities are not readily available.

The POI exercises devoted to training marksmanship fimdamentals have been conq)leted and

published in Appendix C ofthe coordinating draft ofField Manual (FM) 3.22.9 (23-9) "Rifle

Marksmanship M16A1, M16A2/3, M16A4, and M4 Carbine" (Headquarters, Department ofthe

Army, 2001, August). In addition, an LMTS-based, live-fire prediction tool has been developed

to provide unit commanders with the needed yardstick upon which to base pre- and posttesting,

as well as validate live-fire qualification status. Current predictions only apply, however, to

scores fired on the Army's 25m Alternate Qualification Course (ALT C) where shooters fire at

stationary paper targets contmning E- and F-Type silhouettes scaled to represent known downrange distances offi-om 50-300m (e.g.. Headquarters, Department ofthe Army, 1989; Smith &

Hagman, 2000).

Although most USAR soldiers typically fire for qualification on ALT C because ofits limited

space and instrumentation requirements, firing on the Army's standard pop-up target

qualification course, often accon:5)lished using a Remote Electronic Target System (RETS), is

preferred whenever possible because ofits demand for a greater array ofcombat-related

marksmanship skills (e.g., target acquisition). To produce a corresponding demand for these

skills during simulation-based qualification firing, BeamHit (2002) has developed an LMTSbased, pop-up target qualification course called "mini-RETS." Research, simikr to that

conq>leted with ALT-C, is therefore needed to determine what the relation is between simukted

and live-fire pop-up target course performance and whether predictions ofthe latter from the
former are accurate enough to support LMTS-based pretesting, posttesting, and a record fire

qualification/validation alternative to live fire.

Current live-fire predictions are also based on an LMTS dry-fire mode oftarget engagement

where shooters manually cock their weapons before firing each simulated (Le., laser) round. To

add realism to this firing experience, and potentially boost predictive accuracy above that

associated with dry fire, a recoil and soimd repUcator enhancement to LMTS, called the Blazer,

has been developed by the device's manufecturer. The Blazer provides a dry-fire alternative that

captures fully simulated Uve-fire fimctionality, using noiq)olluting blanks in place ofUve

ammunition, with nearly 100% ofthe associated recoil and 50% ofthe noise.

The question is whether this added capability results in better Uve-fire predictive accuracy

than that achieved under dry fire. Research is needed, therefore, to identify the relation between

Blazer- and live-fire-based performance, develop and validate a Blazer-based tool for predicting

the latter fi-om the former for the Army's standard pop-up target qualification course, and

conqjare the relative accuracy ofBlazer- vs. dry-fire-based predictions. The present research

provides this information.

Experiment 1

This experiment examined the relation between LMTS-based, dry-fire scores fired on miniRETS and Uve-fire scores fired on an Army-certified, pop-up target qualification course.

Ejq)eriment 2 examined the relation between Blazer-based and Uve-fire scores.

Method

Participants

One-hundred and ten soldiers from an Idaho Army National Guard armored cavalry brigade

support battaUon voluntarily participated in the research as part ofyearly rifle quaUfication firing

at Orchard Range near Boise, Idaho. None ofthe participants had prior e3q)erience firing LMTS.

Equipment

LMTS is a laser-emitting device, designed primarily for indoor use, which enables soldiers to

engage targets without the use ofUve ammunition. Its major components include a laser

transmitter, a mandrel to which the transmitter is attached/aUgned, a variety oflaser-sensitive

targets, and a dedicated computer with optional printer (F^ure 1). One end ofthe mandrel holds

the laser transmitter while the other end sUps into the barrel ofthe weapon. Vibrations fi-om the

weapon's mechanical firing mechanism activate the laser when the trigger is puUed and the

location ofthe emitted beam is "picked up" by the laser-sensitive target(s) and recorded/stored

on the computer for fiiture analysis and printout. Soldiers can shoot with LMTS under a diy-fire

mode using their own weapons, or they can replace the upper receiver assembly with the Blazer

(Figure 2) to enable weapon bolt and ejector mechanisms to fimction with nontoxic blank

ammunition. In this first experiment, soldiers dry fired their own M16A2 weapons on the scaled

15m LMTS mini-RETS pop-up target qualification course shown in Figure 3.
Figure 1. LMTS conq)uter/monitor, sample electronic target, and laser transmitter with attached

mandrel.

Figure 2. LMTS Blazer.

Figure 3. Soldier firing on the 15m LMTS mini-RETS range.

Procedure

To control for possible sequence affects, approximately halfthe soldiers fired for live-fire

qualification first and LMTS-based qualification second. The other halffired this sequence in

reverse, with no more than 24 hr occurring between the two firings under either sequence.

Live-Fire qualification Dedicated cadre fi-om the Idaho Army National Guard conducted all

range-based qualification firing in accordance with procedures stipulated in FM 23-9

(Headquarters, Department ofthe Army, 1989) with shot grouping and weapon zeroing

accomplished immediately beforehand. All pop-up targets were automatically presented and

scored. Hit numbers associated with specific shooting classifications were as follows: 0-22,

Unqualified; 23-29, Marksman; 30-35, Sharpshooter; 36-40 Expert. All statistical analyses were
based on first-run hit scores (Le., the number oftargets hit, out ofa possible 40, on the soldiors'

first qualification atten^t).

LMTS-Based qualification. Simulated qualification firing with LMTS was conducted

indoors by dedicated cadre from the USAR's 84* Division and Small Arms Readiness Group

who followed the same procedures as those associated with live fire (Headquarters, Department

ofthe Army, 1989). LMTS was fired in a dry-fire mode that required soldiers to use their firing

hand to manually cock their weapons after each round by recyclii^ the charging handle located

at the rear ofthe upper receiver assembly. All LMTS targets were presented and scored

automatically.

Analytic Approach

To determine the relation between LMTS- and live-fire-based qualification scores, a splitgroup, cross-validation design was used (Tatsuoka, 1969). The initial sample of 110 soldiers was

randomly divided into two groups. Group 1 was used to develop a prediction equation between

device and live-fire scores. This equation was then applied to Group 2 soldiers to see iftheir livefire scores could be predicted successfully from the equation derived from Grovq) 1 data.

Results

Group 1 Data (n = 55)

For Group 1, LMTS hit scores ranged from 7 to 39 (M= 27.8, SD = 7.7). Live-Fire hit scores

raided from 10 to 37 (Af= 27.5, SD = 5.8). A paired-sanq)les ^test revealed that the mean hit

scores for the two firing methods did not differ significantly. The rejection region for this and all

other statistical analyses was .05.

A least-squares regression prediction equation ofthe form Y' = Bo + Bi(Xi) was developed

in which Y' was the predicted live-fire criterion score. Bo was the intercept (ortheoretical livefire score when the LMTS score equals zero), Bi was the en^irically derived regression

coeflBcient linking chaises in the live-fire criterion variable with changes in the LMTS predictor

variable, and Xi was the obtained LMTS first-run quaUfication score. A significant linear

relation, Y' = 16.485 + .395(Xi), SE = 2.55 was found between LMTS and live-fire

performance, F(l, 53) = 19.96. In addition, the correlation between predicted and actual live-fire

scores was significant (r = .52). Thus, Group 1 LMTS scores were both linearly related to, and

reasonably good predictors of, live-fire scores.

Group 2 Data (n = 55)

For Group 2, LMTS scores ranged from 3 to 39 (M= 28.4, SD = 7.5). Live-fire scores ranged

from 8 to 39 {M= 26.9, SD = 5.9). As with Group 1, Group 2 mean LMTS and live-fire scores

did not differ significantly.

Following cross-validation procedures described by Tatsuoka (1969), the Group 1 regression

equation was used to predict Group 2 live-fire scores, and then the relative amount ofvariance
accounted for in each group was compared. A significant linear relation, Y' = .145 + .966(Xi

[predicted]), SE = 6.56, was found between actual live-fire scores and live-fire scores predicted

from the equation based on Group 1 data, i^l, 53) = 16.33. The resulting correlation (r = .49)

was significant, and the associated Group 21^ of.236 did not difiTer significantly (z = .267) from

the R^ of.274 found for Group 1, indicating that the Group 1 prediction equation accounted for a

con5)arable amount oflive-fire score variance in the two groups. Thus, the predictive model was

found to be vaUd and, therefore, likely to maintain similar efficiency when used to predict livefire scores ofother soldier samples.

Pooled Data (N= 110)

The significant Imear relation found among Group 1 soldiers was successfiilly crossvalidated on Group 2. Given the similar outcomes from these two analyses, and the

generalizability ofthe preliminary model based on only halfthe sair^le, data from the two

groups were pooled in order to develop the best possible live-fire predictions.

Order effects. Approximately halfthe soldiers fired LMTS first and then live fired. For the

other soldiers, this sequence was reversed. Both LMTS and Uve-fire scores were examined to

determine ifthey had been affected by firing order. In neither case was there a significant effect.

Therefore, no ftirther mention offiring order will be made.

Descriptives and mean differences. LMTS scores ranged from 3 to 39 (M= 28.1, SD=1.5).

The minimum qualification score of23 was achieved on LMTS by 80.9% ofsoldiers on their

first quaUfication attenq)t (Ql). Live-fire scores ranged from 8 to 39 (il/= 27.2, SD = 5.8) with

ninety-five ofthe soldiers (86.4%) achieving live-fire Ql. A paired-samples r-test revealed that

live-fire and LMTS mean scores did not differ significantly.

Prediction model. A least-squares regression analysis revealed a significant linear relatioi^

F(l, 108) = 36.29, between the predictor variable (LMTS score) and the criterion variable (livefire score). The correlation (r = .50) between LMTS and live-fire fire scores was significant, with

the former accounting for about a fourth ofthe variance in the latter (^ = .25; adjusted 1^ = .24).

The prediction equation took the form Y' = 16.323 + .387(Xi), SE = 1.87, where Y' = predicted

live-fire score, Xi = LMTS score, and iSE = standard error ofthe prediction.

For this obtamed relation to be ofpractical value, it should allow marksmanship trainers to

predict Uve-fire qualification scores/probabilities based on LMTS dry-fire performance.

Specifically, trainers need to know what levels ofLMTS performance are associated with livefire cut scores of23 (Marksman), 30 (Sharpshooter) and 36 (Expert). To determine the LMTS

score associated with these scores, we took the pooled data prediction equation,

Y' = 16.323 + .387(Xi)

substituted the desired live-fire cut score for Y', and solved for the required score on LMTS.

Thus, to determine the LMTS score associated with a live-fire score of23, we found:

23 = 16.323 + .387(Xi)
6.68 = .387(Xi)

(Xi) = 17.25

Soldiers with an LMTS score of 17 will, on average, obtain a live-fire score of23, the

minimiiTn score required for qualification at the Marksman level. Not all soldiers with an LMTS

score of 17 will obtain a live-fire score of exactly 23, of course. Some will score lower than 23

and some will score higher, but their average score will be 23. Similarly, it can be determined

that, on average, an LMTS score of35 will be associated with a live-fire score of 30, the

minimum score needed for Sharpshooter qualification.

30 = 16.323 + .387(Xi)

13.677 = .387(Xi)

(Xi) = 35.34

Unfortunately, insufficient data in the extreme regions ofthe two scales were available to

permit the accurate prediction ofE)q)ert qualification. The required LMTS score lies outside the

range ofpossible scores (i.e., 0 to 40) and indicates that soldiers wishing to Ql as experts had

best fire the highest score possible on LMTS.

36 = 16.323 + .387pCi)

19.677 = .387(Xi)

(Xi) = 50.84

Assuming that the actual probability offiring these predicted live-fire scores will follow a

normal distribution with approximately equal variability (Hays, 1963, p. 523.) and M= 23,30,

and 36, the probability ofan individual soldier shooting each ofthese live-fire scores or higher

was calculated using the ARI Live-Fke Prediction Tool (H^man, 1998). Table 1 shows the

results. A soldier with an LMTS score of24 (Colvmm 1), for instance, would be predicted to fire

26 on the live-fire range (Coliram 2) and have a 70% chance ofsuccessful record fire

qualification at the Marksman level (Column 3), a 20% chance of qualification at the

Sharpshooter level (Column 4), and less than a 10% chance ofqualification at the E3q)ert level

(Column 5). A soldier with an LMTS score of28 would be predicted to fire 27 on the range and

have an 80% chance ofqualifying Marksman, a 20-30% chance ofqualifying Sharpshooter, and

less than a 10% chance ofqualifying as an Expert, and so forth.

Another way ofusing Table 1 is to read down one ofthe last three colimans to the desired

level oflive-fiore qualification probability, and then across to the first colimin to determine the

LMTS score associated with that probability. For example, ifa marksmanship trainer wants a

soldier to have at least a 50% chance ofrecord fire qualification, then an LMTS score of 17 is

required. The chance ofrecord fire qualification rises to 70% ifthe soldier's LMTS score is 24,

and it climbs to 90% with an LMTS score of 34.

In summary, the observed predictive correlation (r = .50) between LMTS- and live-fire-based

scores was both reliable and generali2able, as indicated by the successfiil cross-validation

procedure using randomly constituted subgroups ofthe total sample. The relation was also of

sufficient strength to be ofpractical value in predicting live-fire qualification performance.
Table 1.

IMTS-BasedPredicted Chances of First-Run Qualification atMarksman (>23 hits).

Sharpshooter (>30 hits), andExpert (>36 hits) Levels on a Standca-dPop-Up Tea-get

Predicted Mean

Chances ofa Live-Fire Score of...

LMTS >23 >30 ^36

Score Record Fire Score Marksman Sharpshooter Exoert

16 10 — —

6 19 20 — --

10 20 30 ~ 'r —

14 22 40 — —

17 • 23 50 ~ —

19 23 ~ 10 —

21 24 60 — ■■ —

24 26 70 20 —

28 27 80 — --

29 27 — 30 —

32 29 ~ 40 —

34 29 90 — 10

35 30 — 50 —

39 31 — 60 __ -

40 32 — — 20

Experiment 2

This ejqperiment examined the relation between LMTS Blazer-Based mini-RETS scores and

live-fire scores collected on a standard Army pop-up qualification course enq)loying RETS

technotogy.

Method

Participants

Seventy-six soldiers from an Oregon Army National Guard infantry battalion voliratarily

participated in the research as part ofyearly rifle qiialification firing at Camp Rilea, Oregon.

None ofthe participants had previous ejqjerience firing LMTS.

Procedure

To control for possible order effects, approximately halfthe soldiers fired for live-fire

qualification first and LMTS-based qualification second. The other halffired this sequence in

reverse, with no more than 24 hr occurring between the two firings under either sequence.

Live-Fire qualification. Live-Fire qualification was fired with the M4 Carbine (using backup

iron sights) and conducted by dedicated range cadre from the Oregon Army National Guard in

accordance with procedures stipulated in FM 23-9 (Headquarters, Department ofthe Army,
1989) with shot grouping and weapon zeroing acconq)lished immediately beforehand. Target

presentation and scoring were automated. Hit numbers associated with specific shooting

classifications were as follows: 0-22, Unqualified; 23-29, Marksman; 30-35, Sharpshooter; 36-

40, Expert. All statistical analyses were based on first-run hit scores.

LMTS-Based qualification. Simulated qualification firing with the LMTS M4 Carbine Blazer

(Figure 4) was conducted on a scaled, indoor, 15m mini-RETS range by dedicated cadre from

the USAR's 84* and 98* Institutional Training Divisions. Blazer firing followed the same

procedures as those used for live fire and employed automated target presentation and scoring.

Analytic Approach

A split-group, cross-validation design was used to examine the relation between LMTS

Blazer- and live-fire performance. The initial sample of 76 soldiers was randomly divided into

two groups. Group 1 was used to develop a prediction equation between Blazer- and live-fire

scores. This equation was then applied to Group 2 soldiers to see iftheir live-fire scores could be

predicted successfiilly from an equation based on Group 1 data.

Figure 4. Soldiers firing with the M4 Carbine Blazer.

Results

Group 1 Data (n = 38)

For Group 1, Blazer hit scores ranged from 8 to 39 (M= 28.8, SD = 8.4). Live-fire scores

ranged from 2 to 36 (A/= 24.1, SD = 7.0). A paired-sanq)les Mest revealed that the mean number

ofBlazer hits was significantly greater than that achieved with live-fire, /(37) = 4.00.

A least-squares regression prediction equation ofthe form Y' = Bo + Bi(Xi) was developed

in which Y' was the predicted live-fire criterion score. Bo was the intercept, Bi was the

empirically derived regression coefficient linking changes in the live-fire criterion variable with

changes in the Blazer predictor variable, and Xi was the obtained Blazer hit score. A s^nificant

linear relation, Y' = 10.675 + .466(Xi), SE = 3.42 was found between Blazer and Uve-fire

performance, F(l, 36) = 16.74. In addition, the correlation (r = .56) between predicted and actual

live-fire scores was significant. Thus, Group 1 Blazer scores were both linearly related to, and
reasonably good predictors o^ live-fire performance despite the finding that soldiers fired better

with Bla^ than with live ammunition.

Group 2 Data (n = 38)

For Group 2, Blazer hit scores ranged fi:om 13 to 40 (M= 31.1, SD = 6.3). Live-fire scores

ranged fi-om 9 to 35 (M = 24.4, SD = 5.6). As found for Group 1, the mean Blazer score was

significantly greater than the mean live-fire score, /(37) = 7.04. Thus, Blazer scores were

significantly higher than live-fire scores in both groups.

Following the same cross-validation procedures used earlier, the Group 1 regression equation

was used to predict Group 2 live-fire scores, and then the relative amount ofvariance accounted

for in each group was coTapesred. A significant linear relation, Y' = -1.021 + 1.0(Xi [preditted]),

SE = 6.56, was found between actual Uve-fire scores and live-fire scores predicted (Xi) fi-om the

equation based upon Group 1 data, F(l, 36) = 13.85. The resulting correlation (r = .53) was

significant, and the associated Group 21^ of.278 did not differ significantly (z < 1) fi-om the R^

of.317 found for Group 1, indicating that the Group 1 prediction equation accounted for a

comparable amount oflive-fire score variance in the two groups. Thus, the predictive model was

found to be valid, in spite ofmean differences between Blazer and live-fire scores and, therefore,

likely to maintain similar efficiency when used to predict live-fire scores ofother soldia:

samples.

Pooled Data (N = 76)

The significant linear relation found among Group 1 soldiers was successfully crossvalidated on Group 2. Given the similar outcomes fi-om these two analyses, and the apparent

generalizability ofthe preliminary model based on only halfthe sample, data fi-om the two

groups were pooled in order to develop the best possible live-fire pr^ictions.

Order effects. Approximately halfthe soldiers fired Blazer first and then live fired. For the

other soldiers, this sequence was reversed. Both Blazer and live-fire scores WCTC examined to

determine ifthey had been influenced by firing order. In neither case was there a significant

effect. Therefore, no fiirther mention offiring order will be made.

Descriptives and mean differences. Blazer scores ranged fi-om 8 to 40 (Af= 29.9, SD = 7.5).

The minimum Ql qualification score of23 was achieved with Blazer by 85.5% ofthe soldiers in

the pooled sample. Live-fire scores ranged fi-om 2 to 36 (M= 24.2, SD = 6.3). Fifty-two soldiers

(68.4%) achieved live-fire Ql. A paired-samples Mest revealed that the mean hit scores found

for Blazer were reliably higher than those found for live-fire, f(75) = 7.50.

Prediction model. A least-squares regression analysis revealed a significant linear relation,

F(1,74) = 31.46, between the predictor variable (Blazer hit score) and the criterion variable

(live-fire hit score). The correlation (r = .55) between Blazer and live-fire scores was significant,

with the former accounting for more than a fourth ofthe variance in the latter (I^ = .30; adjusted

R^ = .29). The prediction equation took the form Y' = 10.470 + .460(Xi), S£: = 2.53, where Y'

= predicted live-fire score, Xi = Blazer score, and SE = standard error ofthe prediction.

9
For this relation to be ofpractical value, it should allow marksmanship trainers to predict

live-fire qualification scores/probabilities based on Blazer performance. Specifically, trainers

need to know what levels ofBlazer performance are associated with live-fire qualification cut

scores of23 (Marksman), 30 (Sharpshooter), and 36 (Expert). To determine the Blazer score

associated with these live-fire scores, we took the pooled data prediction equation,

Y'= 10.470+ .460(Xi)

substituted the desired live-fire cut score for Y', and solved for the required Blazer score. Thus,

to determine the Blazer score associated with a live-fire score of23, we found:

23 = 10.470+ .460(Xi)

12.53 = .460(Xi)

(Xi) = 27.24

Soldiers with a Blazer score of27 will, on average, obtain a live-fire score of23, the

minimum required for quahfication at the Marksman level Of course, not all soldiers with a

Blazer score of27 will obtain a live-fire score of exactly 23. Some will score lower than 23 and

others will score higher, but their average score will be 23.

Unfortunately, further examination revealed that the model cannot successfully predict livefire performance levels above Marksman. For example, when we pli^ged in a live-fire score of

30, tiie minimum live-fire score needed for Sharpshooter qualification, we obtained a Blazer

score of42, which lies outside the range ofpossibility.

30 = 10.470+ .460(Xi)

19.53 = .460(Xi)

(Xi) = 42.46

An even more extreme outcome occurred when we calculated the Blazer score necessary to

predict live-fire scores of 36, the minimum necessary score for Expert qualification.

36 = 10.470 + .460(Xi)

25.53 = .460(Xi)

(Xi) = 55.50

The resulting score of 55 was well outside the possible range ofBlazer scores. Thus, soldiers

interested in live-fire qualifying as either Sharpshooter or Expert would be advised to train to the

highest possible LMTS Blazer score.

ARI Prediction Tool results (Table 2) for LMTS Blazer data reveal that a soldiers with an

LMTS Bkizer score of33 (Column 1), for instance, would be predicted to fire 26 on the live-fire

range (Column 2) and have a 70% chance ofsuccessful qualification at the Marksman level

(Column 3), a 20% to 30% chance ofqualification at the Sharpshooter level (Column 4), and less

than a 10% chance ofqualification at the Expert level (Column 5). From the last two columns in

10
Table 2, it can also be seen that the model cannot support Sharpshooter or Expert live-fire

predictions with a high degree ofprobability.

Even though the model cannot confidently support either Sharpshooter or E:q)ert live-fire

predictions, it can predict live-fire qualification vs nonqualification. For exanq)le, ifa

marksmanship trainer wants a soldier to have at least a 50% chance ofrecord fire qualification,

then an LMTS Blazer score of27 is required. The chance ofrecord fire qualification rises to 70%

ifthe soldier's Blazer score is 33, and it climbs to 80% with a Blazer score of37. Notice,

however, that it tops out at 80%. Even an LMTS score of40 fells short ofsupporting a 90%

prediction offirst-run qualification. Likewise, a perfect Blazer score of40 produces no more

than a 40% probability ofSharpshooter qualification, and no more than a 10% probability of

Expert qualification.

Table 2.

LhdTSBlazer-BasedPredicted Chances ofFirst-Run Qualification atMarksman (^3 hits).

Sharpshooter (>: 30 hits), andExpert (> 36 hits) Levels on a StandardPop-up Target

Predicted Mean

Chances ofa Live-Fire Score of...

LMTS >23 >30 >36

Score Record Fire Score Marksman Sharpshooter Expert

12 16 10 ~ ~

17 18 20 ~ ; —

21 20 30 ~ —

24 22 40 ~ —

27 23 50 10 --'

30 24 60 ~ —

32 25 ~ 20 —

33 26 70 ~ —

36 27 — 30 —

37 28 80 ~ —

40 29 ~ 40 10

Discussion and Conclusions

Overall, these findings show that (a) positive linear relations exist between simulated rifle

marksmanship performance on LMTS mini-RETS and live-fire rifle marksmanship performance

on automated (e.g., RETS) standard Army pop-up target ranges, (b) these relations exist for both

dry-fire- and Blazer-Based simulation, and (c) these relations are both reliable and ofsufficient

magnitude to support implementation ofan LMTS-based tool for predicting first-run, live-fire

qualification rates.

Moreover, the results were consistent with earlier findings (e.g.. Smith & Hagman, 2000;

2001) that established the ability ofLMTS performance to predict ALT-C live-fire qualification

scores/probabilities. Thus, the scores from both ALT-C and standard pop-up target qualification

courses have been successfully predicted through use ofLMTS-based simulation. To date, ALT11
C predictions have only been validated for LMTS dry-fire, whereas pop-vap target qualification

course predictions have now been validated for both dry-and Blazer-Based fire.

In both current experiments, obtained predictive relations were ofsufiBcient strength to be

practically usefiil in addition to statistically significant, permitting the prediction offirst-run

qualification with up to 90% confidence using dry-fire-based scores (Table 1) and with up to

80% confidence using Blazer-Based scores (Table 2). Thus, with either LMTS dry- or BlazerBased fire, it is possible for marksmanship trainers to set the minimum acceptable probability of

first-run, live-fire quaUfication (either for individual soldiers or for the unit as a whole) and then

conduct LMTS-based training/evaluation to meet or exceed that standard.

Simulation Methods: Dry-Fire vs. Blazer-Based Fire

Firing LMTS imder a dry-fire mode is less expensive, less conq)lex, and less prone to

malfimction than firing LMTS under a Blazer-Based mode, primarily because the former

requires less equipment. Under the dry-fire mode, soldiers use their own (intact) standard-issue

weapons equipped with a laser-emitting barrel insert/attachment. Under the Blazer-Based mode,

however, a weapon's upper receiver assembly must be replaced by the LMTS Blazer assembly.

While the replacement process is not difficult, and while the Blazer assembly is realistic in

appearance, weight, balance, and (sunulated) function, firing with it involves additional cost

(e.g., for the assemblies themselves and the nontoxic blanks they fire). Moreover, Blazer

assemblies require additional storage space, require maintenance during operation, and were

observed to be somewhat susceptible to malfunction.

Thus, although the Blazer-Based mode offire has the potential ofdeUvering enhanced

realism, including features such as semi-automatic ammunition cycling, spent cartri(^e ejection,

recoil, sound effects, and smoke, these added features are achieved at addition^ cost witfiout

additional predictive benefits. That is, live-fire predictions derived from Blazer-Based firing

were no more accurate than predictions associated with dry-fire. Thus, the dry-fire method of

LMTS-based simulation seems preferable at this time. The dry-fire version ofLMTS mini-RETS

works now and provides a viable marksmanship training and evaluation tool that is ready for

immediate implementation

LMTS Implementation

Based on the results ofthis and other supporting investigations (e.g.. Smith & Hagman,

2000; 2001), it is clear that LMTS can be used to predict first-run, live-fire qualification scores

with a reasonably high degree ofsuccess, and that this predictive capability can be applied either

to the standard Army pop-up target course offire, as demonstrated here, or to ALT-C, as

demonstrated earUer (Smith & Hagman, 2000). When this predictive capability is coupled with

the previously developed and validated (Hagman, 2000) LMTS-based rifle marksmanship

program ofinstruction (POI) (Headquarters, Department ofthe Army, 2001, August), the result

is a con5)rehensive package ofLMTS-based technology that can be used as an effective

di^nostic tool for (a) identifying soldiers most m need ofremediation/sustainment training, (b)

signaling when enough such training has been provided, and (c) providing simulation-based,

live-fire performance standards for enabling the substitution ofLMTS-based qualification firing

12
for live-fire qualification firing when outdoor range facilities are not readily available. In doing

so, these tools can form the basis for con^ency-based instructional delivery within tbs USAR's

newly developed rifle marksmmiship POL

Figure 5 shows, in flowchart format, how this competency-based delivery approach would

work. For example, soldiers would first be pretested by dry firing with LMTS on simulated miniRETS (or ALT-C). Based on their pretest scores, soldiers would receive either a "Go" or "NoGo"

depending upon which LMTS-based cut-offscore were set beforehand by the unit commander.

Let's say, for instance, that the LMTS pretest cut-ofiFscore was set at 28, the score associated

with an 80% probability oflive-fire qualification at the Marksman level (Table 1). Soldiers firing

at or above tMs cut offwould receive a Go and be considered simulation qualified. Soldiers

firing below the cut offwould be identified as needing remediation (to be delivered via the

current LMTS-based POI). Remediation is provided only for those in need ofit, thereby, making

the most ofvaluable training time while saving range time and ammunition in the process. Those

convicting remediation would then be posttested on LMTS. Those receiving a Go on the posttest

would be considered simulation qualified, whereas those receiving a NoGo would undergo

fijrther remediation until they were able to meet the posttest cut-offscore and its associated livefire ejq)ectancy standard of 80% probability oflive-fire qualification.

Pre/Posttest ^

r

^Scare \ ]SfoGo ^ Train ) y n

Go

r

Device

Qualified

Figures. FtowchartofdeUverystrategy

The LMTS prediction tool (Table 1 for dry fire; Table 2 for Blazer) provides an empirically

derived set ofmarksmanship performance probabilities for use in determining live-fire

qualification standards. Such standards, in the form ofcut-offscores, would be required to

support a decision to use LMTS to validate live-fire marksmanship proficiency. Unit standards

could be set at an acceptable probability level, using the values in Column 3 of either Table 1 or

2. Standards could be chained over time to reflect unit proficiency, or different standards might

be applied to different unit subgroups.

This competency-based delivery strategy for LMTS-based marksmanship training, with

empirically derived live-fire performance standards serving as its basis, should enable the USAR

to take a substantial step forward in its ongoing commitment to meeting the Total Army

13
readiness challenge through more productive home-station small arms marksmanship training

and evaluation. Current LMTS technology has the potential ofoptimizing the payofffrom each

soldier's live-fire range e3q)erience, while conserving ammunition and ensuring that scarce

trainii^ resources are devoted to tlrose most in need. With their marksmanship skills at the ready.

Reserve Conqwnent soldiers will be better prepared to mobilize and deploy on short notice when

called upon to support worldwide contingency operations.

14
References

BeamHit (1999). Marksmanship training that works (Brochure). Columbia, MD: Author.

BeamHit (2002). M/M-J?£:r5/?a«ge (Brochure). Columbia, MD: Author.

Dunlin, E. G. (1999, January-February). BEAMHIT: This marksmanship training system

uses lasers and can go anywhere. Armor, 44-46.

Hagman, J. D. (2000). Basic rifle marksmanship training with the Laser Marksmanship Training

System (Research Rep. 1761). Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Science. (ADA380109)

Hagman, J. D. (1998). Using the Engagement Skills Trainer to predict rifle marksmanship

perfonmace. Military Psychology, 10(4), 215-224

Hayes, W. L. (1963). Sto/i^ftcj. New York: Holt, Rinehart, & Winston.

Headquarters, Department ofthe Army (2001, August). Rifle Marksmanship M16A1, M16A2/3,

M16A4, andM4 Carbine (Field Manual 3-22.9(23-9); Coordinating Draft). Washington,

DC: Author.

Headquarters, Department ofthe Army (1989). M16A1 andM16A2 Rifle Marksmanship (Field

Manual 23-9). Washington, DC: Author.

Plewes, T. J. Jr., MG. (1997, November 24). Memorandum: Subject; U.S. Army Reserve

(USAR) Weapons Training Strategy for the 21st Century.

Smith, M. D., & Hagman, J. D. (2000). Predicting rifle andpistol marksmanshipperformance

with the Laser Marksmanship Training System (Tech. Rep. 1106). Alexai^ria, VA: U.S.

Army Research Institute for the Behavioral and Social Science. (ADA384045)

Smith, M. D., & Hagman, J. D. (2001). A review ofresearch on the Laser Marksmanship

Training System (Research Note 2001-05). Alexandria, VA: U.S. Army Research Institute

for the Behavioral and Social Science. (ADA388198)

Tatsuoka, M.M. (1969). Selected topics in advanced statistics: Validation studies.

Champaign IL: Institute for Personality and Ability Testing.

15