Technical Report 492j

tIo EFFECTS OF PRESENTATION- AND TEST-TRIAL

0 TRAINING ON ACQUISITION AND RETENTION

OF MOVEMENT END-LOCATION

r-eal Joseph D. Hagman

____ _DTIC

JUL2 198t*

U. S. Army A

Research Institute for the Behavioral and So~cial Sciences

LUj

LA. November 1Q80

Approved for public release; distribution unlimited.

81 6' 29 186
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

FRANKLIN A. HART

JOSEPH ZEIDNER Colonel, US Army

Technical Director Commander

NOTICE S

D ISTR IBUT ION Pr in ary distribut on cof thIs report has bean mnade by A RI Please address corresPondenzs,

Conrcerning distribution of reports to U S ArmyV Research Institute for the Behavioral and Social Sciences.

ATTN: PERl TP, 5001 Eisenhowerr Avenue, Alexandria. Virginia 22333

FINAL DISPOSITION This report may be destroyed when it is no longer needed Please do not return it to

trie U S, Army Research Institute for the Behavioral and Social Sciences

NOTE The findings in this ret.ort are not to be construed as arn cficei Department of the Armyi position'

uiess so designated fly other ruthpried documnts
UNCLASS IF lED

SECURITY CLASSIFICATION OF THIS PAGE (When D'ata, ... nred)

READ INSTRUCTIONS REPORT DOCUMENTATION PAGE BEFORE COMPLETING FORM

1. REPORT NUMB q 2. GOV ACESO/o .RCIIN TLGNME

Technical Re port 1492j - 2 OT CESONN.3 RI N~T O NUBE

4. TITLE (and $qblitle) 5,J, .lRrPORT A PERIOD COVERED

IFCSOF PRESENTATION- AND TEST-TRIAL TRAINING k FECS F..'"(9 'i / * ""

ON ACQUISITION AND RETENTION OF MOVEMFT END- -_/_ : _______-_-_.__

LOCATION- 6. X5.gfRMaU One. REPORT NUmBEf'

7. AUTHOR(e) 8. CONTRACT OR GRANT NUMBER(.)

Joseph D./agman/

9. PERFORMING ORGANIZATION NAME AND ADDRESS t0. PROGRAM ELEMENT. PROJECT. TASK

AREA A WORK UNIT NUMBERS

US Army Research Institute for the Behavioral & /

Social Sciences, PERI-Il

5001 Eisenhower Avenue, Alexandria, VA 22333 2T1611IA9IB

II. CONTROLLING OFFICE NAME AND ADDRESS". 12. '*l- -

Deputy Chief of Staff for Personnel /I November 1181

Washington, DC 20310 -3I NUMBER OF PAGES

14. MONITORING AGENCY NAME & ADORESS(f different from Controlling Office) 15. SEC URITY CL ASS. (of thie report)

1- nc lass i fied

_ . .. . / .. ASSI UDECL FIC ATICN 'DOWNGRADING

SCHEDULE

16. DISTRIBUTION STATEMENT (of this Report)

Approved for public release, distribution inlimited.

17. DISTRIBUTION STATEMENT (of the abetract entered In Block 20, lt different froo Report)

IS. SUPPLEMENTARY NOTES

19. KEY WORDS (Continue on reverse side If neceeary and Identify by block number)

Training Repetition

Motor Skills Presentation Trials

Learning Test Trials

Retention Linear Positioning

20. ABrR ACT (-len1se am Mr eld. if necIasa" ad Identltfy by block inmber)

The experiment examined relative eff(cts of three training methods on acqtl,iisition and retention of movement end-location information. 'Three groups of l'q

governmental employees performed three training trial cv'les consisting of six

trials each. Training methods differed in ('mphas is on prestent;ition (p) inl Itest

(t) trials performed during each cycle. For ont grop, ;a cvcle consist ei ot

three p- and three t-trials administered in alternation. For another grouip, tilt,

first five trials of each cvc( were p-trials. and the sixth was a t-trid1l.

D IJA 7) E,,,,o , O' ,ETE :Ur" ,NCI.ASS IFt':1).

SECURITY CLASSIFICATION Of THIS PAG, (When l-le Pnt , e
UNCLASSIFIED

CURITY CLASSIFICAI ION OF THIS PAGE(Whm Dole Eaiemd)

For the last group, the first trial was a p-trial and the next five were ttrials. Group acquisition performance was compared at the last trial of each

training cycle, while retention was compared 3 minutes and 24 hours after

acquisition.

Absolute (unsigned) error revealed that final acquisition was better when

training emphasized p-trial repetition or p- and t-trial alternation within

cycles. Long-term retention was better when training emphasized t-trial

repetition. It was concluded that testing is an effective way to improve longterm retention of motor skill and that improvements could be realized by

changing the emphasis of training from presentation to testing. This could be

done without added training resource expenditures,.

INCLASS I F I ED

SECURITY CLASSIFICATION OF THIS PAG(W*.n P0. F-nfed)
Technical Report 492

EFFECTS OF PRESENTATION- AND TEST-TRIAL

TRAINING ON ACQUISITION AND RETENTION

OF MOVEMENT END-LOCATION

Joseph D. Hagman

Submitted by:

Robert M. Sasmor

Director, Basic Research

Approved by:

Joseph Zeidner

Technical Director

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

November 1980

Army Project Number In-house Laboratory

2T161 101A91 B Independent Research

ADpro d for public nrbai " distribution unlimited.

iii
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

iv
FOREWORD

The Training Technical Area of the Army Research Institute for the

Behavioral and Social Sciences (ARI) conducts research in support of the

systems engineering concept of training. A major objective of the research

is to develop the fundamental data and technology necessary to improve

training procedures and enhance individual job performance.

This report examines the relative effects of different training methods

on motor skill performance and is one of a series on specific topics in the

area of skill acquisition and retention. In response to requirements by the

Deputy Chief of Staff for Training of the Army Training and Doctrine

Command (TRADOC), the long-term research goal is to develop methods for

predicting proficiency loss for all types of skills and for determining

effective training procedures for reducing this loss. The present work

represents a basic research effort completed by ARI personnel under Army

Project 2TI6110IA91B.

JEPH Z NER

e hnical Director

VV

,,Vi % ,
EFFECTS OF PRESENTATION- AND TEST-TRIAL TRAINING ON ACQUISITION AND

RETENTION OF MOVEMENT END-LOCATION

BRIEF

Requirement:

To evaluate the relative acquisition and retention effects of three

motor task training methods that differ in their emphasis on presentation

(study) and test (recall) trials.

Procedure:

Three groups of 15 participants performed 18 training trials on a

simple linear positioning motor task. Training trials were divided into

three cycles of six trials each, containing both presentation and test

trials. During presentation trials, participants studied movement endlocation by moving a sliding mechanism along a linear track until contacting

a mechanical stop that defined the end-location to be learned. During

test trials, they tried to recall the end-location by positioning the

slide without the aid of the stop.

The experiment contained an acquisition and a retention segment.

During the acquisition segment the sequence of presentation and test

trials performed within cycles differed for each training method group.

For the STANDARD group, a cycle consisted of three presentation and

three test trials administered in alternation. For the PRESENTATION

group, the first five trials of each cycle were presentation trials and

the sixth was a test trial. For the TEST group, the first trial was a

presentation trial and the next five were test trials. During the

retention segment, all participants performed a single test trial at

both 3 minutes and 24 hours after the last training trial.

Findings:

The three training methods had different effects on acquisition and

retention. Absolute (unsigned) error revealed that final acquisition

performance was best when training emphasized either repeated presentation

(PRESENTATION group) or alternation of presentation and testing (STANDARD

group). Repeated testing during training (TEST group), produced superior

long-term retention.

Utilization of Findings:

Testing during training is an effective way to improve long-term

retention of motor skill. This improvement can be achieved by changing

the emphasis of training from presentation to testing without the need

v i RAG SLAW -N OT F1 A*
for additional training resources. If instead of long-term retention,

the goal of training is rapid acquisition, training which emphasizes

either repeated presentation or alternation of presentation and testing

is most effective. Additional research is needed to determine if these

laboratory results generalize to military-related motor skills.

viii
EFFECTS OF PRESENTATION- AND TEST-TRIAL TRAINING ON ACQUISITION AND

RETENTION OF MOVEMENT END-LOCATION

CONTENTS

Page

INTRODUCTION....................................

METHOD......................................4

Subjects.................................4

Apparatus................... .............. 5

Design......................................5

Procedure...................................7

RESULTS......................................7

Acquisition...................................7

Retention.................................11

DISCUSSION.......................... .. .. ...... . .... .. .. .... 1

CONCLUSIONS....................................15

REFERENCES..................................17

DISTRIBUTION.....................................21

ix
LIST OF FIGURES

Page

Figure 1. Trial sequence for each training group

at acquisition and retention (P=Presentation;

T=Test) ....... ...................... 6

2. Mean t-trial algebraic error for the STANDARD,

PRESENTATION and TEST training groups at

acquisition and retention ...... ............ 8

3. Mean t-trial absolute error for the STANDARD,

PRESENTATION and TEST training groups at

acquisition and retention ...... ............. 9

x
EFFECTS OF PRESENTATION- ANT) . i-TRIAL TRAINING ON ACQUISITION

A;D KrmENTION OF MOVEMENT END-LOCATICN

INTRODUCTION

A long-term goal of the Army is the development of effective methods

for training Army-related skills. Of particular interest is the question

of which training methods promote the highest levels of skill acquisition

and retention. Much of the theoretical and empirical information rel .ng

to this question has originated from basic research experiments investigating

verbal and motor task learning. In these experiments, training has

involved the execution of both presentation (p) and test (t) trials.

During p-trials, participants study information to be learned, whereas

during t-trials they attempt to recall it from memory. The number and

sequential arrangement of p- and t-trials performed during training has

depended on the particular method adopted. The standard training method

has involved the alternation of p- and t-trials (e.g., Tulving, 1967;

Wrisberg & Schmidt, 1975) but other methods that emphasize either p- or

t-trial repetition have also been used (Adams & Dijkstra, 1966; Bilodeau

& Bilodeau, 1958; Hogan & Kintsch, 1971; Wenger, Thompson & Bartling,

1980).

The Army's question of which training methods promote the best

acquisition and retention is difficult to answer because relevant theories

make conflicting predictions about the relative contribution of p- and

t-trials to the learning process. For example, from a traditional

learning theory viewpoint where p-trials are seen as having a beneficial

effect similar to reinforcement, training methods which repeat p-trials

are predicted to be more effective than those which repeat t-trials.

Repetition of t-trials reduces the number of reinforcement opportunities,

and therefore, should retard both acquisition and retention. From a

contemporary cognitive viewpoint, information processing activities such

as memory retrieval and internal generation of to-be-learned items are

considered important aspects of acquisition and retention (Bjork, 1975;

Dosher & Russo, 1976; Graf, 1980; Slamecka & Graf, 1978). Because ttrials provide an opportunity to practice these activities on information

studied during p-trials, training methods which repeat t-trials should

also enhance acquisition and retention.

Examination of the empirical evidence related to the issue of which

training methods are most effective also reveals inconsistencies when

both motor and verbal task learning are considered. Most of this evidence

has come from research on verbal task learning. In general, investigators

have found that p-trial repetition during training produces superior

acquisition (Hogan & Kintsch, 1971; Thompson, Wenger & Bartling, 1978,

Exp 1I), whereas t-trial repetition during training produces superior

retention (Allen, Mahler & Estes, 1969; Hogan & Kintsch, 1971; Raye,

Johnson & Taylor, 1980; Rosner, 1970; Thompson, et. al., 1978. Exp ITT).

. .. .... .. ..
The :-attern of results for motor task learning, on the other hand, has

beer, somewhat different. Generally, p-trial repetition during training

has enhanced both acquisition (Holding & Macrae, 1964) and retention

(Adams & Dijkstra, 1966), whereas t-trial repetition has had little

beneficial effect on either process. Although both subjective recall

consistency and error detection ability have developed as a function of

repeated t-trials (Newell, 1974; Seashore & Bevelas, 1941), movement

accuracy has not been found to improve (Holding & Macrae, 1967; Newell,

1976; Exp I; Thorndike, 1927) except after considerable prior t-trial

repetition (Newell, 1976, Exp 1 and III). In fact, accuracy typically

has decreased during both acquisition and retention when t-trials have

been repeated during training (Bilodeau & Bilodeau, 1958; Duffy, Montague

& Laahs, 1975). Thus, training methods stressing p-trial repetition

have had consistent beneficial effects on both motor and verbal task

performance especially at acquisition. In contrast, methods stressing

t-trial repetition have positively affected verbal task retention but

have negatively affected both motor task acquisition and retention.

Specific reasons for the differential effect of t-trial repetition

on verbal and motor task performance are difficult to pinpoint because

of the many differences that exist between the two areas of research.

One suggested reason, however, centers around the difference in information

processing activity required of persons at p- and t-trials during motor

and verbal task training. In verbal task training, p- and t-trials are

procedurally distinct and require dissimilar information processing

activities. During p-trials, for example, items to be learned are shown

to persons for study or encoding. During t-trials, these to-be-learned

items are removed and persons are required to recall or retrieve them

from memory. In motor task training, p- and t-trials are procedurally

similar and require similar information processing activities. At ptrials, persons attempt to recall a to-be-learned movement from memory.

This recall attempt is followed by knowledge of results (KR) regarding

recall accuracy typically in visual or verbal form. At t-trials, persons

are also required to recall the to-be-learned movement but KR is not

provided. Thus, motor task training requires recall at the execution of

both p- and t-trials, whereas verbal task training requires study at ptrials and recall at t-trials. Since study and recall processes have

been found to differentially affect verbal acquisition and retention

(e.g., Hogan & Kintsch, 1971; Wenger, Thompson & Bartling, 1980) it is

necessary to create the same procedural environment in motor task training

to examine their relative effects on motor acquisition and retention.

Another suggested reason for the differential effects of t-trial

repetition on verbal and motor task performance stems from the difference

in retention interval lengths used to investigate the retention of each

type of task. The effects of repeated t-trial training on verbal task

retention have been examined primarily using long-term retention intervals

(e.g., Allen, et. al., 1969), whereas short-term retentions intervals

typically have been used in examining t-trial effects on motor task

retention (Duffy, et.al., 1975; Stelmach & Bassin, 1971). Because the

2
effect of repeated t-trials on motor task retention may vary with interval

length as it does for verbal retention (Hogan & Kintsch, 1971; Thompson,

et. al., 1978, Exp Ill; Wenger, Thompson & Bartling, 1980) meaningful

comparisons between t-trial repetition effects on verbal and motor task

retention have been difficult to make.

The present experiment examined the relative effectiveness of different

motor training methods under acquisition and retention conditions similar

to those used in verbal learning experiments. The general approach was

to allow either repetition or alternation of p- and t-trials prior to a

given t-trial during training and to compare the relative effects of

this variation on both the acquisition and retention of motor skill.

Consistent with previous laboratory work, the motor task chosen for

examination was linear positioning. Participants were required to move

a sliding mechanism along a linear track and to learn and remember the

end-location (final stopping position) of the movement performed. In

order to create acquisition and retention conditions similar to tnose

used in verbal task training, the present experiment differed from other

motor task training experiments in three ways. First, training procedures

were designed so that p-trials required study and t-trials required

recall of movement end-location. This was accomplished by using an

experimenter-defined movement procedure at p-trial execution and a

learner-defined movement procedure at t-trial execution. Experimenterdefined p-trial movements were performed with the aid of a mechanical

stop which was prepositioned by the experimenter along the linear track

to define end-location. Participants moved the sliding mechanism along

the track until contacting this stop. In doing so, they performed, and

thereby studied, the actual movement end-location to be learned. Learnerdefined t-trial movements, on the other hand, were performed with the

mechanical stop removed. This ensured recall of the to-be-learned

movement end-location from memory. Second, p- and t-trial training

effects were measured over both short- and long-term retention intervals.

This allowed for examination of the potential interaction between training

method and retention interval length and permitted a more meaningful

comparison of repeated t-trial training effects on verbal and motor

retention. Third, training was restricted to the kinesthetic cue of

end-location. Although multiple cues (such as distance, end-location

and force) underlie the recall of positioning movements (Hagman & Williams,

1977; Gundry, 1975), training was restricted to the specific cue of endlocation to prevent both the possibility of unsystematic participant

selection of individual cues during training and the possibility that

certain cues might react differently to p- and t-trial repetition

because of their differential retention characteristics (Laabs, 1973;

Posner, 1967).

3
Based on the results of recent research, predictions can be made

regarding the relative effects of p- and t-trial training methods under

the present experimental conditions. This research has examined the

relative retention characteristics of learner-defined and experimenterdefined movement information (e.g., Kelso, 1977; Stelmach, Kelso &

Wallace, 1975). The findings have been that both distance and endlocation cues are better retained when generated under a learner-defined

mode (with stop absent) than under an experimenter-defined mode (with

stop present). This superior movement cue retention has been suggested

to be caused by either (1) increased ability to preprogram efferent

commands to the muscles, or (2) by more accurate coding of kinesthetic

information under a learner-defined mode which allows prediction of

distance and end-location prior to movement initiation and termination

(Kelso, 1977; Stelmach, Kelso & Wallace, 1975).

Repetition of t-trials during training should force participants to rely

more on learner-defined t-trial end-location than on experimenterdefined p-trial end-location for accurate recall. Because of the

superior retention of learner-defined information, retention of endlocation should be best when t-trials are repeated during training. In

contrast, p-trial repetition should produce better acquisition than ttrial repetition because at p-trials learners receive more exposure to

the correct end-location. Retention following p-trial repetition,

however, should deteriorate rapidly because end-location was generated

under experimenter-defined procedures.

The present hypothesis predicts a training method by experiment

phase interaction with repeated p-trial training producing superior

acquisition and repeated t-trial training producing superior retention

of end-location information. Alternation of p- and t-trials during

training should have an intermediate effect on both acquisition and

retention. Partial support for this predicted interaction has been

reported previously (Hagman, 1980) for movement distance information.

The present experiment examined the generality of this prediction to

movement end-location information. Separate investigation of endlocation was prompted by previous findings that distance and end-location

cues possess different retention characteristics (e.g., Laabs, 1973).

Therefore, it was thought that end-location might react differently to

p- and t-trial training method variations.

Method

Subjects

Forty-five governmental employees (33 men and 12 women) volunteered

to serve as participants in the experiment. All were members of the

professional and clerical staff of the Army Research Institute for the

Behavioral and Social Sciences.

14
Apparatus

Movements were made from left to right using a metal slide which

slid along a linear track consisting of two stainless steel rods 35

inches (88.90cm) in length. Two Thompson Ball Bushings supported the

slide on the rods which were mounted in parallel on a metal frame 4.25

inches (11.00 cm) apart. The rods were located 11 inches (27.94 cm)

above the base of the frame. The base rested on a standard table top 31

inches (78.74 cm) from the floor. A second slide was used to stop ptrial movements. This slide could be securely positioned by the experimenter

along the entire length of the steel rods. A pointer attached to the

experimenter's side of each sliding element ran along a meter stick

calibrated in millimeters to indicate slide position. Additional apparatus

included a chin rest to control head movements and body position; earphones

through which participants heard tape-recorded procedural instructions;

and a blindfold to prevent participants from using visual cues during

the experiment.

Design

The experiment contained an acquisition and a retention segment as

shown in Figure 1. The acquisition segment consisted of 18 training

trials divided into three cycles of six trials each. Each cycle contained

p- and t-trials. P-trials were experimenter-dtfined movements during

which participants contacted the mechanical stop which defined the

movement end-location to be learned and remembered. T-trials were

learner-defined movements during which participants attempted to recall

(reproduce) the studied end-location with the stop removed. The sequence

of p- and t-trials within cycles differed for each of three training

groups. For Group STANDARD, a cycle consisted of three p- and three ttrials administered in an alternating sequence. For Group PRESENTATION,

the first five trials of each cycle were p-trials and the sixth was a ttrial. For Group TEST, the first trial was a p-trial and the next five

were t-trials. Training was such that a t-trial occurred every sixth

trial for all three groups. As a result, a 3 x 3 mixed factorial design

was used to examine acquisition performance with the between-subjects

variable being Groups (STANDARD, PRESENTATION, TEST) and the withinsubjects variable being Trials (6, 12, 18).

The retention segment of the experiment consisted of a single ttrial performed at both 3 minutes and 24 hours after Trial 18 of acquisition.

Performance at Trial 18 was used to evaluate immediate recall accuracv

and incorporated into a 3 x 3 mixed factorial design used to analyze

retention. In this design, the between-subjects variable was Groups

(STANDARD, PRESENTATION, TEST) and the within-subjects variable was Time

of Recall (Immediate, 3 minutes, 24 hours). Fifteen participants were

assigned randomly to each of the three groups with the constraint that

each group contained the same number of men and women.

.... .5m .. . . . . .. I 1 I i ..... . J l= I [m . . . .
H--

LUL

r'cr

LU .)

-J-

-I-

()) 0-z

0

Iia:

UNl H-- H- H-C

- 0-
Procedure

At the beginning of the experiment, participants were instructed to

learn and remember end-location. They were shown a written copy of the

entire p- and t-trial command sequence appropriate to their training

group and told the meaning of each command that they would be hearing.

The p-trial command was "Movement" and the t-trial command was "Recall

Movement." Each of these commands were preceded by "Ready" and followed

by "Rest." At "Ready" the experimenter grasped the participant's right

hand and placed it on the handle of the slide. Five seconds later,

participants heard either "Movement" or "Recall Movement" depending on

their training group. At the "Movement" command, they moved the slide

across the track at a moderate pace until contacting the stop. At the

"Recall Movement" command, participants moved the slide along until they

felt that they had moved it to the correct end-location. Five seconds

were allowed for execution of p- and t-trial movements. During this

interval, white noise was delivered through the earphones to eliminate

auditory cues resulting from displacement of the slide. "Rest" marked

the start of a 10 second time interval during which participants renoved

their hand from Lhe slide and placed it in a predetermined resting place

on the table. During rest periods the experimenter recorded recall

accuracy to the nearest millimeter and repositioned either the slide

alone or both it and the mechanical stop in preparation for the next

trial. After "Rest," participants heard "Ready" and the sequence (f

commands for the next trial began. During the retention segment of the

experiment, intervals of 3 minutes and 24 hours were inserted between

"Rest" and "Ready." Participants were asked not to count while moving

the slide and shown the approximate movement speed (125 mm/sec) desired

by the experimenter. Prior to making the first movement, participants

donned their blindfold and earphones and were given an opportunity to

move the slide to get a feel for its basic movement ci.aracteristics.

Results

Algebraic (signed) and absolute (unsigned) error scores were recorded

for each t-trial performed during the acquisition and retention segments

of the experiment. Algebraic error was used to reveal the direction

(i.e., overshooting versus undershooting of the criterion) of recall

error, whereas absolute error was used to reveal recall error without

respect to direction. Each performance measure was analyzed separately.

Acquis it ion

Mean algebraic and absolute error scores for acquisition t-trials

are shown on the left in Figures 2 and 3. Initial statistical analyses

were restricted to the scores on those t-trials which coincided temporally

for all three training groups, i.e., Trials 6, 12, and 18. These scores

were analyzed using a Groups (STANDARD, PRESENTATION, TEST) by Trials

(6, 12, 18) mixed factorial analysis of variance.

7
z=

0 -

'LA

0

z

'-4

0

C.

0 'L cn

>-

L&JO

= ia

0 c,

$4 ca

I-w

'0 00

P 6u o -4 aa

I wo

U.'

(WWu) UOUN3 31VU91V NV3WP
cLA

2M

I-C

CO

z

C4

CV)

U* 0

w .- u La ,-(~ LAJ 4.j

4..4

C-CL

0 0

- c"MM

0 . (C-H

-4

cv))

C14

cm c

(ww) WOUN3 31lOS8V NV3IN
Analysis of algebraic error revealed no significant main effects or

interactions. The rejection region for this and all future analyses was

.05. Although there were no differences between groups, visual inspection

of Figure 2 suggested an increase in error for the TEST group across

trials within Cycle I but not within Cycles 2 and 3. A subsequent Cycle

(1-3) by Trials (1-5) analysis of variance, however, performed on all ttrial scores for the TEST group showed this observation to be unreliable

although the overall cycles effect did approach significance F (2,28) =

2.65, 05. p e.10. Thus, it can be concluded that algebraic error at

acquisition did not differ as a function of training method.

Training method variation did affect absolute error, as shown in

Figure 3. An analysis of variance revealed significant main effects of

Trials, F (2,84) = 6.16, and Groups, F (2,42) = 8.07 with no significant

interaction. Individual comparisons by the least significant difference

(LSD) methcd (Carmer & Swanson, 1973) revealed that the groups effect

resulted from the TEST group displaying greater error than either the

STANDARD, LSD (42) = 14.62, or PRESENTATION group, LSD (42) = 18.71,

whereas error for these latter two groups did not differ significantly.

Thus, p-trial repetition and p- and t-trial alternation during training

produced better acquisition performance than t-trial repetition during

training. Because TEST and STANDARD group performance was almost identical

at Trial 2 (the first t-trial), the inferior acquisition performance of

the TEST group at the end of each training cycle cannot be attributed to

differences among groups present at the start of training. Unfortunately,

PRESENTATION group performance at Trial 2 could not be measured because

of the training trial sequence used. However, one can be reasonably

confident that its performance level at the start of training was comparable

to that of the other groups based on the relatively homogeneous participant

sample and the random allocation of participants to groups.

Inspection of TEST group performance at all t-trials revealed a

serrated acquisition curve typified by alternation of increased error

within-cycles followed by decreased error between-cycles. To determine

the reliability of the within-cycle changes, a Cycle (1-3) by Trials (I5) analysis of variance was performed on all TEST group t-trial scores.

As expected, the Trials effect was significant, F (4,56) = 5.28 as was

the Cycles effect, F (2,28) = 3.83. Individual comparisons showed a

decrease in error from Cycle I to 2 but not from Cycle 2 to 3. The

trials effect was the result of error being greater on the last four ttrials of each cycle than on the first t-trial of each cycle. This

increase in error across repeated t-trials is consistent with the results

of previous motor (Duffy, et.al., 1975; Hagman, 1980) and verbal learning

studies (e.g., Bregman & Wiener, 1970) using repeated t-trials during

training.

To examine the decrease in error between cycles, group performance

was compared before within-cycle error had a chance to develop for the

TEST group. This required an additional Groups by Trials analysis of

variance in which STANDARD and PRESENTATION group error at the end of

i0
each cycle, i.e., Trials 6, 12, and 18 was compared with TEST group

error at the beginning of each cycle, i.e. Trials 2, 8, and 14. Although

the trials effect was still significant with this analysis, F (2,84) =

13.83, the groups effect was not. The lack of a groups effect indicated

that between-cycle error decreases offset the significant increase in

error found within-cycles. Thus, TEST group error was greater than that

of the other groups only after within-cycle error had a chance to increase

with t-trial repetition. From the preceeding analyses, it can be concluded

that t-trial repetition during training produced both positive and

negative effects on acquisition. The negative effects took the form of

increased within-cycle error,while the positive effects took the form of

decreased between-cycle error. Both of these effects are consistent

with the findings of previous verbal (e.g., Izawa, 1970; Tulving, 1967)

and motor learning experiments (Hagman, 1980).

Retention

Mean algebraic and absolute error scores obtained during the retention

phase of the experiment are shown on the right side of Figures 2 and 3.

Retention was examined using a Groups (STANDARD, PRESENTATION, TEST) by

Retention Interval (Immediate, 3 minutes, 24 hours) analysis of variance.

Performance at Trial 18 was included in the analysis to indicate immediate

recall at the end of acquisition.

Consistent with the findings for acquisition, no significant retention

effects were found using algebraic error. in contrast, absolute error

revealed a significant main effect of Retention Interval, F (2,84)

12.53, and a Retention Interval by Group interaction, F (4,84) = 7.03.

Further analyses were restricted to examination of the interaction

effect. As shown in Figure 3, the interaction resulted from an increase

in error across retention intervals for the STANDARD and PRESENTATION

groups and a decrease in error for the TEST group. Individual comparisons

of simple main effects showed that at the end of acquisition (Trial 18)

the TEST group displayed greater error than either the STANDARD, LSD

(84) = 15.53 or the PRESENTATION group, LSD (84) = 21.73, and that no

differences existed between the error scores of the latter two groups

(i.e. TEST>PRESENTATION = STANDARD). Three minutes after acquisition,

the only significant change in group performance was an increase in

error for the PRESENTATION group, LSD (84) = 13.93. Although visual

inspection of the retention scores suggests an increase in error for the

STANDARD group and a decrease in error for the TEST group over the 3

minute retention interval, both performance changes were nonsignificant.

As a result of these numerical changes, however, no differences in error

were found among the three groups 3 minutes after acquisition (i.e.,

TEST = PRESENTATION = STANDARD). Between 3 minutes and 24 hours after

training, error increased significantly for both the PRESENTATION and

the STANDARD groups with LSD (84) = 23.47 and 20.53 for each group,

respectively. In contrast, TEST group error did not change significantly

over this same time interval. As a result, 24 hours after acquisition

TEST group error was less than that of the STANDARD, LSD (84) = 18.2,

11
and the PRESENTATION group, LSD (84) = 24.8, whereas the error for these

latter two groups did not differ significantly (i.e., STANDARD = PRESENTATION

TEST). Thus, emphasis on p-trial repetition or p- and t-trial alternation

during training resulted in rapid and extensive forgetting, whereas ttrial repetition during training prevented forgetting from occuring.

These results show the benefit of examining training method effects over

both short- and long-term retention intervals.

DISCUSSION

The three motor task training methods examined in the present experiment

produced differential acquisition and retention performance in terms of

absolute error. Performance at acquisition improved in the usual negatively

accelerated manner when training emphasized p-trial repetition or p- and

t-trial alternation with both methods being equally effective. When

training emphasized t-trial repetition, acquisition performance was

irregular and characterized by increased within-cycle error followed by

decreased between-cycle error. Because of the irregular nature of the

TEST group's performance, differences among groups did not occur uniformly

throughout acquisition and the relative superiority of the three training

methods depended on where performance was compared during acquisition.

When comparisons were made before within-cycle t-trial repetition inflated

TEST group error, no performance differences were found among the three

groups. When comparisons were made after the TEST group had completed

t-trial repetition within-cycles, the groups performing p-trial repetition

and p- and t-trial alternation displayed superior acquisition performance.

This was the case at the end of training.

Of particular interest was the question of why t-trial repetition

produced both increased within-cycle error and decreased between-cycle

error. Within-cycle error could have occurred for at least two reasons.

First, participants may have been attempting to reproduce an ever decaying

memorial representation of movement end-location established during

prior p-trial execution. As the time interval between p- and t-trial

execution increased with successive t-trial repetitions, recall got

progressively worse. Although a decay interpretation is consistent with

earlier research findings for distance information (Hagman, 1980), its

validity for end-location information appears low in that end-location

is relatively resistent to decay (e.g., Laabs, 1973). A second and more

probable explanation is that within-cycle error increased due to interference

produced by t-trial repetition. Although t-trials were not intended to

interfere with acquisition, their end-location usually was different

than that of the criterion end-location due to inaccurate participant

recall. Because of the difference between recalled end-location and

criterion end-location, t-trials possessed the potential for producing

interference (Craft . Hinrichs, 1971; Patrick, 1971; Hagman, 1978).

Thus, increased witnin-cycle error could have been caused by interfering

movements occuring at t-trial repetition. This notion also receives

indirect support from the results of verbal task learning studies showing

12
that prior recall can interfere with subsequent recall (e.g. Roediger &

Schmidt, 1980). One potential way of reducing the within-cycle error

increases would be to introduce verbal KR regarding error direction and

magnitude after each t-trial. Such a method might both improve acquisition

by reducing within-cycle error increases and maintain the superior

retention benefits associated with a repeated testing strategy. Future

research will be directed toward examination of this issue.

Reasons why t-trial repetition produced such large between-cycle

error decreases during acquisition are not readily apparent. Verbal

researchers have suggested that one function of testing via recall is to

enhance a person's ability to recognize past words recalled (Klee &

Gardiner, 1976). Perhaps, the same effect of testing via t-trials

occurs for iotor tasks. After repeated t-trial attempts, persons may

know more about their recall performance than persons who do not perform

repeated t-trials. If this notion is correct, participants in the TEST

group may have been better able to discriminate their recalled endlocation from that of the criterion end-location, and thus, may have

been more capable of making the appropriate adjustments needed to improve

accuracy. Although speculative, the notion that t-trial repetition

improves discrimination ability is consistent with previous motor research

findings (Newell, 1974). In essence, t-trial repetition may serve to

potentiate, or increase, the effectiveness of subsequent p-trial execution.

Although this type of potentiation has been reported to occur in verbal

task learning (Izawa, 1970) it has only been hinted at in motor task

learning (Bilodeau & Bilodeau, 1958; Hagman, 1980; Henderson, 1977).

Clearly, more research is needed before a firm conclusion can be made

regarding the potentiating effects of t-trial repetition in motor task

learning. One approach suggested earlier (Hagman, 1980) would be to

vary the number of t-trials repeated prior to p-trial execution. If

potentiation does occur, one should find that increased prior t-trial

repetition improves performance following a subsequent p-trial.

Differential effects of training method were also evident at retention

testing. Forgetting was rapid and extensive when p-trials were repeated

and when p- and t-trials were alternated during training. In contrast,

t-trial repetition prevented forgetting and produced better relative

long-term retention than either of the other two training methods.

An answer to the question of why t-trial repetition promotes long-term

retention relies on the distinction between experimenter-defined and

learner-defined movements and the types of information available to

persons under each movement mode. in an experimenter-defined mode,

where end-location is not known prior to movement termination, persons

base their recall on kinesthetic information originating from muscles

and joints (Goodwin, McCloskey & Matthews, 1972; Marteniuk & Roy, 1972.

In a learner-defined mode, where end-location can be predicted prior to

movement initiation or termination, persons are able to rely not only on

kinesthetic cues but also on efferent commands to the muscles (Jones,

1974). This creates the possibility of preprogramming of movement endlocation (Stelmach, et. al., 1975) and implies a greater involvement of

central rather than peripheral mechanisms in movement reproduction.

13
Prediction of movement end-location might also promote coding of kinesthetic

information. This could occur under a corollary discharge process

(Sperry, 1950; Teuber, 1974), where cortical sensory centers are preset

prior to movement initiation to improve reception of kinesthetic information.

Opportunities for preprogramning and more effective kinesthetic coding

have both been suggested to explain superiority of learner-defined

movement retention (e.g., Kelso, 1977; Stelmach, et. al., 1975).

In the present experiment, repeated t-trial execution during training

may have produced superior long-term retention because t-trials were

learner-defined. Emphasis on t-trials during training forced participants

to rely on retention of their t-trial recall performance rather than

their p-trial study performance to support later recall attempts. In

contrast, p-trial repetition during training may have forced participants

to rely more on experimenter-defined information which is forgotten

rapidly over time. Consistent with the above interpretation, p- and ttrial alternation during training should have produced effects intermediate

to those found for p-trial repetition and t-trial repetition. This was

found although the effects were not significant. Support for this

notion, however, has been reported in an earlier experiment examining

retention of movement distance information (Hagman, 1980).

Other interpretations could also be offered to account for the

present results. First, one could argue that movement retention is

based primarily on retrieval of memorial information. T-trial repetition

gave participants an opportunity to practice and improve retrieval, and

thereby eliminated or reduced long-term retention losses due to ineffective

retrieval. This interpretation receives some support from verbal learning

studies showing that subjects can learn to improve their retrieval

(Halff, 1977). Also, the findings of hypermnesia (the increase of

recall over repeated recall trials) using pictured materials (e.g.,

Erdelyi, Bischke & Finkelstein, 1977) support this argument. Second, it

could be argued that more elaborate coding of movement information

occurs under conditions of t-trial interference during acquisition and

this facilitates both retention and transfer. In the present experiment,

repeated t-trials produced interference during acquisition and may have

improved the coding of end-location at p-trial execution. This argument,

although somewhat more formalized elsewhere (Battig, 1979) is not unlike

the potentiation interpretation suggested earlier. lastly,

it could be suggested that added variety of movement per se under ttrial repetition improved retention because of enhanced schema developme:nt.

The schema is an abstraction of movement characteristics which develops

over practice as a function of variability during training (e.g. , Schmidt

1975), and its strength is postulated to directly affect retention and

transfer. In the present experiment, variability was highest under a ttrial repetition method because participants were inaccurate in their recall

[his variability may have caused the superior retention. A simple

experiment could tc conducted to distinquish between thc variabalitv

interpretation and the earlier interpretation of the rcsults based on

the distinction between expurimentet -defined and learntr-it, lined movmnt.

14'
This experiment would compare task retention of the t-trial repetition

group with that of another group yoked to the t-trial repetition group's

recall performance. The yoked group would perform experimenter-defined

p-trial movements identical to the learner-defined t-trial movements

performed by the t-trial repetition group. Under this procedure both

groups would receive the same movement variability during training but

would differ as to the movement mode (learner-defined versus experimenterdefined) under which it was generated. If the t-trial repetition group

is superior to the yoked group, then support would be provided for the

notion that learner-defined movements are easier to remember than experimenterdefined movements for movement variability would be identical for each

group. Future experiments will examine this issue.

CONCLUSIONS

Results of the present experiment reveal the relative effects of

presentation and testing on motor task acquisition and retention. In doing

so, they address the Army's question of which specific training methods

effectively promote the highest levels of motor skill proficiency. It can be

concluded that:

(1) Final acquisition performance is Setter when training emphasizes

either repeated presentation or alternation of presentation and testing rather

than repeated testing;

(2) The primary benefit of repeated testing during training is enhanced

long-term retention. Pelative to other training methods, long-term retention of

motor skill is improved substantially by a training method which emphasizes

repeated testing. Thus, testing during training not only demonstrates what

has been learned during presentation but also contributes to learning and

retention processes;

(3) The retention benefits derived from repeated testing can be achieved

by changing the emphasis of training from presentation to testing. This

could be done without the negative aspects of additional expcnditures in time,

money and personnel;

(4) The effect of presentation and testing during training is similar for

both verbal and motor tasks. This is true when presentation and testing procedures employed during training are similar for both kinds of tasks;

(5) Future research should be directed toward determining; (a) the specific

reason or reasons why t-trial repetition during training enhances retention, and

(b) whether the benefits of repeated testing found in the laboratory will generalize to military-related tasks. Of particular interest should be the investigation

of procedural tasks. These tasks require execution of successive simple motor

movements in a serial fashion and are characteristic of many motor tasks performed

within the Army.
Re ferences

Adams, J. A., & Dijkstra, S. Short-term memory for motor responses.

Journal of Experimental Psychology, 1966, 71, 314-318.

Allen, G. A., Mahler, W. A., & Estes, W. L. Effects of recall tests

on long-term retention of paired associates: Journal of Verbal

Learning and Verbal Behavior, 1969, 8, 463-471.

Battig, W. F. The flexibility of human memory. In L. S. Cermak & F. I.

M. Craik (Eds.), Levels of processing and human memory. Hillsdale,

N. J.: Erlbaum, i978.

Bilodeau, E. A., & Bilodeau, 1. Mc D. Variable frequency of knowledge

of results and the learning of a simple skill. Journal of

Experimental Psychology, 1958, 55, 379-383.

Bjork, R. A. Retrieval as a memory modifier: An interpretation of

negative recency and related phenomena. In R. L. Solso (Ed.),

Information processing and cognition the Loyola symposium, Hillsdale

N. J.: Erlbaum, 1975.

Bregman, A. S., & Wiener, J. R. Effects of test trials in pairedassociate and free-recall learning. Journal of Verbal Learning

and Verbal Behavior, 1970, 9, 689-698..

Carmer, S. G., & Swanson, M. R. An evaluation of ten pair-wise multiple

comparison procedures by Monte Carlo methods. Journal of the

American Statistical As'zciation, 1973, 68, 66-74.

Craft, J. L., & Hinrichs, J. V. Short-term retention of simple motor

response: Similarity of prior and succeeding response. Journal

of Experimental Psychology, 1971, 87, 297-302.

Dosher, B. A. & Russo, J. E. Memory for internally generated stimuli.

Journal of Experimental Psychology: Human Learning and Memory,

1976, 2, 633-640.

Duffy, T. M., Montague, W. E., & Laabs, G. J. The effect of overt

rehearsal on motor short-term memory. Journal of Motor Behavior,

1975, 7, 59-63.

Erdelyi, M., Buschke, H., & Finkelstein, S. Hypermnesia for Socratic

stimuli: The growth of recall for an internally generated memory

list abstracted from a series of riddles. Memory -nd Cognition,

1977 5, 283-286.

Goodwin, G. M., McCloskey, D. I., & Matthews, P. B. C. The contribution

of muscle afferents to kinesthesia shown by virbration induced

illusions of movement and by the effects of paralyzing joint afferents.

Brain, 1972, 95, 705-748.

17 -NOT F7 jo =
Gundry, J. The use of location and distance in reproducing different

amplitudes of movement. Journal of Motor Behavior, 1975, 7, 91-100.

Graf, P. Two consequences of generating: Increased inter- and intraword

organization of sentences. Journal of Verbal Learning and Verbal

Behavior, 1980, l9, 316-327.

Hagman, J. Specific-cue effects of interpolated movements on distance

and location retention in short-term motor memory. Memory and

Cognition, 1978, 6, 432-437.

Hagman, J. D. Effects of Presentation- and Test-Trial Training on Motor Acquisition and Retention. (Technical Report 431) Alexandria,

VA: US Army Research Institute for the Behavioral and Social Sciences,

January, 1980.

Hagman, J. D., & Williams, E. Use of distance and location in short-term

motor memory. Perceptual and Motor Skills, 1977, 44, 867-873.

Halff, H. M. The role of opportunities for recall in learning to retrieve.

American Journal of Psychology, 1977, 90, 383-406.

Henderson, S. E. Role of feedback in the development and maintenance

of a complex skill. Journal of Experimental Psychology: Human

Perception aad Performance, 1977. 3, 224-233.

Hogan, R. M. & Kintsch, W. Differential effects of study and test

trials on long-term recognition and recall. Journal of Verbal

Learning and Verbal Behavior, 1971, 10, 562-567.

Holding, D. H., & Macrae, A. W. Guidance, restriction and knowledge of

results. Ergonomics, 1964, 7, 289-295.

Izawa, C. Optimal potentiating effects and forgetting-prevention effects

of tests in paired-associate learning. Journal of Experimental

Psychology, 1970, 83, 340-344.

Jones, B. Role of central monitoring of efference in short-term memory

for movements. Journal of Experimental Psychology, 1974, 102,

37-43.

Kelso, J. A. S. Planning and efferent components in the coding of

movement. Journal of Motor Behavior, 1977, 9, 33-47.

Klee, H., & Gardiner, .1. M. Memory for remembered events: Contrasting

recall and recognition. Journal of Verbal Learning and Verbal

Behavior, 1976, 15, 471-478.

18
Laabs, G. J. Retention characteristics of different reproduction cues

in motor short-term memory. Journal of Experimental Psychology,

1973, 100, 168-177.

Marteniuk, R. G., & Roy, E. A. The codability of kinesthetic location

and distance information. Acta Psychologica, 1972, 36, 471-479.

Newell, K. M. Knowledge of results and motor learning. Journal of

Motor Behavior, 1974, 6, 235-244.

Newell, K. M. Motor learning without knowledge of results through the

development of a response recognition mechanism. Journal of

Motor Behavior, 1976, 8, 209-217.

Patrick, J. P. The effect of interpolated motor activities in short-term

motor memory. Journal of Motor Behavior, 1971, 3, 39-48.

Posner, M. I. Characteristics of visual and kinesthetic memory codes.

Journal of Experimental Psychology, 1967, 75, 103-107.

Raye, C. L., Johnson. M. K., & Taylor, T. H. Is there something special

about memory for internally generated information? Memory and

Cognition, 1980, 8, 141-148.

Roediger, H. L., & Schmidt, S. R. Output interference in the recall of

categorized and paired-associate lists. Journal of Experimental

Psychology: Human Learning and Memory, 1980, 6, 91-105.

Rosner, S. R. The effects of presentation and recall trials on organization

in multi-trial free recall. Journal of Verbal Learning and Verbal

Behavior, 1970, 9, 69-74.

Schmidt, R. A. A schema theory of discrete motor skill learning.

Psychological Review, 1975, 82, 225-260.

Seashore, R. H., & Bevelas, A. The functioning of knowledge of results

in Thorndikes line drawing experiment. Psychological Review,

1941, 48, 155-164.

Shea, J. B., & Morgan, R. L. Contextual interference effects on the

acquisition, retention, and transfer of a motor skill. Journal

of Experimental Psychology: Human Learning and Memory, 1979, 5,

179-187.

Slamecka, N. J., & Graf, P. The generation effect: Delineation of a

phenomenon. Journal of Experimental Psychology: Human Learning and

Memory, 1978, 4, 592-604.

Sperry, R. W. Neural basis of the spontaneous optokinetic response produced

by visual neural inversion. Journal of Comparative and Physiological

Psychology, 1950, 71, 482-489.

19
Stelmach, G. E., & Bassin, S. L. The role of overt motor rehearsal in

kinesthetic recall. Acta Psychologica, 1971, 35, 56-63.

Stelmach, G. E., Kelso, J. A., & McCullagh, P. D. Preselection and

response biasing in short-term memory, Memory and Cognition, 1976,

4, 62-66.

Stelmach, G. E., Kelso, J. A., & Wallace, S. A. Preselection in

short-term memory. Journal of Experimental Psychology: Human

Learning and Memory, 1975, 1, 745-755.

Teuber, H. L. Key problems in the programming of movements. Brain

Research, 1974, 71, 535-568.

Thompson, C. P., Wenger, S. K., & Bartling, C. A. How recall facilitates

subsequent recall: A reappraisal. Journal of Experimental Psychology:

Human Learning and Memory, 1978, 4, 210-221.

Thorndike, E. L. The law of effect. American Journal of Psychology, 1927,

39, 212-222.

Tulving, E. The effects of presentation and recall of material in free recall

learning. Journal of Verbal Learning and Verbal Behavior, 1967, 6, 175-184.

Wenger, S. K., Thompson, C. P., & Bartling, C. A. Recall facititates

subsequent recognition. Journal of Experimental Psychology: Human

Learning and Memory, 1980, 6, 135-144.

Wrisberg, C. A., & Schmidt, R. A. A note on motor learning without

post-response knowledge of results. Journal of Motor Behavior,

1975, 7, 221-225.

20
DISTRIBUTION

2 HGOA RESEARCH AND STUDIES OFC

22 ARI iIAISON OFFICE

I INTEw-UNIV SEMINAR ON ARMLU FORCES + SOC

I OASA (RDA) DEPUTY FOR SCIENCE AND TECHNOLOGY

i OFC OF NAVAL RESEARCH /

I AFHRi/LRT

I AFHRI/LRLG

I AIR .ORCE HUMAN RESOURCES LAB ATTN: AFHRL/TSR

I NAVY PERSONNEL q AND D CENTER DIRECTOR OF PROGRAMS

1 NAVY PERSONNEL R AND 0 CENTER /

2 OFC uF NAVAL RESEARCH PEHSONNFL AND TRAINING RESEARCH PROGRAMS

I NAVAi PERSONNEL R + U CENTER /

I OFC oF NAVAL RESEARCH PROJECT OFFICER* ENVIRONMENTAL PHYSIOLOGY

I NAVAI AEROSPACE MEDICAL RSCH LAB AIRBORNE RANGER RESEARCH

I NAVAi AEROSPACE MEDICAL RS(,H LAb AEROSPACE PSVCHOLOGY DEPARTMENT

I HEADwUARTERS, COAST GUARD CHIFF, PSYCHOLOGICAL RSCH RP

1 USA kESEARCH AND TECHNOLU~iY LAB ATTN: DAVDL-AS (DR* R. S. DUNN)

1 NAFEc. HUMAN ENGINEERING bkANCH

I HO WwAIR DIV OF 4EUROPSYCHIATRY

I USA oSCH DEVEL * STANDARDIZA GP, UoK.

I USA wESEARCH AND OEVELOPMENT LAbS CHIEF, BEHAV SCIENCES DIV 9 FOOD SCI LAB

I UNIFORMED SERVICES UNIT OF THE HEALTH SCI DEPARTMENT OF PSYCHIATRY

I CENT'R FOR NAVAL ANALYSIS

I NAVAi HEALTH RSCH CEN LIBRARY

I NAVAI PERSONNEL R AND U CLN LIBRARY ATTNI CODE P106

I AIR FORCE HUMAN RESOURCES LAb ATTN: AFHRL/OTS

I USA ACADEMY OF HEALTH SCILNCES STIMSON LIeRAR* (DOCUMENTS)

I ERIC PROCESSING AND REFERENCE FAC ACQUISITIONS LIBRARIAN

I USMA DEPT OF BEHAVIORAL SCI ANU LEADERSHIP

I OLD )jOMINION UNIVERSITY PLRFURMANCE ASSESSMENT LABORATORY

I NMRDc PROGRAM'MANAGER FOR HUMAN PERFORMANCE

I U OF TEXAS CEN FOR COMMUNICATION RSCH

I INSTITUTE FOR UEFENSE ANALYSES

I AFHRI TECHNOLOGY OFC (H)

I PURDuE UNIV DLPT OF PSYCHOLOGICAL SCIENCES

I USAFA DEPT OF BEH SCI + LEADERSHIP

I US MILITARY ACAtE14Y UEPT. OF HISTORY, BLDG 601

1 USA INTELLIGENCE CEN AND SCH ATTN: SCHOOL LIBRARY

I US MILITARY ACADE4Y DIRECTOR oF INSTITUTIONAL RSCM

I NAVAi POSTGRADUATE SCH ATTN: nUDLEY KNOx LIBRARY (COOF 1424)

i HQ AIC/XPTD TRAINING SYSIEMS DEVELOPMENT

I USA iIELO ARTILLERY SCHOOL MORRIS SwETT LIBRARY

I USA I'NSTITUTE OF ADMINISIHATION ACADEMIC LIBRARY

I USA wAR COLLEGE ATTN: LIBRARY

I USA tNGINEER SCHOOL LIBRARY AND LEARNING RESOWRCES CENTER

I USA ARMOR SCHOOL (USARMS) ATTN: LIBRARY

I US COAST GUARD ACADEMY LIBRARy

I ORGANIZATIONAL EFFECTIVENLSS CEN * SCH ATTN: LIBRARIAN

? BRITISH EMBASSY BRITISH UEFENCE STAFF,

I CDLS (W) LIBRARY

P CANAWIAN FORCES PERSONNEL APPL RSCH UNIT

1 ARMY PERSONNEL RESEARCH ESTABLISHMENT

6 LIBRARY OF CONbRESS EXCHANGE AND GIFT DIV

i DEFENSE TECHNICAL INFOkMA1ION CEN ATINI DTIC-DDA-2

140 LIBRARY OF CONbRESS UNIT UOCUMLNTS EXPEDITING PROJECT

I US GOVERNMENT PRINTING uFC LIBRARY, PUBLIC DOCUMENTS DEPARTMENT

I US GIVERNMENT PRINTING OFL LIBRARY AND STATUTORY, LIS DIV (SLL)

I THE aRMY LIBRARY ATTN: ARMY STUUIES SEC

21