hmserch Note 85-62

ENHANCEMENTS TO NOTIVATIONAL SKILLS TRAINING

FOR MILITARY TECHNICAL TRAINING STUDENTS:

PHASE 11 EVALUATION tPII

1%. Barbara L. McCombs, Kathleen A. Lockhart,

*Kathy L. Bruce, and Gregory P. Smith

NDenver Research Institute

* University of Denver

for

Contracting Officer's Representative

Richard P. Kern

Instructional Technology Systems Technical Area

Zita M. Simutis, Chief

TRAINING RESEARCH LABORATORY

Harold F. O'Neil, Jr., Director

B~~~EC 26 M95,: -.-

U. S. Army

Research Institute for the Behavioral and Social Sciences

June 1985

7 ..

Approved for public release; distribution unlimited.

85 12 27 O t;
·•· 

THIS DOCUMENT IS BEST 

QUALITY AVAILABLE. THE COPY 

FURNISHED TO DTIC CONTAINED 

A SIGNIFICANT NUMBER OF 

PAGES WHICH DO NOT 

REPRODUCE LEGIBLYo 
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

L. NEALE COSBY

EDGAR M. JOHNSON Colonel, IN .

Technical Director Commander

.'-..*,q

6R."

TW epot fm beesn gloed for relese s t O Defense Teehnisel Information Center IDTIC). It has been given

II'--

SOWh PrhSwV dlttbutioit end will be mailabl ts roquestors only through DTIC or other reference seriess suchez

fte Nie Id Tenloed formation Be iW)8. The views, opinions, endor findings gontained in this ,_.

report we tme of the mdss(s) end shouid not be oonetrued san official D*p mwnt of the Army position.

Sw .

... .......... . . . .. .. .... =.
4*V Ck,"OPATION OF THIS PAGE (11hon bets Xtove*_______________

- ... DOMUMNTATON PAGE READ INSTRUCTIONS

______-______________A________ BEFORECOMPLETINGFORM F mpW "man VT ACIMO O .RCIPiENT'S CATALOG NUMBER

ART -Na Note 85-62

SS. TYP OF REPORT & PERIOD COVERED FOR RWLTA O MOTIVATIONAL SKILLS TRAINING Interim Report

FOR MIL]TARY TECHNICAL TRAINING STUDENTS: Feb. 1982 - Dec. 1984

PHASE 11 EVALUATION s- PERFORMING ORG. REPORT NUMBER

A. 11I s. CONTRACT OR GRANT NUMBER(a)

Barbara L. McCombs, Kathleen A. Lockhart,

Kathy L. Bruce, and Gregory P. Smith MDA 903-82-C-0169

S. PERFOININ ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

Social Systems Research and Evaluation Division, AREA EORKUNTNU..eRS

Denver Research Institute - University of Denver, 2Q162722A791

Denver Colorado 80208 _ _.__"

II. CONTROLLING OFF'ICE NAME AND ADDRESS 12. REPORT DATE

U.S. Amy Research Institute for the Behavioral June 1985

and Social Sciences, 5001 Eisenhower Avenue, M. NUMBER O-FPAGES

Alexandria, Virginia 22333-5600 99 - 1. MONITORING AGENCY NAME & ADORESS(If differen 11m Cetrllling Office) IS. SECURITY CLASS. (of thl report)

Uncl assified

15o. )ECLASSI FI CATION DOWNGRADING SCHEDULE

IS. DISTRIBUTION STATEMENT (of this Report)

Approved for public release; distribution unlimited.

17. DISTRIGUTION STATEMENT (of the abetact entered In Block 20. it different from Repo")

1. SUPPLEMENTARY NOTES

Dr. Richard P. Kern, contracting officer's representative and technical

monitor for this project. This research, reported in two volumes, is one of

five learning strategy research projects performed under the research component of the Basic Skills Resource Center contract.

19. KEY WORDS (Cotiae on rvme side It neceeary and Identify by block number)

Learning Strategies

Motivational Skills Training

Basic Skills Education --.

-ns researnnote s a part o the asic M1i s Resource Center research

component, and was undertaken to design and evaluate the effectiveness of

Computer Assisted Instruction (CAI) enhanced versions of the Motivational

Skills Training Program. This report describes the technology-based CAI

enhancements, and the results of an experimental examination of the skills

training program. Recommendations for implementation in Army Technical programs are also discussed. (See also Phase I Evaluation, RN 85-61.)

JIM 14T -- . o0rM UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAiE (When Pat Entered)

j . s..t
FOREI6OR

The Instructional Technology Systems Technical Area of the U.S. Army Research

Institute for the Behavioral and Social Sciences directs research in learning

strategies applications with a special focus on educational technology and

links to military education and training. These research and development

efforts are aimed at the overall Improvement of the Army's Basic Skills

Education Program.

This research effort included the design and d elopment of CAI enhancements

for the Motivational Skills Training Program. ' This report describes the

results of an evaluation of the effectiveness of the technology-based instruction and identifies recommendations for implementation of the training program

within Army technical training efforts. Overall, this effort was undertaken

to explore procedures to remedy motivational deficiencies related to unsatisfactory performance in military technical training and to determine if microcomputer-based instruction might be used to offset or reduce instructor

requirements associated with motivational traininq.

'Acceslin Fc~-

cap"'

INSP.C.S -.- ..- '.,---

Iii K- + + '-..
mACKNOWLEDGEMENTS

The Denver Research Institute was greatly aided by the efforts and

copraoien *I civilian and military personnel in the Electronics Communication

cuse at Ft. Sill, Oklahoma. We wish to especially express our gratitude to Mr.

BVmy Bowman, Mr. Larry Pollack, %t. Alan Arrowood, Mr. Terry Anderson,

MSt. Ben Burris, and St. 3erry Zahn, for planning, scheduling, and hosting our ,

site visits to their installation and for conducting the training and providing the -..*

information needed for this investigation. We also wish to thank Dr. Richard Kern

of the Army Research Institute for his advice and support throughout this effort.

\4.-..,.

'. -°*-=*

ell. 

..

,?!:....
TABLE OF CONTENTS

Au e aSummery .... ..... ... ...... .................. H

Flndlnp i'i

Bac ru n o o ............................. o ....... ... ... 1

Phase ! Evaluation Flndlngs ................................. 2

Nature of this Reserch .... ................. 1 -om 

Reantr~ ********e*******t* o ....... ****..... 000000000 16 

Phase 11 Evaluation Study Reults ...... ........ ............. oo ...... 2

Re!e!rc nlsinrsur ........................................ 2 2

Fina ub d eveloeto CAI/Audio. Segent .. a 0 . . . . . . . ......... 2

ExamInetion ot Peti .......o. ............................ 12

Mras ues o n........................................... 12.....-..

Sbefcacy an. Sr...nd..or .......... ............ o. .................. 347

DEplortr Iniilo Dfe reneC ompardisopns for-t

CA vs........ de ................... . s o ................ 1

a erial ConStrl Rel ts ............... ................. 23

Dhase a Conlios 1d Prcdue ....................... .. . ............. 43

Sua rybe oeluation oin ............................. ..

ssignet ofSuetst..ermna.Cniins........ 2

mPAietion ass o d res .......................................... 27

RecomFmendti of udet ImEpementatiCondos the ...... - . otition R .ogas . .. .. .... . .. ... .. ... . .................. .. 27 48-

Ex amination of Potential Covariates ...... o... ... ............. 27 .- _

Analyses of Major Dependent Variables o ..................... 27 APe-er Critical siols Thinking o eal Te nd n Set -t, 

efficacy, and Stress Indicators .hee

Exploratory Individual Difference Comparisons for theal

Historical CAI vs Control Conditions Group .36- (HC) ................. 38

Anecdotal Data ............................................. 41 .f

Discussion and Conclusions ................... ........................ 43

Summary of Evaluation Findings ........ o...................... 43

Implications of Study Results .. .............. 5"

0Recommendations of Future Implementations of the-.-.

Motivational Program ... o............................ ...... 48 . ..

References .... o....... ............................................. 51 -.- 4

Appendix A.- Critical Thinking Scills Measure and Pretest instructions '

Appendix B.: Instructor Training Materials . . Appendix C: Student Assignment Sheet,'.-

Appendix D: Descriptive Statistics for NoncateSorical Variables by Experimental -'

Group :::::

vii '::.:-
LIST OF TABLES

L Distribution ol Students by Military Rank Within

Eperismortal Condition *....... . 6*0....a...90........... 24

2. Distribution of Students by Army Component

I. Distribution ad Students by Sex W ithin

Experimental Condition .................... ** ..... *.*......... 26 '

4. Distribution of Students by Graduation

Status Within Experimental Condition .. .. ........ .. .. . .... .. .. .. . .. 28 N

5. Chi-Square Group Canmparisons for Students

Who Graduated: Military Rank and Armny ~ Component ..................................................... 29

6. T-Test Results for GropComparisons on ASVAB Variables ........... 30

7. Group Comparisons on Major DependentVaiables................... 32

8L One-tailed T-test Probabilities: Pre-Posttes-t

Comparisons of Critical Thinking, Self-ef ficacy,

and Stress Indicators ... .. . .... . .. ... . .. .. .. .. . .. ... .. .*. . . . . ... 35

9. limmary of Condition (2) by Independent

Variable (3) Significant Analysis of

Variance Interactions: CAI vs. IConditions....................... 37

10. Descriptive Statistics: Historical

LIST OF FIGURES

Figure 1. Schematic representation of the'relationship between course performance and ASVAB

general scores . ....................................... 39

Figure 2. Schematic representations of the relationship

between f allures on Tasks 1- 16 and self-ef ficacy/ self-perception pretests. .................................................... 39 T

ix
Executive Summary

proin AA sidils training program for remedying motivational deficiencies related

to unsatidactory performance in military technical training has been found to

improve the subsequent test scores, test failure rates, and/or eliminations from

technical training in two prior studies conducted with Air Force trainees. In these

studies, the seven printed, self-instructional modules of the Motivational Skills

Training Program were augmented by instructor introductions and small group

discussion and practice sessions. Results of the second study, conducted as the

first phase of this project for the Army Research Institute, indicated that the

inclusion of instructor introductions and group practice sessions for each module

contributed to the effectiveness of the motivational program. Of additional

interest to the Army, however, is the question of whether microcomputer-based

technologies might be used to offset or reduce instructor and group requirements

for this motivational training, thereby increasing the practical utility of this

training for large scale implementation within Army technical training.

Purpose

This is the second of two technical reports describing activities in the

two phases of this project, respectively. The goals of Phase I were to determine

the separate contribution of instructor augmentation and group experiences to

motivational program effectiveness and to identify specific computer-assisted

instruction (CA) enhancements for those training components which benefited

from instructor augmentation and group experiences. The goals of Phase 1 were

to design and evaluate the effectiveness of CAI enhanced versions of the

Motivational Skills Training Program and to make recommendations for future

xi .-
Implem~uwtations of the program within Army technical training. This report

dMWrOs the design of CAI enhancements and the results of an experimental

study directed at accomplishing Phase U goals.

Appoach

A simple computer-controlled audio capability which interfaces with the

Apple lie microcomputer was developed to achieve the personalization desired in

the simulation of instructor and group functions. The character "PC," created to

simulate instructor functions, was designed to enact three primary roles:

facilitator, modeler, and motivator. To provide the identified group functions of

peer identification, opportunities for shared problem solving, and peer modeling

and feedback, a set of military characters was defined. The characters were

designed to "grow" as a result of their skill training from an initial inability to

solve particular problems to competent problem solvers and self-managers. This

transition occurred between PC's guided CAI introductions and CAI practice

sessions for each module. Much of the modeling provided in the CAI segments is

accomplished via the computer-contolled audio capability, while most of the

actual skill practice is provided through the CAI exercises. The contractordeveloped audio interface consists of a specially designed interface card which

plugs into the Apple He game I/0 port. This capability allows for computer

control of a linear sequence of audio messsages that coincide with particular CAI

frame sequences, as well as provides for the personalization of skill training

introductions and practices, at about one-eighth the cost of videodisc technology..

The design for the experimental study consisted of six conditions: an

historical control group (HC), a current control group (CC), a CAI introduction ..

and practice group (CAI), a CAI introduction and instructor practice group

Xii
(CAMP an intructor Introduction and CAI practice group (CA), and an instructor

introduction and practice group (IM. Participants in the study were male and

fmwale students scheduled to begin the Electronic Communications (EC) course at

Ft. Sill, Oldahoma, during the period of 25 October 1933 through 17 February

1934. At the conclusion of the study, data on a total of 532 stuents were

available for analysis. The number of students in each condition was as follows:=

53 in the HC condition, 253 in the CC condition, 55 in the CAI condition, 61 in the

CAR condition, 53 in the ICAI condition, and 57 in the R condition. Data available

for analysis included (a) pretreatment student differences by condition in ASVAB

scores, Army component, sex, initial critical thinking skills, initial judgments of

self-efficacy, and initial indices of anxiety and ability to cope with stress; and 0)

posttreatment differences critical thirking skills, judgments of self-efficacy,

indices of anxiety and ability to cope with stress, instructor ratings of students'

self-management skills, times to complete the first and second EC course

segments, test failures in the first and second .C course segments, progres

indices for the entire EC course, and whether students attrited or graduatea.

Findinxs

Study results indicate that students who received the motivational skills

training with instructor introductions/group practice sessions (ii condition)

performed better during subsequent EC course technical training than either

students who received no motivational skills training (CC condition) or students

who received the motivational skills training via CAI with no instructor practice

(CAI and ICAI conditions). This better performance was manifested in terms of

both significantly fewer test failures and less training time. In addition, as

compared to the CC condition, students receiving either the CAI or II conditions

o . '.
tended to have lower attrition rates. Although these differences were not

statistically sigiflcant, the .2 percent reduction for the CAI condition and the I.

*.4 percent reduction for the II condition may have some practical significance in

terms of training costs. For similar students going through the EC course a year

earlier (the HC condition) attrition rates were .2 percent higher than for

students in the current study and progress indices were approximately 12 percent

higher. The higher attrition and progress index rates for the HC as compared Li

with current groups was at least in part a function of new procedures being

implemented in the student battery iousing area) to improve the performance of

students not progressing through the EC course at a satisfactory rate. Thus, it is

likely that findings with the Motivational Skills Training Program were attenuated

in the present study.

Since study findings indicated no overall superiority of the CAI vs. H.

conditions, some exploratory analyses of potential individ.ral dfferences in

subsequent student performance as a function of treatment condition were

conducted. These analyses generally indicated that the CAI condition is at least

equally effective for approximately half of the students (ie., those students of

high general ASVAB ability and those students with low perception of competence

or self-esteem). These findings imply that there may be systematic and reliable

individual differences that could be used in differential treatment format

assignments, thereby reducing some of the instructor support requirements for the

motivational program as well as capitalizing on the use of microcomputer

technology for this type of skill training. _1

In general, the Motivational Skills Training Program appears to have

achieved the goal of providing needed and relevant training and to have

Xiv

.' -
contributed to improvements in students' subsequent technical training

performance. The Phase UI evaluation findings also generally support Phase I

evaluation findings, i4, they substantiate the importance of instructor and group

experiences for this type of skills training at least for some types of students.

Additional research issues to be explored, however, include (a) an analysis of what

is learned in the motivational program, a development of well defined measures of

the identified knowledges and skills, and an exploration of relationships between

these measures of what is learned and subsequent student performance; (b)

investigations of individual differences predictive of what types of students need

the motivation program and of those who would most benefit from this training, - -

investigations of individual differences predictive of CAI vs. instructor/group

versions of the training; and (c) an analysis, identification, and evaluation of a set

• , of skill maintenance strategies that can operate within the total context of an

* individualized Motivational Skills Training Program.

-7a

4...

%a .

4%,

'4.

xv

................
Introduction

Be kfround 

4.

Not only have many students entering military technical training been

shown to have deficiencies in basic reading skills, study skills, and cognitive

learning strategies, but they have also been shown to demonstrate skill

deficiencies of an attitudinal or motivational nature (McCombs, 1982; McCombs &

Dobrovolny, 1980, 1982). These motivational skill deficiencies are generally

reflected in traineese inability to positively adjust to the requirements of military

technical training and to implement necessary self -management, personal

responsibility, and positive self-control strategies that are related to j

s f-m o tiva t io n (M c ~ o m b s , 1 9 .9 4) M o r e s p e c if ic a lly , m o t i v a t i o n a l s k ill: 

'' "-: I

deficiencies related to unsatisfactory performance in technical training have been

found to inciude insufficient goal setting and problem solving skills, inadequate

self-evaluation and planning skills, inappropriate approaches or strategies for

dealing with anxiety and stress, and ineffective communication skills (McCombs,

1982).

A program for remedying these motivational deficiencies, entitled the

~~~Motivational 

Skills Training Program, was developed by McCombs and Dobrovolny.-.:,

(1982) and evaluated with Air Force trainees. The program includes seven

self-instructional, printed modules that have been implemented in an

instructor-led, small-group format which provides trainees with the opportunity

to practice new strategies and skills, share experiences, and develop feelings of

rapport with their instructors and fellow trainees. The evaluation of this program

of self-instructional materials augmented by instructor support and group

experiences in the Air Force's Precision Measuring Equipment (PME) course

,9 .."',. .... -.,.. ...'...'.', ........ ..... .. -. .. . ., ..- .. ,.. .. .- - 1 , "
54

Indicated that trainees liked the program and found it helpful in their course work I.

and personal lives. Further, trainees participating in the program had

slgniflcantly higher block test scores and lower test failure rates than control

group trainees (McCombs & Dobrovolny, 1982). NO

Although the evaluation findings with the Motivational Skills Training

Program pointed to its success, several research questions remained. One set of

questions concerned the separate contributions of instructor augmentation and

group experiences to the effectiveness of the program, and their implications for

the identification of motivational skills training components that could be

':- -

enhanced by the use of a computer-assisted instruction (CAI) format. A second

set of questions concerned the issue of the format of this training and whether the

cost effectiveness of the program could be enhanced by reducing instructor and/or

group interaction requirements through the use of CAI for selected portions of the

training. These questions were addressed in the present research, undertaken by

the Army Research Institute. The first set of questions was addressed in a report

describing the first phase of this project (McCombs, Bruce, & Lockhart, in press).

This report addresses the second set of questions and describes the results of

Phase 1U activities.

Phase I Evaluation Findings

Activities in Phase I of this project included an experimental study which

explored the relative effectiveness of the printed modules under conditions of

varying instructor and group augmentation, and a contractor analysis of instructor

and group functions which facilitated student acquisition of concepts and skills in

the motivational program. The integration of findings from both of these

activities led to the specification of general strategies and procedures for CAT

27' - '. ... ,
'73T

A.

components to be developed that could simulate the identified instructor and

group functions.

The Phase I experimental study was conducted with a total of 120 Air

Force trainees scheduled to enter the Weapons and Armament School at Lowry

Air Fore Base. These students were assigned to one of four conditions following

their arrival on base a printed modules only condition, a modules plus instructor

introductions condition, a modules plus instructor introductions and group

discusiom condition, and a no training control condition.

Primary findings from this study were that experimental students who

received instructor introductions to each module or instructor introductions and ' .Y

group practices had lower elimination rates once they entered their technical

traiing course than students who received no motivational training (control) or

printed modules only. In addition, test failure rates were lowest for students who

received the motivational skills training with both instructor introductions and

groul practices, as compared with other experimental and control conditions. :.

These findings suggested that instructor and group experiences enhance the I-6

subsequent effectiveness of the Motivational Skills Training Program, with

implications for student performance and attrition in technical training (McCombs $ .

et al., in press).

The contractor analysis of instructor roles in the motivational training

led to the definition of three primary roles that could be provided by the

computer: modeler, motivator, and facilitator. A character named "PC" was

defined to personalize and enact these roles. The analysis of the roles and

functions of the group experiences indicated that facilitative functions included

peer identification, opportunities for shared problem solving, and peer modeling

3
amd feedbck. A set of military characters was defined to personalize these group

functions and represent specific personal responsibility and self-control problems

related to each module's content area. The CAI development guidelines specified

the use of PC and the set of military characters in CAI introductions to the

program and each module and in CAI practice sessions at the end of each module.

It was determined that introductory segments would be approximately 10 minutes

each and the practice segments would be approximately 15 minutes each, for a -

total of about 2%i hours of CAI (McCombs et al, in press).

Findings from both the experimental study and contractor analysis

suggested the importance of personalization and the- need for simulation of

instructor and group functions within a "rich" training medium. For students to

identify with and sense the human qualities of the defined instructor and group

characters, both interactive visual and audio capabilities appear necessary. To

provide these capabilities within the cost constraints and practicalities of this _

project, McCombs et al. (in press) recommended that a simple computercontrolled audio capability for the Apple lie system be developed in Phase H of

the project.

Nature of this Research

The goals of this second phase of the present project were to design and

evaluate the effectiveness of CAI enhanced versions of the Motivational Skills

Training Program and to make recommendations for future implementations of

the program within Army technical training. The selection of motivational

program components for CAI design and development was based to a large extent

on the analysis of Phase I evaluation data, as presented in the McCombs, et al. (in

press) report and summarized in the preceding section. This report, therefore,

4
smwmaizs activities related to the deulg, development, and evelustion of the

motivationAl program components selected for implementation in a CAI format.

Rele vnt Literature

The llterature selectively reviewed in this section is Intended to

supplement the literature on the role of instructors vnd the group process in

motivational skiDls traning presented In the McCombs at al. (in press) report on

Phase I evaluation finding. This literature covers both potential enhancements

that can be provided by a CAI format and suggested design strategies that can be

incorporated in the CAI components.

In discussing the use of media such as CAI for the enhancement or

replacement of conventional instructional media such as instructors or group

instruction, Clark (1983, 1984) makes the cogent point that it is not the media per

se that influence learning. Rather, it is the content and method of instruction -

that are critical and the medium is merely an alternative delivery vehicle. Clark

argues that in using the medium of computers, one must focus on available

instructional theory in finding the necessary instructional methods for fostering

the desired learning outcome. He also points out that decisions to use computers

are more a matter of implementation issues such as cost, practicality, resources,

and equity of access and that this medium can be maximized to address particular

implementation problems by focusing on the computer's special features (Clark,

1984). For the goals of this particular project, therefore, it is important to keep

in mind that in using CAI to simulate critical instructor or group process

functions, special delivery features must be carefully matched to content and

function requirements.

Some general instructional features of CAI that have been shown to be

potentially useful to students who have responded poorly to traditional

.5s.',
instructional methods include individualization, mastery learning, and self-pacing ,.

(Crom, 1970. More recent research with these instructional features,

particularly the mastery learning model, however, has raised some question about \.

the effectiveness of these features for the disadvantaged background, low ability

student (Covington & Omelich, 1981; Federico, 1981; Stinard & Dolphin, 1981;

Thomlson, 1980). A question raised by Covington and Omelich (1981) is whether .

or not instructional features which include repeated test trials and grading against

absolute standards perpetuate a negative failure cycle for low ability/low

self-confidence students. Recognition of this possibility in utilizing CAI

technology, therefore, is an important consideration in making optimum use of

this medium with military student populations.

On the more practical side, however, Siegel and Simutis (1979) point out

*that within the Army there are many problems associated with providing basic

skills training to the large number of individuals at many different locations

needing this training. These problems include inconsistent quality in the content

of basic skills courses, inconvenience of the training times for many soldiers,

inappropriate matches of skill levels and basic skills curriculum, and difficulties in

* handling the large amount of paperwork involved in a basic skills training

program. Problems such as these have led the Army Research Institute (ARI) to

explore the use of CAI for basic skills training in the Army. The advantages seen

7: in this instructional medium are the opportunities to provide standardized,

efficient, and individualized training. In initial studies to evaluate CAI for

various types of skills training, Siegel and Simutis (1979) report that CAI was at

least as effective as traditional instruction, particularly if instructors are given :7]

the opportunity to become comfortable with the new technology and are given

training in the roles required by the new technology.

*. 6
J~

An Issue of concern In the implementation of CAI with a skills training

curriculum, then, is the role instructors play in the learning process. Regardless

of whether the entire curriculum is presented via CAI or whether segments of CAI

are embedded within an off-line program, the instructor's role is key to the

success of the program. At a minimum, Burns and Davisson (1979) suggest that

instructors have input into how CAI is used and be given short inservice workshops

wherein CAI applications are explained and demonstrated. Others have made

similar points and have particularly stressed the importance of involving

instructors in the design and implementation of CAI as well as providing them

meaningful role training (e4g., Bloom, 1984; McCombs & Dobrovolny, 1980, 1982;

McCombs, Dobrovolny, & Loclart, 1983; Stasz, Winkler, Shavelson, Robyn, &

Feibel, 1984; Swing & Peterson, 1981).

Jemstedt (1983) has argued that a problem in the implementation of

individualized instruction and computer technologies is that they have been used

as replacements for teachers and for group learning. He cites evidence that (a)

teachers have a profound impact on students! cognitive and affective development

and (h) peer group interactions impact academic achievement, cognitive problem

solving skills, and intellectual self-esteem. The need for successful combinations

of interpersonal relations and computer technologies is stressed. Jernstedt (1983)

thinks we now know enough to define the functional purpose of the interpersonal

and computer roles and to arrive at a synergetic combination. Interpersonal or

human functions seen as important include a focus on peer relationships and

cooperative goals, and defining leadership roles for students and instructors such

that high task engagement results. Computer functions cited as important include '. -,

frequent and varied active student involvement or interaction and the use of .*.*

visual and other sensory feedback to maintain student attention.

7

..,....
* S

TiM point has been made by LIjin (1984) that for computer-based

Imtruction to be effective# a new learning envirornment has to be created that

mirrors the strong teaching/learning characteristics of live instruction. That is, a

"human element" must be provided that can manage student attention and

comprehension. A similar point has been made by Podensl4 (1984), who has argued

that microcomputer-based technology should simulate ideal student-teacher

interactions in a tutorial sense and thereby free teachers for more complex tasks,

such as diagnosing general student learning problems, helping students develop

appropriate learning strategies, and montoring the effects of instruction. Barger

(1933) has even more strongly argued that computers can show what it means to

be human and help students become more human. He discusses how computers

can be used to promote the human traits of autonomy, individuality, rationality,

affectiveness, responsiveness, and creativity through creatively designed CAI

programs that include interactive dialogues, feedback, and simulations.

Sprecher and Chambers (1980) discuss the fact that increases in the rate

and amount of learning with CAI, as well as increases in student motivation, are

largely due to the use of interactive capabilities of this medium in simulations,

games, and tutorials. They point out that one factor which has affected the use

of CA!, in addition to lack of quality instructional materials, is the perceived

impact-of computers on the socialization function of education and the concern

among some educators that widespread or exclusive uses of computers in

education may interfere with or prevent students! development as mature and

responsible individuals in a larger social system. One suggestion given by

Sprecher and Chambers to offset this dilemma is to use computers in a

free-choice environment where students have the option of learning from the

8 Leit'#
computer or from the teacher. Another suggestion, advanced by David and

Williams (1980) is to use CAI for group instruction in the form of demonstrations,

cooperative team learning, group games, or a number of other strategies that

could reduce the number of terminals used. Dugdale (1979) has similarly

suggested that computers can be used to foster creative interactions among

students through the use of games.

Intrinsically motivating features of games that can be incorporated into

CAI enhancements have been the topic of some recent work by Malone and Lepper

(in press; Lepper & Malone, in press). Particularly critical features include

whether the game has a goal, whether the computer kept a score, and whether

there were audio effects. In addition, the use of fantasy (eqg., graphics

characters, story plots, sound effects) in evoking mental images helps students

vicariously experience the satisfactions of power, success, fame, and mastery. It -)

is suggested that CAI instruction capitalize on these features and help students

identify with imaginary characters to increase the personalization of the medium

and thereby promote identification and processing in self-referent terms.

Evidence is presented that motivational embellishments that include multiple

channels (eg., visual graphics, text, and audio) can enhance learning because they

(a) reinforce and make material to be learned more accessible and (b) permit

students to seek out or employ the presentation mode most suited to their

knowledge state or learning style.

Recent advancements in computer-based technologies now make it

possible to include a rich array of audio and visual capabilities within interactive

CAI lessons. Ginther (1983) discusses advances in the area of audio/speech

devices that can be connected to a variety of common microcomputers. Benefits

9
cited for these devices include the elimination or reduction of reading

requirements, the provision of multiseruory exploration of new information, and

the supplementation of visual materials. Interactive videodisc technology has also..

been cited as a potentially powerful device for enriching CAI presentations (e g., .

3onassen, 191). The problems of lack of theoretical support and phenomenally

high development costs are cited, however, as potential drawbacks for which

lower-cost alternatives must be sought in order to provide the multisensory

richness and personalization required to simulate instructor and group process

functions.

An additional advantage of CAI as a medium for skills training is that it,

in itself, can be motivating (Steinberg, 1979). To take full advantage of these

inherently motivating qualities, it is necessary to employ effective instructional

design principles that can help dictate appropriate use of this medium within

particular content areas (Gagne, Wager, & Rojas, 1981; Steinberg, 1979).

Steinberg (1979) points out that a mistake commonly made in the design of CAI

lesson strategies is failure to take into account an essential difference between

the classroom and CAl-the learning that can take place as a result of the

res ponses of other students and the benefit of classroom interaction that must

now be supplied by the CAI lesson. Implications for how CAI might be used to

simulate group interactions can be drawn from the work of Bloom (1984), Bouton

and Garth (1983), Cubberly, Omizo, & Longano (1984), Gagne (1983), Michaeison

(1983), and Neale (1983). These include the use of multiple context case histories

of meaningful peer problems in which stucdents can interactively engag in

identifying the problem, consequences, and alternative solutions through such

techniques as computer-guided inquiry, imagery, and explanations.

10

. am
A cognitive approach to computer courseware design and evaluation has ..

recently been presented by 3ay (1983). Within such an approach, considerations

must be given to memory and attention demands, language and text

characteristics, graphics and visual processing, and implications of the cognitive

model for feedback. Specific suggestions given by 3ay include integrating CAI

with supplemental materials, encouraging depth of processing through elaboration

techniques, communicating in a personal and friendly way, using multiple

presentation formats, encouraging a dialogue with the learner, using highlighting

and cuing to direct attention, using mnemonics and imagery techniques, using

audio and enlarged text to emphasize important information, using inquiry

techniques to encourage active responding, allowing for multiple answers, and

using personalized feedback. These suggestions have relevance to the design of

CAI enhancements for the presen, effort and for simulating instructor and group

process functions.

In suxnmary, this selective review of enhancements that can be provided

by a CAI format and suggested design strategies has generally indicated features

of this medium that can be used to simulate critical instructor and group process

functions. In particular, careful selection of training content, careful design of -

CAI strategies, the incorporation of personalization through an integration of

audio and visual capabilities, the identification of meaningful roles for instructors

within this context, and use of the inherently motivating qualities of this medium

should contribute to the effectiveness of the CAI enhancements.

~ - a.. .* .
V2

Phase 1i Evaluation Study Method

i*. .,'

Research Questions

The primary questions being addressed in this Phase II evaluation study

were: (1) What is the relative effectiveness of CAI-enhanced versions of the

Motivational Skills Training Program as compared with versions augmented by

instructor and group process experiences? (2) Are there various combinations of

CAL, instructor, and group experiences that contribute to the differential

effectiveness of the motivational program? and (3) What are the implications of

study findings for future implementations of the motivational program?

To answer the preceding questions, CAI introductory and practice

segments were designed and developed for each of the seven motivational skills

modules. An experimental evaluation was then conducted using instructors and

students from the Electronic Communications (EC) school at Ft. Sill, Oklahoma.

The following sections describe the design and evaluation procedures.

Design and Development of CAI/Audio Segments

The Phase I report for this project (McCombs et al., in press) laid out

some general design guidelines for operationalizing the identified instructor and

group functions in the CAI format. In addition, the decision was made in Phase I

to develop a simple computer-controlled audio capability to achieve the

personalization desired in the simulaton of instructor and group functions. The

specific design strategies used to integrate the audio/CAI enhancements and

simulate instructor and group functions are described below.

The character "PC," created to simulate instructor functions, was

designed to enact three primary roles: facilitator, modeler, and motivator. In the

12
facilitator role, PC was designed to help students acquire new concepts, skills,

and strategies via introductory explanations and practice exercises. In the

modeler role, PC was designed to demonstrate the application of new concepts,--.

skills, and strategies in guided practice segments. In the motivator role, PC was

designed to coach and encourage students to apply new concepts, skills, and

strategies in both introductory and practice segments. To assist students in

knowing which role PC was playing at any point in time, screen messages were

designed to appear in the upper left-hand portion during the time either the

graphic figure of PC was presented, or text and audio messages from PC were

. being presented. When playing the facilitator role, the message, "Now you try it!"

is displayed; when playing the modeler role, the message "Watch me!" is displayed;

and when playing the motivator role, the message "Go for itP' is displayed.

To provide the identified group functions of peer identification,

opportunities for shared problem solving, and peer modeling and feedback, a set of

military characters was defined. As indicated in the introduction section of this

* report, these characters represented specific personal responsibility and

self-control problems related to each module's content area. Case studies and

audio scripts were prepared for each character and for the introductory and

practice CAI segments. The characters were designed to "grow"' as a result of

their skill training from an initial inability to solve particular problems to

competent problem solvers and self-managers. This transition occurred between

"-' PC's guided CAI introductions and CAI practice sessions for each module. (See

McCombs et al., in press, for a definition of each character.) Not only are the

characters' problems presented in the CAI segments, but processes used by the

characters to work through these problems are exemplified.

13

o~* "
Much of the modeling provided in the CAI segments is accomplished via

the audio enhancements, while most of the actual skill practice is provided

through the CAI exercises. These two functions are not entirely discrete,

however, but are integrated into a single unified presentation via the audio

interface. That is, the audio is integrated with screen presentations in the

following ways: (a) the CAI screen reinforces audio information, (b) the audio

information precedes a CAI segment or frame; (c) the audio information follows a

CAI segment or frame; or (d) the audio is an integral part of a CAI segment or

frame. Used in this way, the computer-controlled audio capability increases the

level of personalization, simulates instructor and group process functions,

enhances motivation through novelty, reduces reading demands, and adds to the

cost effectiveness of the program as compared with other media.

Scripts were developed for the audio portions of the CAI segments, and

professional and semiprofessional personnel were selected to play the parts of PC

and the seven characters. Once the audio tapes had been recorded, pulses were

added at points that coincided with CAI screen changes. The contractordeveloped audio interface consists of a specially designed interface card which

plugs into the Apple lie game I/O port. The interface receives the pulses from a

standard slide-sync audio cassete player. These pulses trigger screen changes and,

in turn, allow the CAI software Gn this case, the Apple SuperPILOT Authoring

System) to control the on/off function of the audio player. This capability allows

for computer control of a linear sequence of audio messages that coincide with

particular CAI frame sequences, as well as provides for the personalization of

skill training introductions and practices, at about one-eighth the cost of

videodisc technology.

4 -

41

. . • -..- " % .- - --.. --.°. .,- -.- .-",. ... ' • .. .. ." .. • ..- . - • .J . ' . . .. . .' . . . - . - .. . . - -. - ." -. . .2 ..- A .
In general, the structure of the CAI introduction is to present case

studies of a particular character's problem, provide a description of the tools

students will acquire to solve each particular problem, and present a brief

Interactive practice of these skills. In the CAI practice segments, students

receive a recap of the skills they learned in the printed module, exemplified by

the module's character. Optional and nonoptional practice exercises are provided,

followed by an integrative summary of skills and strategies learned. Special

features of the CAI design also include the use of student contracts, special

instructional games in the goal-setting area, problem solving scenarios, and some

off-line practice exercises integrated with the CAI segment. Following the initial

construction of the CAI/audio segments, personnel from the Ft. Sill Electronic

Communications course visited the contractor and reviewed the materials.

Suggestions regarding revisions were collected and the revisions were made as

necessary.

Experimental Design

To address questions regarding the effectiveness of the CAI-enhanced

versions of the motivational skills program, six experimental conditions were

defined: an historical control group (HC), current control group (CC), a CAI

introduction and practice group (CA), a CAI introduction and instructor practice

group (CAI), an instructor introduction and CAI practice group (ICAD, and an

instructor introduction and practice group (10. Independent variables of interest

(in addition to conditions) were student scores on the General, Electrical, and

Clerical subscales of the Armed Services Vocational Aptitude Battery (ASVAB);

military rank; Army component (Regular Army, National Guard, Reserve), sex;

initial critical thinking skills; initial judgments of self-efficacy; and initial indices

-.4
of anxiety and ability to cope with stress. Dependent variables of interest were

instructor ratings of studentsP self-management skiUs; time to complete the first

and second EC course segments; test failure rates in the first and second EC

course segments; progress index for the entire EC course; and whether students

attrited or graduated.

Measures

The self-efficacy, anxiety and ability to cope with stress, and instructor

questionnaire measures are the same as those described and included in the

McCombs et al. (in press) report. The Critical Thinking Skills measure was one

developed by McCombs (1980), with items patterned after Watson and Glaser's

(1964) Critical Thinking Appraisal. The six items on this measure assess students'

inferential and deductive reasoning skills, using examples relevant to the military.

The response format is multiple-choice, with one correct a. Ter per item. (A

copy of the Critical Thinking Skills measure can be found in Appendix A of this

report, along with the instructions students received prior to taking the initial

pretest battery.) . -

Measures included in the pretest battery were "How I Feel About Myself"

(self-efficacy measure), "Stress Profile" (anxiety and ability to cope with stress .--

measure), and the "Critical Thinking Skills" (inferential and deductive reasoning

skills measure). These measures were also administered following the first %-7 7

portion (approximately four weeks) of the EC course in order to assess any prepost changes in these skills for experimental and control group students. The

Instructor Questionnaires (ratings of students' self-management skills) were filled

out by instructors at the end of the first course portion who had taught students

during this portion of the course. It was expected that one possible impact of the

16

0, .. A-

.. . . . . . . . . ..i . p w . . *
6

motivational skills training, in addition to increased course performance, might be

increases in students! perceptions of competency, abilities to cope with stress,

critical thinking or problem solving skills, and self-management skills.

Student ASvAB scores, times-to-complete, failure rates, progress

indices, attritions, and graduates information were obtained from student records

and computer printouts supplied by personnel in the EC school at Ft. Sill.

Subjects

Participants in the Phase 11 evaluation study were male and female

students scheduled to begin the Electronic Communications school at Ft. Sill

during the period of '25 October 1983 through 17 February 1984. Students who

participated were in classes 3 through 16 of the course;, performance data for "WR

these classes were collected through May 1984. Students were assigned to one of

the five current experimental conditions by designated Ft. Sill personnel, using

guidelines and procedures specified by the contractor. These guidelines specified

that students in each entering class be randomly assigned to experimental and

control conditions such that a balance of Army component (Regular Army,

National Guard, Reserve) and male and female students was achieved across

experimental and control conditions. A schedule of when each experimental . o4

condition occurred was prepared which specified that a single experimental

condition be conducted during any given week, with any given class, beginning

with the CAI condition and rotating through conditions until data on

approximately 50 students per condition were collected. At the conclusion of the

study, data on a total of 479 students were available for analysis. The number of

students in each condition were as follows: 253 in the CC condition, 55 in the CAI

conditon, 61 in the CAlI condition, 53 in the ICA! condition, and 57 in the I1

condition.

17

4.1.

* ;~.* * *~~*' ~c-.;**.
An historical control group of 53 students was formed from EC course

records for the purpose of having a comparison group removed in time that could

be used to assess any possible effects of peer discussions among current

experimental and control students about the content of the motivational training. a

This historical control group consisted of a representative sample of males and

females, proportionately divided into Regular Army, National Guard, and Reserve

components, from classes 2 through 15 who had been in the EC course in the

previous year. Data from this historical control group was separately analyzed in _._

terms of descriptive information on variables such as ASVAB scores, course

completion times, failure rates, attritions, and graduates.

Procedures

A 4-day, 16-hour training program for three Ft. Sill EC course

instructors designated as being responsible for the implementation of the

Motivational Skill Training program and evaluation procedures was designed. The

purpose of this training was to acquaint instructors with the purpose of the

evaluation study, familiarize them with concepts and skills presented in the

training program, provide guidelines for introducing each module and conducting

the small group discussion/practice sessions at the end of each module, and

describe specific procedures for implementing each experimental condition.

Instructors were also trained in the content of the CAI materials and operation of

the CAI equipment. A workshop format was used for this training, wherein three

contractor personnel guided instructor participants through a training outline and - -

provided models of the instructors! role in introducing the modules and leading the

group sessions. Instructors were given opportunities to generate their own

examples to be included in the introductions and their own practice exercises,

18
thereby arriving at personalized plans for conducting the skill training. To ensure ml

a degree of comparability between the CAI and instructor conditions, instructors

were provided with scripts of the case studies applying to each CAI character

which they could use as part of their introductions and practice sessions for each

module. (A copy of the instructor training outline and other handouts are included

in Appendix B.) The instructor training was conducted from 18 through 21

October 1983, at which time five Apple lie systems were also installed in the

specially-designated classroom in the EC school.

The pretest measures were administered to control- and experimentalgroup students on the-day they were regularly scheduled to begin the EC course.

Following completion of the pretests, students were assigned to the control or

experimental groups per contractor procedures (see Student Assignment Sheet in

Appendix C). Control students began the EC course following this pretesting and

their regular course orientation; experimental students were told to report to the

specially designated classroom for the motivational training. Ft. Sill personnel "'.

responsible for assigning students to experimental and control conditions were

briefed by contractor personnel to inform students that some of them would be

randomly selected to participate in an experimental program. To minimize

possible negative student reactions to being selected or not selected, no further

details about the nature of the program were given. The number of students

assigned to experimental conditions in any given class varied from 10 to 15,

depending on the class size. The length of training for experimental students

averaged 25 hours; training was conducted for 6 hours a day for 4 to 5 days,

depending on the rate at which students completed the motivational training.

For students assigned to designated experimental conditions, they were "'

instructed regardir the nature and purpose of the training and were told the

19 -'

______ ,______ ______ ______ __r .__" " "• , '.,. "; . -.. ",: •.- .._.J.... -'_ . . . _' 3 , " "".' " '" "" "" " '""C.. 
Pneral procedures for completing this training. For students in the CAI

conditions, they were told how to operate the computer equipment and were given

a special Apple lie orientation disk on system operation. Depending on the

condition assignd, students were told that they would receive an introduction to

the prolram and each module from either the instructor or computer, were then

to read each module on their own, and return to the instructor or computer for

practice sessions. Instructors responsible for the training monitored student

progress through each module and for the instructor practice conditions,

assembled students for group sessions when the last student had finished a

particular module. -

Following the completion of the motivational training, experimental

students began their regular EC course work. The EC course is divided into two

portions, the first of which is approximately 4 weeks long and the second of which

is approximately 3 weeks long, for an average total length of approximately 9

weeks. The course is implemented in a self-paced mode wherein times to

complete vary as a function of student ability and motivation. When the

evaluation study began, course attrition rates were averaging approximateiy 28

percent. After the first week of the evaluation study, however, it was learned

from Ft. Sill personnel that procedures had been implemented by the commander

of the student battery responsible for students in the EC course wherein students

whose progress rates met or exceeded 1.2 (where 1.0 is the average rate and more

than 1.0 is a longer than average rate to complete the course) were being assigned

to mandatory study sessions and denied leave privileges until their progress rates

fell below 1.2. In subsequent discussions, Ft. Sill personnel indicated that this new

battery procedure appeared to be reducing student attrition rates by

approximately 10 percent. Thus, the possible influence of the motivational

20

.. . .: . -..

) .. :iZt .l , "3,' P ,s.a. [ • - ._. ",'"- Z. ._ - '"% ' •'', . .. - '-"." ,' . . , ,' ,.,. ' , .'.'-' z*• o o' .o. . .'-'." . -"-'.
sIdis tr)ning on student attrition was partially attenuated throughout the

duration c the evaluation study period.

Postmeasures were administered by Ft. Sill personnel in the EC course at

the end of the first course portion. Ft. Sill personnel were also responsible for

collecting these measures for pickup by contractor personnel during monthly trips

for the purpose of monitoring evaluation procedures. The monitoring trips were

also conducted for the purpose of providing any follow-up training to instructors

responsible for implementing the experimental conditions, as well as to answer

special questions or solve unanticipated problems. Throughout the duration of the

evaluation, monitoring visits indicated that procedures were being implemented

with no serious problems and that instructors conducting the training were

enthusiastic and very competently handling their training responsibilities.

21~.

4??-:-

•9\ .

So. -

r, *o.°., . -

'4 5. . ' %~ ** .*.* .:*. .. ):. i--* -
Phase 1 Evaluation Study Results

Phase 11 Analysis Procedures

The major interest in the study was to examine whether the independent variable

"experimental condition" was significantly related to various indices of student

performance (iA., the study dependent variables). The interest was not in

omnibus comparisons involving the five experimental groups but, rather, in

psirwise comparisons of groups on the variables of interest. Descriptive data

from the historical control group (HC) was examined but not included in

statistical comparisons. .'..-.

Initial chi-square and t-test comparisons were made to determine

whether students had been randomly assigned to groups according to instructions.

In general, they had. Next, pairwise comparisons were made on the independent

variables (iOe., possible covanates) which were not of central empirical interest,

but which could possibly explain (influence) any group differences in performance.

These possible covariates included measures of intellectual functioning (ie.,

ASVAB scores) and pretraining measures of stress. Results showed that there

were few, if any, meaningful differences among experimental conditions on the

possibly confounding covariates. Thus, the group comparisons on the variables of

interest were conducted with simple t-tests. For directional hypotheses (iGe.,

where one group was expected to score better than another) one-taled

probabilities were employed.

Final Subject Selection

Initial examination of the data revealed that the small number of

students (n = 21) of military rank E-5 and above had been disproportionately

22

ze'•o
*L

nssiped to the curent control (CC) group. These students were subsequently

excluded from the study resulting in a final sample of 458 students. The number

of students remaining in each condition were as follows: 233 in the CC condition,

55 in the CAI condition, 60 in the CAU condition, 53 in the ICAI condition, and 57

in the II condition. Table I presents the distribution of students by rank within

condition. Pairwise chi-square comparisons of experimental conditions indicated

that students were randomly distributed by rank across conditions.

Assimnent of Students to Experimental Conditions

Recall that Ft. Sill personnel had been instructed to randomly assign

students to conditions to achieve a balance of Army component and sex of student

across experimental conditions. Table 2 shows that, with respect to Army

component, this balance was not completely achieved. In particular, a

disproportionately low percentage of National Guard and Military Reserve

students were assigned to the CC condition. This result indicated that students'

Army component might need to be controlled for in further analyses. However, a

correlational analysis revealed that Army component was not significantly related

to any other variable included in the study. Thus, the underrepresentation of

National Guard and Military Reserve students in the CC condition was not

problematic.

Table 3 shows that a very small percentage of the sample (4.1%) was

female. Although females were not assigned randomly across all conditions, no

provisions were made for controlling for sex of student in further analyses since

such small numbers of females were present in the student population.

~. ;~.t~%'. .. 4 g%.~~ 23 -. . .-..
Table 1

Distribution of Students by Military XLank

Within Experimental Condition

Military Rank* A.).

Group E-1 E-2 E-3 E-4 Row Total :J

CC 18 (79.4) 19 (8.2) 14 (6.0) 15 (6.4) 233

CAI 41 (74.5) 5 (9.1) 6 (10.9) 3 (5.5) 5'

CAll 46 (76.7) 8 (13.3) 2 (3.3) 4 (6.7) 60

ICAI 44 (83.0) 7 (13.2) 2 (3.8) 0 (0.0) 53

11 46 (80.7) 3 (5.3) 5 (8.8) 3 (5.30 57

362 (79.0) 42 (9.2) 29 (6.3) 25 (5.5) 458

Comparison X2(3) .

CC vs. CAI 1.79 .618

CC vs. CAnI 2.06 .559

CC vs. ICAI 5.08 .166

CC vs. 11 1.15 .765

CAI vs. CAII 2.91 .406

CAI vs. ICAI 5.40 .145

CAI vs. I .84 .839

CAll vs. ICAI 3.69 .297

CAll vs. II 3.63 .305

ICAI vs. l 5.79 .122

*Row percentages in parentheses.

24
Table 2 -

Distribution of Students by Army Component

Within Experimental Condition

Army Comnponent* %. %

Regular National Guard

Group Army or Reserve Row Total

cc 198 (85.0) 35 (15.0) 233

CAI 40 (72.7) 15 (27.3) 55

CABl 46 (76.7) 14 (23.3) 60

ICAI 31 (58.5) 22 (41.5) 53

II41 (71.9) 16(28.1) 57

356 (77.7) 102 (22.3) '458

* Comparison X2(1)*

CC vs. CAI 3.84 .050

CC vs. CAll 1.81 .179-

CC vs. ICAI 17.36 .000

CC vs.II1 4.52 .033

CAI vs. CAII .07 .786

CAI vs. ICAI 1.84 .175

CAI vs. 11 .00 1.000

*CAII vs. ]CAI 3.49 .062

CAII vs. 11 .14 .708 -

ICAI vs.!11 .19 .665- ,

*Row percentages in parentheses.

C **Yate's correction applied.

25

................................... *
* .

'F'j

Table 3

Distribution of Students by Sex -

Within Experimental Condition

Sex* -"

Group Female Male Row Total

CC 3 (1.3) 230 (98.7) 233

CAI 3 (5.5) 52 (94.5) 55

CAll 2 (3.3) 58 (96.7) 60

ICAI 4 (7.5) 49 (92.5) 53

II 7 (12.3) 50 (87.7) 57 --

19 (4.1) 439 (95.9) 458

-oO-.

Comparison X2 (1)** .-

CC vs. CAI 2.02 .155 2'

CC vs. CAll .28 .595

CC vs. ICAI 4.71 .030

CC vs. 11 13.48 .0002

CAl vs.CA .01 .921

CAI vs. ICAI .003 .960

CAI vs. II .87 .350

CAll vs. ICAI .33 .564

CAII vs. ll 2.16 .142

]CAI vs. Il .26 .611

*Row percentages in parentheses.

**Yate's correction applied.

26

. .. . . . . . . . . ... . . .. . . o . . . ,
1- W W

Attrition Rates

The first dependent variable of interest was graduation status: ie.,

whether students graduated or attrited from the course of study. The pairwise

chi-square group comparisons presented in Table 4 indicate that there were no , /- *%".

statistically significant differences in student attrition rates across groups. The V

overall attrition rate was 22.7 percent. From a practical standpoint, however, it

is of interest that the lowest percentage of attritions were in the CAI (14.5%) and

II (19.3%) conditions.

Examination of Potential Covariates

Following the analysis of attrition rates, students who did not graduate

were excluded from further analyses since scores for these students were not

available on the remaining dependent variables (e.g., progress index). Thus, in the

results which follow, the total sample is comprised of 354 students. The number

of students remaining in each condition was as follows: CC (n : 175), CAI (n =

47), CAII(n = 46), ICAI(n = 40), and 1I(n = 46).

The pairwise group chi-square comparisons presented in Table 5 show

that these remaining students were randomly distributed across study conditions

* by military rank, but not by Army component. As in the previous analysis, the

lack of random assignment to conditions by Army component was not considered

problematic since correlational analyses revealed that Army component was not

significantly related to any other study variables for the sample of interest.

Recall that the independent variable of primary interest is experimental

condition. All other independent variables are potential covariates. Table 6

shows that there were no pre-existing significant differences across groups on

ASVAB scores. Thus, each of the five groups was composed of students of similar

27

. .. 

.~ % S~~ ~ ' % ~ * . . ~ - ' . .2 a -~ * . ~. ~ ~ ..x ~1
M R

Table 4

Distribution of Students by Graduation StatutWithin Experimental Condition

Graduation Status*

Group Graduated Attrited Row Total

CC 175 (75.1) 58 (24.9) 233 -

CAI 47 (85.5) 8 (14.5) 55

CArl 46 (76.7) 14 (23.7) 60

ICAI 40 (75.5) 13 (24.5) 53

II 46 (80.7) 11 (19.3) 57

354 (77.3) 104 (22.7) 458

- .. o -

Comparison X2 (1)** PCC vs. CAI 2.14 .143

CC vs. CAD .01 .935

CC vs. ICAI .00 1.000

CC vs. II .51 .474

CAI vs. CAI] .92 .337

CAI vs. ICAI 1.14 .286

CAI vs. 11 .17 .676

CAl vs. ICAI .00 1.000

CAII vs. II .09 .759

ICAI vs. II .19 .665

*Row percentages in parentheses.

**Yate's correction applied.

28

.¢7

i-. .. . . .* - ,.- .-.-~ ,. .-. * - . .. . . . . .. o- .. .--.. . .-. -. . . . •- , - ... . ..-

=,' z _.C L% '_L _.L :'. -- ' . . .. .;2 '' " ... * " . . " . " .. - k. " " .. " =- .. " " . .. e ' % ". . .. . "' €","-_^',''- ... .. . , .C
Table 5

Chi-Square Group Comparisons for Students Who

Graduated: Military Rank and Army Component

Military Rank Army Component

Comparison X1(3) pX ()

CC vs. CAI 2.99 .393 4.63 .032

CC vs. CAII 1.15 .766 1.54 .214

CC vs. ICAI 4.25 .236 19.04 .000

CC vs. rt 4.52 .211 8.15 .004

CAI vs. CAt -1.38 .710 .16 .686

CAI vs. ICAI 3.90 .273 2.18 .140

CAI vs. II 3.08 .380 .09 .769

CAnI vs. ICAI 2.73 .435 4.25 .039

CAr vs. l1 5.18 .159 .84 -360

ICAI vs. 11 6.99 .072 .96 -329

*Yate's correction applied.

I. 29

.~• 

-. .. . . .

°..
Table 6

T-Test Results for Group Comparisons on ASVAB Variables

t

ASVAB Variables '

General Electrical Clerical

T 2-Tail T 2-Tail T 2-Tail

Comparison Value DF Prob. Value DF Prob. Value DF Prob.

CC vs. CAI .43 178 .67 .76 178 .45 .14 1'24 .89

CC vs. CAII .56 163 .57 .55 163 .59 .37 116 .72 ,.

CC vs. ICAI 1.1.6 176 .25 1.54 176 .13 .49 112 .63

CC vs. 13 .87 173 .39 1.38 173 .17 .15 1114 .9. -.

CAI vs. CAII .83 55 .41 .88 55 .38 .39 32 .70-

CAI vs. ICAI 1.28 68 .20 1.56 68 .12 .57 28 .5

CAI vs. II 1.23 65 .22 1.63 65 .11 .05 30 .96

CAII vs. ICAI .30 53 .77 .51 53 .61 .10 20 .92

CAII vs. I .13 50 .90 .47 50 .64 .32 22 .75

ICAI vs. 11 .24 63 .81 .14 63 .89 .47 18 .64

30

,S.'

t + . o .. ll .','.'l +' . ' . . . o . o' . *+ . . . . + . '.... . . . . . . . . . . . . . . . . . . . . . .. . " ... " " " " ' . . . ' " % % -+

L -.-/--+-+"+--J+-- [-t--- .-t."-- m- - t. -- ., r~l --It. m.+k -

" " . ' m-- . . +L. . '.I-JL-I-. . . . . - +I-I--I*. . . . . . . . . . . . . . ". • • "
abilities. (A table of group means and standard deviations for all study variables

is included in Appendix D.)

Examination of other potential covariates (i. critical thinking skills,

self-efficacy, adaptation, frustration, overload, deprivation, nutrition,

self-perception, type A behavior, and anxiety pretests) revealed few significant

pre-existing differences among student groups. Of the 100 t-tests conducted

(ie., 10 pairwise comparisons of groups on 10 variables) only seven t-tests proved

to be significant. There was no identifiable pattern to these differences and,

indeed, the number of significant t-tests (7) was not much different than that

expected by chance (5) employing the .05 probability level as a significance

criterion. Correlational analyses revealed that, of these 10 variables, only the

self-perception subscale of the stress pretest was significantly related to any

independent variable (ie, the progress index). No two groups differed on the

self-perception pretest. Thus, there was little reason to "contror' for the effects

of these potential covariates in subsequent analyses.

Analyses of Major Dependent Variables

There were seven dependent variables of major interest: instructor

ratings of students, number of student test failures on the first portion of the

course (tasks 1-16), number of student test failures on the second portion of the

course (tasks 17-26), total number of student test failures, time to complete tasks

1-16, time to complete tasks 17-26, and student progress index (essentially an

indicator of total time for tasks 1-26). Table 7 presents the results of pairwise

comparisons of the five study conditions on these variables.

There were no significant differences in instructor ratings of students or

in number of failures on tasks 1-16 across conditions. For number of failures on

31
.~~~. .. .. . -

W, ;z All

r4 N ' 9

a' % 9% %D - a' y 0401

q4.~

ca N - -,l

1. 9 40 9 - 0% - D I-. a

V% P4 0 % 0% N C N f

600 41 4p gO P. P .

N IL

PtA

aZ

u- u 15

IP 31, 3 I

u uu

- -v - - 0 us 3-

.32
tasks 1-16 the results of two analyses approached significance. Specifically,

sudents In the CAI condition tended to fail more frequently than students in the

CAB (p a .058) and 11 (p z .079) conditions. For failures on tasks 17-26, students in

the ICAI condition failed significantly more often than students in the CAII

condition ( u .046) and tended to fail more often than students in the II condition

(2. = .075). In terms of total number of test failures, a number of findings were

either significant or approached significance. Specifically, students in the CA-.

and 1U conditions tended to fail less often than students in other conditions: CAI.

vs. CC .032), CAB vs. CAI (p = .050), CAB vs. ICAI (a = .064), I1 vs. CC (P _

.068), I vs. CAI (p = .077), and U vs. ICAI (p = .094). Thus, the pattern that

emerges is that students who received "instructor practice" as part of the

Motivational Skills Training Program tended to have fewer total test failures than

other groups of students. This finding cannot be accounted for by pre-existing

group differences.

The results for "time on tasks" and progress index show that the II group

completed tasks significantly faster than the CC group. time on tasks 1-16 (p

.014), time on tasks 17-26 (p = .025), and progress index (p_ = .014). The students in

the H condition also tended to complete tasks significantly faster than students in

the CAI condition: time on tasks 17-26 (p = .007) and progress index (a = .011).

Further, students in the ]H condition tended to complete tasks faster than students

in the ICAI condition: time on tasks 17-26 (p = .109) and progress index (p = .092).

Thus, in general, students who received Motivational Skills Training in the 11

condition tended to complete training faster than students in the control condition ,

and faster than students who received Motivational Skills Training in conditions

which did not include instructor practice.

33

€ , ,4i ;b, # . .w .\?.' . •.' . ," ,* • ,-.... • : -. .-.-. - •* • . - . ,. -. ... ** . :.
Pre-Pmttest Camvariuons of Critical Thinkinit, Self-efficacy, and Stress

Indcators

A Critical Thinking test, a self-efficacy scale and eight stress indicator

scales were administered before the Motivational Skills Training Program and

after students had completed the first 16 training tasks. Table 8 shows the results

of within-group comparisons of pretest and posttest indicators. Variable means

and standard deviations are included in Appendix D to aid in interpretation of

results.

On the Critical Thinking test, pre-posttest comparisons were significant

for the CC, CAll, and ICAI groups. The fact that the CC group, who did not

receive Motivational Skills Training, improved significantly on the Critical

Thinking posttest indicates that improved scores on the posttest may have been

due to test familiarity rather than Motivational Skills Training since the same test

form was employed in both the pre- and posttest. One anomalous finding was that

the CAI group actually scored lower on the posttest than on the pretest (although

not significantly lower). '"

Table 8 also shows that there were no significant changes in selfefficacy in any group on the pre- posttest comparisons.

In general, across all conditions but II, there was a substantial number of

stress indicators on which pre-posttest differences were either significant or

approached significance. These significant differences were in the expected

direction (e.g., the CAI group had better self-perceptions on the posttest than on

the pretest). These changes, however, cannot be attributed to Motivational Skills

Training for two reasons. First, Table 8 shows that there were many significant

pre-posttest differences on stress indicators for the CC group which did not

receive Motivational Skills Training. Second, further pairwise t-tests of

34
I.. %D

K .

N<.5.

Pl. CI

40-

A 

N

Sn u
V I.

pre-posttest differences between experimental groups indicated that the CC

group was not significantly different from the other groups in terms of amount of

poiive change on the stress indicators. r

One perplexing finding was that the II group, which scored best in terms

of technical training performance indicators, exhibited less positive change (and

in some cases, negative change) over time, as compared to other treatment %7:*

4 groups, on the stress indicators which Motivational Skills Training is thought to

influence.

Exploratory Individual Difference Comparisons for the CAI vs. 11 Conditions

Individual difference comparisons between the CAI and 17 conditions

were of particular interest since these groups represent the extremes of the

computer-versus instructor intensive modes of delivering the Motivational Skills

Trairing Program. A number of exploratory analyses of variance (ANOVAS) were

run on these two groups. These analyses were all 2 by 3 ANOVAs (iqe., Groups =

CAL, U: "other independent variables" = low, medium, high). The "other

independent variables" of interest were ASVAB general, ASVAB electrical, I critical thinking pretest, self-efficacy pretest, and self-perception pretest.

ANOVAs employing these independent variables were run on each of six

independent variables (ie., progress index, failures on tasks 1-16, time to

complete tasks 1-16, failures on tasks 17-26, time to complete tasks 17-26, and

total test failures) for a total of 30 ANOVAs.

Of the 30 ANOVAs run, nine analyses indicated "significant" interaction

effects. Due to the highly exploratory nature of these analyses, the .10

probability level was termed significant. Significant interactions are summarized

in Table 9. These results should be interpreted with caution for two reasons: the

36,.'V ,
** %

414

C4 00N

N M0 0 @0 NO M

C44

00. ONk 00' 20~v Z@ 

ON-Nc4 -4..C.

u -

v 0

E0 <0~ x0 4J N

i!c-N N-2

5U~ U~V~ U 4

UU

u ,

2.0 ~. ~ -

<o <4< > > > > >C V) V) V) V V . 1

< < ,! < < < u u

370..~ o
liberal "sigificance" criterion employed and small cell ns in many cases. With

these limitations in mind, the following results are presented as suggestive.

" In general, students in the CAI group who had high ASVAB

general scores outperformed CAI group students with low

and medium ASVAB general scores, while II group students

who had medium ASVAB general scores performed better

than 11 students with low or high ASVAB general scores.

" In the CAI group, students who scored in the medium range

on the critical thinking pretest performed better in later

coursework, while 1I group students who scored high on the

critical thinking pretest scored higher than other II group

students on training course performance indices.

* Students in the CAI group who scored low on pretest

indices of self-efficacy and self-perception had fewer

failures on technical tasks 1-16 than did other students in

the CAI group, while students in the U group followed the

expected pattern: the higher one's self-efficacy and

self-perception, the lower the number of test failures.

Figures 1 and 2 present generalized graphic representations of the

findings. As indicated, these findings are suggestive and there is a need for a

more large-scale replication before definitive conclusions can be drawn. Taken at

face value, however, these findings indicate that there may be individual

differences on certain ability and self-report measures which are predictive of

differential rates of course performance and related to different modes of

delivery of the Motivational Skills Training Program. The results with ASVAB

general and Self-Efficacy/Self-Perception are particularly interesting in that the

correlation between ASVAB general and these two measures is very low (r's = .08

and -. 10, respectively). This implies that approximately half of the students may

benefit from the CAI treatment (ie., high ASVAB general and low

Self-Efficacy/Self-Perception students).

Historical Control Group (HC)

The descriptive data for the HC group is summarized in Table 10. The

HC group was similar to the students from the other five experimental conditions

38
High

Course Performance

CAI -- -

Low

Low Medium High

ASVAB GENERAL

Figure 1. Schematic representation of the relationship between course

performance and ASVAB general scores.

High

Failures on Tasks

1-16 W'

Low

Low Medium High

Self -Ef ficacy/Self-Percepti on

Figure 2. Schematic representations of the relationship between failures on Tasks

1-16 and self-efficacy/self- perception pretests.

39
Table 10

Descriptive Statistics: Historical

Control Group*

Total Group, (n= 3 1)

E-1 E-2 E-3

Military Rank 33 (64.7) 11 (21.6) 7 (13.7)

Regular National Guard

Army or Reserve

Army Component 40 (78.4) 11 (21.6)- .

Female Male

Sex 6 (11.8) 45 (882)

Graduated Atri ted ___ ___ ___ ___ ___ ___ ___ __ ..- "

Graduation Status 35 (68.6) 16 (31.4)

Graduates Only (n = 51-16 = 35)

E-1 E-2 E-3

Military Rank 23 (65.7) 7(20.0) 5 (14.3)

Regular National Guard

Army or Reserve

Army Component 27 (77.1) 8(22.9)

Mean Standard Deviation

General ASVAB 109.5 10.07

Electrical ASVAB 111.3 9.13

Clerical ASVAB 104.3 14.15

Total Test Time 340.6 53.7

Progress Index 1.02 .148

Total Test Failures 2.14 1.73

*Row percentages in parentheses

40

. . . . . . - . .~a . . .- o- * - t . . . .. .. .Bt - - . . . . .A. ... 
ombined (see Tables 1-4 and Appendix D). For example, when military rank is

compared, 86.3 percent of the HC students and 8&2 percent of the students in the

other five conditions were of rank E-I or E-2. For army component, 78. percent

of the students in the HC group were Regular Army vs. 77.7 percent in the other

five conditions. In the HC condition, 88.2 percent of the students were male vs.

95.9 percent males in other conditons.

The mean scores of students in the HC group on ASVAB test were 109.5,

111.3, 104.3 on the general, electrical, and clerical batteries, respectively.

Comparable means for students in the other five conditions were 106.5, 108.6, and

102.4. Thus, students in the HC group tended to score slightly higher on ASVAB

tests than students in other conditions.

Attrition rates were slightly higher for HC students than other students

(31.4% vs. 22.7%).

Also, students in the HC group tended to take longer than students in the

other five conditons to complete coursework. The progress index means for the

HG vs. other students were 1.02 and .91, respectively. At the same time, students

in the HC conditon tended to have fewer mean test failures (2.14) than other

students (2.81). A possible explanation for progress rate and test failure

differences between the HC and CC groups may be the battery-instituted

procedures for monitoring students' course progress during the current study. (See

pp. 20 -21

Anecdotal Data

Anecdotal data available in this evaluation included instructor

comments, student comments, and contractor observations.

Comments obtained from Ft. Sill instructors responsible for conducting

the motivational skills training indicated that:

412 - -.-.•
I. students were enthusiastic about participating in the

skills training program;

2. students participating in the skills training tended to

continue using their new skills after the training was

over;

3. students participating in the skills training tended to get

more out of the instructor conditions, but the CAI

conditions were easier to implement;

4. instructors were dedicated and committed to the value

of the motivational skills training; and

5. instructors felt most students should get this training to

help them decide what they want in the Army and their

lives.

Student comments generally indicated that (a) students felt positive

about the training program content and expressed the opinion that the training

was of such value that it should be made available to all students; (b) students

were positive toward the CAI conditions but tended to feel they got more out of

the instructor conditions; (c) students felt the skill training would benefit them

both in their subsequent technical training course and in their personal lives; and

(d) students found no portions of the motivational training, including the CAI

segments, that they felt should be changed.

Contractor observations of students participating in the motivational

skills training generally agreed with comments made by the instructors. That is,

students were generally observed to be enthusiastic about the training and to have

no difficulties adapting to different treatment conditions. In addition, instructors

were observed to be highly motivated and competent in their implementation of

the training and treatment conditions.

42

!' --- :''- -"-".':>---:-" -- u -L: 'e -- . " .,. " L'." -, '.,...,,.......-..,........"....'..,'.....',........-'..,..."....".".-......'.......-..'..' ".
_ .

Discussion and Conclusions

Summary of Evaluation Findings

The Phase II experimental study was designed to evaluate the relative o. .....

effectiveness of CAI enhanced versions of the Motivational Skills Training

Program as compared with instructor/group versions, and to make

recommendations for future implementations of the program within Army

technical training. The study results indicate that students who received the

motivational skills training with instructor introductions/group practice sessions

(HI condition) performed better during subsequent technical training than either

students who received no motivational skills training (CC condition) or students

who received the motivational skills training via CAI with no instructor practice

(CAI and ICAI conditions). This better performance was manifested in terms of

both significantly fewer test failures during training and less time for completion

of training. These findings cannot be attributed to pre-existing differences among

groups since group assignment appeared to be random and the groups exhibited no

important differences on possibly confounding covariates (ia.e, ASVAB scores and

stress indicator pretests).

Additional findings of interest are that, as compared to the control

condition, students receiving either the CAI or H conditions tended to have lower

attrition rates. Although these differences were not statistically significant, the

8.2 percent reduction for the CAI condition and 4.4 percent reduction for the II

condition may have some practical significance in terms of training costs. These

attrition findings are particularly noteworthy in light of the fact that during the

conduct of the study, new procedures were being implemented in the student

battery (housing area) to improve the performance of students not progressing

through the course at a satisfactory rate (see pages 20 and 21 of this report). This S°"-.7

battery procedure had some obvious effects on student performance in general, as

43 #,4::':
evidenced by comparisons with the historical control group. That is, for similar ,.

students going through the Electronics Communication course a year earlier,

attrition rates were 5.2 percent higher than for students in the current study, and

progress indices were approximately 12 percent higher. Thus, it is likely that

findinp with the Motivational Skills Training Program were attenuated because of

the new procedures being impdemented in the student housing area.

Since one of Leo original purposes of the evaluation study was to

determine if the CA] treatments could reduce requirements for instructor support

for this type of motivational training, the findings of no overall superiority of CAI

vs. instructor augmented (II) conditions were somewhat disappointing. For this

reason, some exploratory analyses of potential individual differences in

subsequent student performance as a function of treatment condition were

conducted. These analyses generally indicated that the CAI condition is at least

equally effective for approximately half of the students (ie., those students of

high general ASVAB ability and those students with low perceptions of

competence or self-esteem). That is, there is some preliminary evidence that for

students of high general ability and for students who have feelings of

A" incompetence, the CAI treatment provides an effective and perhaps less

*, threatening environment for learning self-management and personal responsibility

skills than the instructor/group (II) treatment. These findings imply that there

may be systematic and reliable individual differences that could be used in

differential treatment format assignments, thereby reducing some of the

instructor support requirements for the motivational program as well as

capitalizing on the use of microcomputer technology for this type of training.

Instructor and student comments and contractor observations generally

supported the positive results found for the Motivational Skills Training Program.

Students and instructors were positive and enthusiastic about the program and felt

44
the prograpn was beneficial for application both in subsequent technical training

and in studenti personal lives. In addition, although students generally preferred

the instructor augmented vs. CAI conditions, they were positive about the CAI

-.,- ..

versions and instructors also indicated that the CAI versions were easier to .. .

implement. In total, then, the Motivational Skills Training Program appears to

have achieved the goal of providing needed and relevant training and to have

contributed to improvements in students' subsequent technical training

performance. Furthermore, Phase H evaluation findings generally support Phase I

evaluation findings, ie., they substantiate the importance of instructor and group

experiences as enhancements to this type of skill training, at least for some types

of students. Additional implications of study findings are discussed in the next

section, followed by recommendations for future implementations of the

motivational program in Army technical training contexts.

Implications of Study Results

major assumption of the current study was that well designed CAI

introductory and practice segments, implemented via a rich microcomputer/audio

technology mix, could provide the degree of personalization and simulation of

critical instructor and group functions to offset or partially reduce instructor and

group process requirements for this type of motivational skills training. Main

effect findings with various combinations of CAI and instructor/group supported

conditions did not bear out this assumption. This raises the possibility that, in line

with Clarks (1983, 1984) arguments concerning the influence of media on

learning, that the CAI/audio combination was not sufficiently matched to the

content and method of instruction required in this training context. The main

effect findings also reinforce Jernstedt's (1983) point that CAI technologies

cannot be used as replacements for teachers and group learning, and that there is

a need for a synergetic combination of the human and computer functions to

achieve maximum instructional effectiveness.

45

%,I*. ~ -,.. ~?*- .~- .A Y 
The exploratory individual difference analyses have suggested, however,

that the CAI enhanced version of the motivational training was at least equally

effective for some types of students. Of the individual difference variables

available for inclusion in these exploratory analyses, the findings with the general. .

ability measure (ASVAB General) are not surprising. That is, a number of studies

have found CA! or other multimedia treatments to be equally effective as

traditional instructor/group methods for high ability students (eg., Clark, 1984;

Kulik, Bangert, & Williams, 1983; McCombs & McDaniel, 1983). On the other

hand, the findings that students low in perceptions of competence or self-esteem

subsequently perform- better if they received the CAI enhanced version vs. the

istructor/group version of the motivational training are, on the surface,

somewhat puzzling. A plausible explanation, however, may be derived from a

consideration of the characteristics of individuals who are low in perceptions of.. "

competence or self-efficacy.

Albert Bandura's (1982a, 1982b) theory of self-efficacy suggests that

individuals low in perceived competence are those who do not judge themselves as

capable of handling particular situations, including interpersonal situations.

Because of their low feelings of personal adequacy, low self-efficacy/self-esteem

individuals are often times threatened in interpersonal or social situations and

fearful of having their perceived inadequacies exposed. For these types of

individuals, then, it is reasonable that the more individual and nonpersonal

medium of CAI may provide a less threatening learning environment, particularly

for this type of self-development training that requires considerable self-analysis

and self-exposure. As Bowman (1982) has pointed out, some advantages of

computer-based technologies include (a) freedom from fear of reprisal, ridicule,

or rejection; and (b) provision for active involvement in tasks that are based on a

high probability of success. Thus, it may be that the medium of CAl, used for

46

, A- *°. ° •
motivational skill training that aims in part at enhancing individuals' perceptions

of competency and personal control, is a more optimal treatment mode for those

students whose initial perceptions of self-efficacy are low.

The preceding speculations do, of course, need to be verified by further

research. As pointed out by Sirnutis (1979) and Sprecher and Chambers (1980), the

real potential of CA! lies in its ability to provide individualized, standardized, and

efficient instruction to learners, particularly adult learners who require remedial

training. These individualization issues thus need to be adequately explored, along

with the general individualization issue of what types of students are most in need

of this type of motivational training, such that differential assignment to the

motivational program can be made prior to differential assignment of individuals

who need the training to particular training formats. There is little question

based on the research reported here that instructors and the group process play a

critical role in the success of motivational training and the success of

individualized computer-based approaches in general (McCombs, in press). This

and additional research issues to be explored with the Motivational Skills Training

Program, therefore, form a basis for the following recommendations for future

implementations of this program in Army technical training.

Recommendations for Future Implementations of the Motivational Program

To capitalize fully on the potential benefits of the Motivational Skills "

Training Program for enhancing subsequent student performance in technical

training, research needs to be addressed at systematically exploring the issues

listed below.

(1) One expectation in the current study was that the Motivational Skills

Training Program might contribute to increases in students' perceptions of

competence, abilities to cope with stress, critical thinking skills, and instructor

ratings of self-management skills. The results indicated no significant pre/post

47
differences in the first three variables in this set, and no significant posttraining

differences in instructor ratings for experimental vs. control conditions. These I

findinps raise the question of what was learned Onowledges and skills) as a result

S . AI

of the motivational training, as well as differences in this learning as a function

of the CAI vs. instructor/group training formats. It also raises the need to

establish the relationship between what is learned in the motivational program

and subsequent performance in technical training. It is recommended, therefore,

that future implementations of the motivational program include an analysis of

what is learned, a development of well defined measures of the identified

knowledges and skills, and an exploration of relationships bet,,een these measures

of what is learned in the motivational training and subsequent student

performance.

(2) The exploratory individual difference analyses were suggestive of the

potential benefits of the CAI enhanced version of the Motivational Skills Training

Program for certain types of students. As indicated in the preceding section,

there is the need to systematically explore (a) what types of students would most

benefit from the motivational program and 0) of thote students who would

benefit from the program, what individual difference characteristics are

predictive of the CAI vs. instructor/group version of this program. It is

recommended, therefore, that in conjunction with investigations of what is

learned in the motivational program, research be conducted in future

implementations of the program that also includes investigations of individual

differences predictive of what types of students need the motivational program

and particular program formats.

(3) The questions in the areas of what is learned, individualization, and

optimal treatment format for the Motivational Skills Training program need to be

addressed in the larger context of additional improvements that can be made to

48
,* I. *'

the program that will enhance skill maintenance and transfer to other training

contexts (eg., subsequent field unit training). The program has currently been

implemented as an intensive skills training course which students receive prior to

entering their technical training course. The primary skill maintenance strategy,

in addition to those that encourage students to continue to monitor their selfmanagement/self-motivation skills following initial training, has been to rely on

course instructors trained to administer the motivational program as the vehicle

for reminding and encouraging students to continue using newly developed

motivational skills. No attempts have yet been made to systematically evaluate

the effectiveness of -more potent and perhaps more practical skill maintenance

strategies such as embedding reminders to students to employ various

motivational strategies at critical points in their subsequent technical training.

(e.g, when they encounter problems with stress or need to use problem solving or

goal setting skills). A final recommendation is thus to analyze, identify, and

evaluate a set of skill maintenance strategies that can operate within the total

context of an individualized Motivational Skills Training Program.

The current evaluation of CAI enhancements to the Motivational Skills

Training Program has contributed to our understanding of the potential benefits of

this program and of augmentations possible with microcomputer/audio

technologies. Once the questions raised in this section are answered, however,

the full benefits of the motivational program can be assessed. In addition, the

benefits of microcomputer/audio technologies for offsetting or reducing

instructor support requirements for this type of skills training can then be more

completely understood.

49

... . . . . .. . .
REFERENCES

Bandura, A. (1 92a). Self-efficacy mechanism in human agency. American

Psycholost, 37(2), 122-147.

Bandura, A. (192b). The psychology of chance encounters and life paths.

American Psychologist. 37(7), 747-755.

Barger, R N. (1983). The computer as a humanizing influence in education.

Technological Horizons in Education 3oumnal, 10(7), 109-111.

Bloom, & S. (1984). The search for methods of group instruction as effective as

one-to-one tutoring. Educational Leadershi 41., 4-17.

Bouton, C., & Garth, R. Y. (1983). Students in learning groups: Active learning

through conversation. In C. Bouton and R. Y. Garth (Eds.), Learning in groups.

San Francisco: 3osey-Bass Inc , Publishers.

Bowman, R. F., 3r. (1992). A "Pac-Man" theory of motivation: Tactical

implications for classroom instruction. Educational Technology, 22(9), 14-16.

Burns, R. E., & Davisson, W. L (1979, February). Faculty reactions to

implementing CAI tutorials: What can happen and what to expect. Computer based education: Mission of the future, Volume 3. Proceedings of the Annual

Convention of the Association for the Development of Computer-Based

Instructional Systems, San Diego, CA.

Clark, P.. E. (1983). Reconsidering research on learning from media. Review of .. . *

Educational Research 53(4), thEc-io9.nal

Clark, R. E. (199, April). Learning from computers: Theoretical problems. Paper presented at the annual meeting of the American Educational Research |- :

Association, New Orleans.

Covington, M. V., & Omelich, C. L. (1981). As failures mount: Affective and

cognitive consequences of ability demotion in the classroom. Journal of

Educational Psychology. 73(6), 796-808

Cross, K. P. (1976). Accent on learning. San Francisco, CA: 3osey-Bass Inc.,

Publishers.

Cubberly, W. E, Omizo, M. M., & Longano, D. M. (1984, 3anuary). The effects of

group counseling on self-concept and locus of control among learning disabled

children. Paper presented at the annual meeting of the Southwestern

Educational Research Association, Dallas.

David, A., & Williams, R. L. (1980, April). Collectivized C.A.I. Paper presented

at the annual meeting of the Association for the Development of ComputerBased Instructional Systems, Washington, DC.

51

I0

r• r.:. .:..
Dudle, W.(1979, October). Using the computer to foster creative interaction

amanx students. Urbana, IL: Illinois University, Computer-Based Education

Laboratory.

Federlco, P. A. (1981, April). Individual differences and mastery learning in

computer-managed instruction. Paper presentd at the annual meeting of the

American Educational Research Association, Los Angeles, CA.

Gagne, E. D. (1983, April). Teaching thinking skills: Suggestions from research.

Paper presented at the annual meeting of the American Educational Research

Association, Montreal.

Gagne, R. M, Wager, W., & Rojas, A. (1981). Planning and authoring computerassisted instruction lessons. Educational Technology, 21(9), 17-21.

Ginther, D. W. (1983). Micro's are talking back in the classroom: The promise of

speech technology in education. Technological Horizons in Education 3ournal, L1,105-107..

3ay, T. B. (1983). the cognitive approach to computer courseware design and

evaluation. Educational Technology, 231), 22-26.

3ernstedt, G. C. (1983). Computer enhanced collaborative learning: A new

technology for education. Technological Horizons in Education Journal, 0(7),

96-101.

Jonassen, D. H. (1984). The generic disc Realizing the potential of adaptive,

interactive videodiscs. Educational Technology, 24(l), 21-24.

Kulik, on secondary 3., Bangert, school R., & students. Williams, Journal G. (1983). of Educational Effects of Psychology, computer-based 7 5 I), teaching 19-26. :'

Lepper, M. R., & Malone, T. W. (n press). Intrinsic motivation and instructional

effectiveness in computer-based education. In R. E. Snow and M. 3. Farr

(Eds.), Aptitude, learning, and instruction: Ill. Conative and affective process

analyses Hillsdale, NJ: Erlbaum.

Lubin, D. A. (1984). What's right with CBT? Data Training, 3(7), 18-19, 22.

Malone, T. W., & Lepper, M. R. (in press). Making learning fun: A taxonomy of

intrinsic motivations for learning. In R. E. Snow and M. 3. Farr (Eds.),

Aptitude, learning, and instruction: III. Conative and affective process

analysis. Hillsdale, NJ: Erlbaurn.

McCombs, B. L. (1980). The development of a measure of critical thinking skills.

Unpublished working paper.

McCombs, B. L. (in press). Instructor and group process roles in computer-based

training. Educational Communication and Technology Journal.

McCombs, B. L., Bruce, K. L., & Lockhart, K. A. (in press). Enhancements to

motivational skill training for military technical training students: Phase I

52 ,jj:
evaluation study report. Alexandria, VA: Army Research Institute for the

Ie1mioral and Socal Sciences.

McCombs, B. L., & Dobrovolny, 3. L (1980, June). Rationale for the design of

spe flc t Des of student skill training in the conative, affective, and cognitive

sk doman. it (Interim Technical Report). Lowry AFB, CO: Air Force Human

Resorces laboratory.

'. -.- ,

McCombs, L L., & Dobrovolny, 3. L (1982, December). Student motivational "-

skill traiini package: Evaluation for Air Force technical training. (AFHRLIP-2-31). Lowry AFB, CO: Air Force Human Resources Laboratory.

McCombs, B. I., Dobrovolny, 3. L., & Lockhart, K. A. (1983, June). Evaluation of

the CMI instructor role training program in the Navy and Air Force. (NPRDCSR-8-43). San Diego, CA: Navy Personnel Research and Development

Center.

McCombs, B. L., & McDaniel, M. A. (1983). Individualizing through treatment

matching: A necessary but not sufficient approach. Educational

Communication and Technology 3ournal, 31(4), 213-225.

Michaelson, L. K. (1993). Team learning in large classes. In C. Bouton and R. Y.

Garth (EdsJ, Learning in groups. San Francisco: 3osey-Bass Inc., Publishers.

Neale, D. C. (1983, April). Specifications for small group activities in

instructional designs. Paper presented at the annual meeting of the American

Educational Research Association, Montreal.

Podenski, R. S. (1984). Implications of electronic learning technology: The

future is now! Technoloical Horizons in Education Journal, 11(g), 118-121.

Siegel, M. A., & Simutis, . M. (1979, February). CAI for adult basic skills

training. Two applications. Computer based education: Mission of the future,

Volume 3, Proceedings of the Annual Convention of the Association for the

Development of Computer-Based Instructional Systems, San Diego, CA.

Simutis, Z. M. (1979, April). CAI as an adjunct to teach basic skills. Paper

presented at the annual meeting of the American Educational Research

Association, San Francisco.

Sprecher, 3. W., & Chambers, 3. A. (1980). Computer assisted instruction:

Factors affecting courseware development. 3ournal of Computer-Based

Instruction, 7(2), 47-57.

Stasz, C., Winkler, 3. D., Shavelson, R. 3., Robyn, A. E., & Feibel, W. (1984,

April). Staff development for instructional uses of microcomputers: The

teacher's perspective. Paper presented at the annual meeting of the American

Educational Research Association, New Orleans.

Steinberg, E. R. (1979, February). Learner interaction in computer-assisted

instruction. Computer based education: Mission of the future, Volume 1.

Proceedings of the Annual Convention of the Association for the Development of Computer-Based Instructional Systems, San Diego.

53

. °

-= . ' . " ' ," ' .- %"% ' .. ,.1 . . .".".". ' '%-, . - '. ., . - .". . . :> . ' "."."."." " .* 
Stinard, T. A., & Dolphin, W. D. (1981). Which students benefit from self-paced

mastery instruction and why. Journal of Educational Psychology. 73(5), 754-

763.

Swing, S. R., & Peterson, P. L. (1982). The relationship of student ability and

small group interaction to student achievement. American Educational

Research Jotrnal. 19(2), 259-274.

Thompson, S. B. (1980). Do individualized mastery and traditional instructional

systems yield different course effects in college calculus? American

Educational Research 3ournal 17(3), 361-375.

Watson, G., & Glaser, & M. (1964). Watson-Glaser Critical Thinking Appraisal.

New Yorl Harcourt, Brace, and World, Inc.

p..:: p.:

'p 54

Ps::?

I.

. . .

? .- -..

--- .. .-- ~ ~ ~ ~ ~ & -k.. .
Appendix A:

Critical Thinking Skills Measure

and Pretest Instructions

,A

.55
N*ZDA___

CRITICAL THINKING SKILLS

Directions:

The ability to think involves many critical skills. Your task on this test is

to read each item carefully and then pick the answer (the conclusion or

statement) which you think follows froq the information given in the item.

Thus, you are to read and judge each conclusion or statement for each item, and

pick the one which necessarily follows from what is said in the item, or which

follows beyond a reasonable doubt. Don't be concerned if some of the items

contain silly, funny, or untrue information. Your job is to reason logicaly

from the information you have in the item.

1. Some Russians want to control the world. All Russians vant a better life.

Therefore,

a. Some Russians who want a better life also want to control the world.

b. All Russians who want a better life want to control the world.

c. All Russians do not like their present standard of living.

d. Some Russians who want a better life do not like their standard of

living.

2. On one technical training base there are 52 courses in five technical

schools. Each course contains from 10 to 40 students. Therefore,

a. No technical school contains fewer than 10 courses.

. b. Most courses contain more than 15 students.

c. There are more than 500 students in these technical schools.

3. At the end of the first block of training, students who were on the first

shift averaged 10 points higher than students on the second shift on the

end-of-block test. The instructors on the first and second shifts used

different methods of teaching. Therefore, •".-.-."

a. The instructors on the first shift had better discipline than the

instructors on the second shift.

b. The students on the first shift were smarter and learned more easily

than the students on the second shift.

c. The higher block scores achieved by students on the first shift were due

to the better teaching method of that shift's instructor.

d. The performance of students and the kind of teaching method on the first

shift were better than on the second shift.

56

.'..
4. On a certain base where school attendance rules were rigidly enforced, it

was found that only 80 percent of the students had a perfect attendance

record. Among those on the second shift, .however, 95 percent had a perfect

attendance record. Therefore,

a. If students who tended to be absent were put on second shift, their

school attendance would improve.

b. Less than 80 percent of the students who were on second shift had a

perfect attendance record. A

c. Strict enforcement of attendance rules on that bave prevented 20 percent

of the students from being absent.

5. Several studies have shown that military recruits from the South make lower

scores on the Armed Services Vocational Aptitude Battery than recruits from

the North. If recruits who were born in the South moved to the North at a

fairly early age, however, their scores were nearly the same as recruits

from the North. Therefore,

a. Southern recruits who move to the North before they can talk will have

the same scores on that battery of tests as Northern recruits.

b. The higher scores of recruits from the North is due to the better

schools they attended in the North.

c. Recruits who are born in the South and stay in the South until they join

the military will all score lower on that battery of tests.

* 6. When Sam entered the military, his posture was poor, he dressed sloppily,

had very few friends, was not comfortable around other people, and in

general was quite unhappy. A close friend said he should go visit the base

chaplain. Sam took this advice, and after 3 months, he carried himself

well, dressed better, was more at ease and popular, and in general felt much

happier. Therefore, ___ ..

a. Had it not been for the chaplain, Sam would not have improved.

b. Sam's improvement was caused solely by the help given him by the

chaplain.

" c. Sam's friend's advice to see the chaplain turned out very well in Sam's

case.

57•

I...

57-' "

*.. .-.. .• .. . . °
S=EW M=TIVATICNIU, SIL TRMU:

P ETST BATTERY lNSTRUCTICNS

You have been selected as part of a group of students who will be

participating in a research study being conducted by the University

of Denver. As part of this study, you are being asked to fill out

sane questionnaires. Your scores on these measures are being

oollected for research purposes only. They will not become part

of your personnel file and will be kept cac letely confidential.

Please carplete the questionnaires, answring them as honestly as

you can. Put your name and today's date on each test form.

Thank you for your cooperation.

58

<.4

* S. . ".- 
.41

59eldX~
STUDENT MOTIVATIONAL SKILL TRAINING

Instructor Guidelines ' .z

I. Introduction/Orientatlon "' .-

A. Purpose of Motivational Skill Training Program

1. Helping students made the transition from adolescence to adulthood:

teaches students how to be good self-managers, how to take charge of their lives and futures, and how to become a responsible adult.

2. Giving students concepts and skills for defining themselves, their

values and career goals, and for learning to be independent, responsible, and in control through specific skill training in goal

setting, stress management, effective communication, and problem

solving strategies.

B. Background of Program

1. Prior research with similar technical training students indicated

that students performing poorly lack skills for taking personal

responsibility and motivating themselves.

2. Development and evaluation of this program led to the findings that

students receiving the motivational skill training, compared to

students not receiving the training, had higher block test scores

and lower test failure rates; students receiving the training also

had higher motivation and involvement in doing well in technical

training.

3. Purpose of the current research is to further explore the benefits

of this type of skill training, to explore the enhancements possible

through the use of computer-assisted instruction (CAI), and to define

the appropriate role for the instructor in this type of training.

C. Description of Program

1. Premise: Self-management skills are learned.

2. Format: Consists of the following elements:

a. Seven printed self-instructional modules of low density with

lots of visuals, embedded questions, and practice exercises;

modules are consummable.

b. CAI introductions and practice segments which can be used in

lieu of instructor introductions and instructor-led small

group practice sessions.

3. Length: Each module takes about 2 hours to read; the entire program

is scheduled to be completed in 5 days, at 5 hours per day.

4. Special Features: Each module includes levels of effort, rationale,

key words, glossary, objectives, personally relevant case histories

and examples, posttest.

60
2

5. Module Topics include:

a. Introduction: Explains the concepts of personal responsibility

and positive self-control; introduces students to the basic

strategies for taking control of their lives as well as the

purpose and organization of the whole program.

b. Self Knowledge: H lps students explore their values and beliefs,

ine what is important to them, and resolve value conflicts.

c. Career Exploration: Helps students explore- their career inter- -

ests and goals and to relate these to their military specialty.

d. Goal Setting: Helps students learn a systematic approach to

setting goals and pursuing them; helps them learn to make

decisions.

e. Stress Management: Helps students understand what stress is and

aging stress in their lives.

f. Effective Communication: Helps students deine their typical

communication style and helps them learn how to use effective

"I" messages and active listening skills.

g. Problem Solving: Helps students learn to apply a systematic

approach to identifying and solving problems; provides a summary

of skills learned in the program.

D. Experimental Conditions

1. Control: Students in each entering class who are not assigned to

one of the four experimental conditions; they do not receive the

motivational skill training, but do receive the pretest battery,

delayed posttests, and are rated by instructors on the instructor

questionnaire. (Condition A)

2. CAI Introductions/CAI Practice: Students who are assigned to a

version of the motivational skill training wherein they receive - --

CAI introductions to the program and each module, go off and read

the printed module on their own, then return to the computer for

a CAI practice session. (Condition B)

3. CAI Introductions/Instructor Practice: Students who are assigned

to a version of the motivational skill training wherein they receive

CAI introductions to the program and each module, go off and read

the printed modules on their own, then join an instructor-led small

group practice session. (Condition C)

4. Instructor Introductions/CAI Practice: Students who are assigned

to a version of the motivational skill training wherein they receive

instructor introductions to the program and each module, oo off and

read the printed module on their own, then go to the computer for

a CAI practice session. (Condition D)

61

*J°**- "%"
3

5. Instructor Introductions/Instructor Practice: Students who are .

assigned to a version of the motivational skill training wherein

they receive instructor introductions to the program and each

module, go off and read the printed module on their own, then

join an instructor-led small group practice session. (Condition E)

E. Instructor's Role in Motivational Training (General)

1. Assists students in the learning and application of concepts and

skills taught in the motivational program.

2. Monitors student learning under one of the four training methods

(Conditions B through E).

3. Administers student measures pre and post motivational training,

and delayed posttest measures after students complete the first

block of the Electronics Communications course. (See Procedures

Chart for Measures.)

4. Administers end-of-module tests to students in Conditions B through

E after the practice sessions; scores tests and provides individualized remediation. (See Scoring Keys and instructions.)

5. Fills out instructor questionnaires to evaluate how well each

control and experimental student is applying skills at the end

of the first course block and at the end of the Electronics Communications course.

6. Fills out Student Data Sheet for each group of students, keeping

track of times starting and finishing each introduction and practice session during the motivational training. (See Data Sheet.)

7. Groups completed tests by type and student group; saves tests for

collection by research team.

F. Instructor Procedures for Each Training Method

1. CAI Introductions/CAI Practice (Condition B): Instructor briefly

explains that students are part of a research study being conducted

by the University of Denver to look at the benefits of a special

training package; instructor introduces students to the computer,

explains in what order they will work through the training program

(CAI intros, read modules, CAI practices), and tells students that

instructor will be available to answer questions or explain concepts as necessary; instructor will also explain procedures for

end-of-module tests (taken at end of practice session, scored by

instructor, remediated if necessary).

2. CAI Introductions/Instructor Practice (Condition C): Same as in

Condition B, except that the instructor will lead the practice

sessions in small groups. During the practice sessions, the

instructor will use scripts of characters relevant to each module.

Students will not work with the CAI practice sessions.

62

, . '. * ,***, -*- . - , - * *t - . C ,.," .. ..-.. ,. .,-,.• % ' . . ,., . .- .. ,,. .,,.....
.w a•a a4at .~- ~ " -. -~r-. -. " ...

4

3. Instructor Introductions/CAl Practice (Condition D): Same as in " Conditions B and C, except that the instructor will provide the

introductions to the program and each module. During the introduction sessions, the instructor will use scripts of characters

relevant to each module. Students will not see the CAI introductions.

4. Instructor Introductions/Instructor Practice (Condition E): Instructor will explain that students are part of a research study as in

other conditions and will introduce students to the program and to

each module; students will read each module on their own and then

participate in instructor-led small group practice sessions. During

the introduction and practice sessions, the instructor will use

scripts of characters relevant to each module. Students will not

see the CAI introductions or practice sessions.

G. Instructor Training Procedures

1. Read seven modules on own, per training schedule.

2. Practice skills in group sessions.

3. Learn procedures for operating computer and for introducing each

module and conducting group practice sessions.

Questions and Discussion

Notes:

6 3"* '

a.-.°

a..63

: • -•~~~~~~~~~~~~~~.. ... .... °.-.°,...... ,........1-,°°...° . .. .. .... ..... . .-.. .° '•,-o-'
S 5

I I. Concepts and Skills in Modules

A. Important Concepts from Program Introduction and Module 1 (Introduction)

1. Introduction to the Student Skills Program

a. Purpose of Program: How to be a good self-manager, to take

charge of your life and future, and to become a responsible

adult.

b. Premise of Program: Self-management skills are learned.

c. Introduction of Characters/Problems in Each Module: Use of

script in describing characters, his/her problem, and how

these problems are resolved by applying skills learned in

the program.

2. Introduction to Module 1

a. Rudy's Problem: Introduce through use of script.

b. Maslow's Hierarchy: Concept of taking responsibility for

meeting our own needs.

c. Self-Fulfilling Prophecy: Concept of how positive or negative

attitudes influence positive or negative outcomes; role of

self-talk and imagination in changing self-image and performance.

d. Explanation of Module Format

Rationale

Key Words

Glossary

Objectives

Levels of Effort

e. Explanation of Study Strategies

Underlining

Posttests

Embedded Questions

3. Practice for Module 1

a. Recap on Rudy: What he learned through use of script.

b. Self-Talk Practice: Identifying positive and negative self-talk.

c. Imagination Excerices: Choosing images that help students feel

positive in different situations. (Instructor exercise of brainstorming to identify example situations and their corres- ponding positive images.)

64

........................................... ?.d•!A - *-{-
6

d. Controlling/Changing Bad Attitudes and Beliefs: Importance

of students working through a systematic process in changing

negative attitudes and irrational beliefs about themselves

and their abilities to take positive sel -control. (Instructor L

exercise of generating meaningful case histories that demon- L

strate the benefits of a systematic approach.)

questions and Discussion .

Notes:

.,..

S.:?

*% "°44

.. -..

.1..

4..-

.4 . ".

.

" ~~65 ":."'

" ".,Z,.,, .}" "-', " Z - ."}' ' ' "''., ' , .; ".'.. ' " ,, ", " ".,4, ". - -' " - -. ". ., - . . . .. . ... * " .
7

B. Important Concepts from Module 2 (Self-Knowledge)

1. Introduction to Module 2

a. Eric's Problem: Introduce through use of script.

b. Purpose of Module t: Exploring values as a way of defining

yourself and what's important to you; knowing how much of

yourself you feel comfortable sharing with others through

introduction to Privacy Squares.

c. Role of Values: Their use in making decisions and setting

goals.

d. Self-Contracting: It's importance as a way to plan changes;

concept of making a committment to change.

e. Types of Values Exercises: Explanation of rank ordering,

values continuum, unfinished sentences, are you someone who . exerci ses,..: -.

2. Practice for Module 2

a. Recap on Eric: What he learned through use of script.

b. Self-Contracting Practice: Use example of Eric's problem.

c. Sharing Exercise: Discussion of personal examples of why

it's important to know yourself.

Questions and Discussion

Notes:

66

,'.-"1

S a A .' °* °
8

C. Important Concepts from Module 3 (Career Exploration)

1.Introduction to Module 3

a. Larry's Problem: Introduce through use of script.

b. Decision Making Process: It's importance and relationship

to career decisions.

c. Self-Directed Search (SDS): Use of the SDS to find career

clusters that match personality types and interests; explan- 4-

ation of six job categories (Realistic, Investigative, Artistic, Social, Enterprising, Conventional) "

2. Practice for Module 3

a. Recap on Larry: What he learned through use of script.

b. Decision Making Steps: Practice identifying steps in the

decision making process using case history examples.

(Instructor exercise of generating examples.)

c. Evaluating Good/Bad Decisions: Practice with relevant

case histories. (Instructor exercise of generating examples.)

d. Evaluating Risks and Costs: Practice with case histories.

(Instructor exercise of gnerating examples.)

e. Summary: Importance of investigatinq and planning in

making career decisions; getting the most of current military

training. %- ° -...°

Questions and Discussion

Notes:

67
9

D. Important Concepts from Module 4 (Goal Setting)

1. Introduction to Module 4

a. Lucy's Problem: Introduce through use of script.

b. Relationship betwken Goal Setting and Career Exploration:

Importance of systematically setting and planning for short

and long term goals.

c. Importance of Goals: Their function in motivating individuals 4 .

to make changes identified in Modules 2 and 3.

d. Goal Setting Skills: Use of questioning, imagination, positive '..,..

self-talk, brainstorming, and self-rewards in putting plans

into action.

2. Practice for Module 4

a. Recap on Lucy: What she learned through use of script.

b. Identifying Good/Bad Goal Statements: Practice with examples.

(Instructor exercise of generating examples.)

c. Evaluating Alternatives: Practice evaluating alternative

plans for achieving goals from examples. (Instructor exercise

of generating goals and alternative plans.)

d. Identifying Good Rewards: Practice identifying good rewards

for specific situations. (Instructor exercise of generating

situations and possible rewards.)

Questions and Discussion

Notes: ""''-

68

A . . . =.

*4. * -. ' . .- .....* :- 4.-. . .5*.4* *°.4° *~ . .4 5- -... ~ ~ * ;~ - ~-.: * .- .- :
*10 o-'-..

E. Important Concepts from Module 5 (Stress Management)

1. Introduction to Module 5

L

a. David's Problem: Introduce through use of script.

b. Signs of Stress: ,Go over list in module and discuss four -. , factors that determine how much stress a person feels. 2V

c. Role of Perceptions and Mistaken Beliefs: Concept that

stress is many times caused by mistaken beliefs that distort

perceptions and cause feelings of anger or anxiety.

d. Stress Management Strategies: Introduce types of do, think,

and say strategies students will be learning.

e. Relaxation Tape: Play tape for students.

2. Practice for Module 5

a. Recap on David: What he learned through use of script.

b. Identifying Mistaken Beliefs: Practice with case histories.

(Instructor exercise of generating 4-5 case histories.) I

c. Countering Mistaken Beliefs: Practice with same case his- ..

tories used above.

d. Using the Worksheets: Practice filling out from examples;

emphasize importance of monitoring stressful situations.

Questions and Discussion

Notes:

469

' ....T .: .. . . ..

:It.-..6?

.,-.

is- , .

w , ,..

.:1 , . " , ' w ' , " " . , " w % , . , . ..' ' " , " ' -" ." . .' . . " -' ." -" - -" -' ." - ' " . -• ." ." - ." .,. ' -' ' . .-.

.. d . ..- ,, . . ia .< L ; ,, . a , , _. . , . . . .. ._, . , _. - . : . . . - , - -. _, : , . _ _, .. _ . _, ,- . -.- ,. . . . .
F. Iatportant Concepts from Module 6 (Effective Communication)

1. Introduction to Module 6

a. Kris and Scott's Problems: Introduce through use of scripts.

b. Communication Styles: Introduce styles of aggressive, nonassertive, and assertive and concept of basic rights in

helping us to be responsibly assertive.

c. Effective Communication Skills: Introduce types of listening

and talking skills students will learn and why these are

important in achieving goals.

d. Identifying and Communicating Feelings: Concept that what

is said and the way it's said influence our feelings and

reactions.

2. Practice for Module 6

a. Recap on Kris and Scott: What they learned through use of

scripts.

b. Identifying Communication Styles: Practice with case histories. (Instructor exercise of generating case histories.)

c. Identifying Feelings: Practice with specific messages.

(Instructor exercise of generating examples.)

d. Generating Responses to Difficult Situations: Practice with

accepting criticism, saying 'no', expressing anger. (Instructor exercise of generating situations in these categories.)

e. Using the Worksheets: Practice filling out from examples.

Discuss why this practice is important.

Questions and Discussion

Notes:

70

.9.

.o9. -

I70
12

G. Important Concepts from Module 7 'Problem Solving)

1. Introduction to Module 7

a. Summary of Each Character: What they learned (from scripts)

and how all these relate to problem solving.

b. Problem Solving Steps and Skills: Overview of process and

skills involved.(questioning, brainstorming, imagination,

self-talk).

c. Importance of Using a Systematic Process: Show how this helps

with examples of Hugh and Angela from scripts.

2. Practice for Module 7

a. Recap on Hugh and Angela: How the problem solving process

helped and the importance of planning.

b. Defining the Problem: Practice with case history. (Instructor

exercise of generating appropriate case history.)

c. Generating and Evaluating Alternatives: Practice with same

case history as above.

d. Making an Implementation Plan: Practice with same case history.

e. Implementing and Evaluating the Plan: Practice with same case

history which exemplifies how to handle setbacks.

f. Sunmary: Discussion of how all the skills work together;

reminder to practice skills and refer back to modules.

Questions and Discussion

" Notes:

7..

..

.%

"L71

.'. -.-
.-.

U .. U Q) Q .

u) 41 21-0 i414

s..

U..CCL

I-m

S .41 0-.IA,.-.-A

o

E S..9u C

41 to:

- S-.4A n.IA:.(A,(

LC

IA

- '. . .-

LU jc 0 

EU .'i.-

0 4j- 0 0 S- cu C

- ra---

g-. 44 inW .

to to 0 S-S-' .. & o , " - •a 

c. w ~ wn un LnCn CAE -&j 00 A0S 41 0 0

OZL -- - r_ U. u o s0~~~~% to0 u-_) 0S

0.

Lii-

.o -7 2,, 

LC - C J - l 41L. . ' ,J ,-. -OL[ CO "- CJ ",' "-

I n :: .J4;: 4: ; . . . tfl: : i..- : 41 .' Li:: . :. ;.4 - -I.-- .'..-4-,- n 1---- -.- :..-:'-:-'* :.
APPLE lie Instructions

System Start-Up

1. Insert appropriate audi.o cassette. Check to be sure tape is rewound

to the beginning. If tape 'is not rewound, follow rewinding instructions in next section.

2. Insert appropriate diskette into disk drive per the instructions in

the third section.

3. Turn on computer and monitor. The switch for the computer is on the

back left hand side of the computer. The switch for the monitor is

on the lower right front of the screen.

4. Press "PLAY" button on the audio cassette player.

5. Put head phones on.

6. Follow instructions on computer screen.

System Shut-Down

1. Press "STOP" button on audio cassette player after the computer has

stopped the tape. -

2. Pull out "REMOTE" plug (the smaller one) from audio cassette player,

press the "REWIND" button, and rewind audio cassette tape to the

beginning. When tape is rewound, press "STOP" button.

3. Reinsert the "REMOTE" plug into the audio cassette player and turn

off the computer and monitor.

4. Remove audio cassette tape and diskette. Return to your instructor.

Care of Diskettes

1. General Handling

a. Never touch the exposed magnetic portions of the diskettes.

b. Store diskettes in envelopes at all times. Remove from envelope

only to insert in disk drive.

c. Although diskettes are made of flexible material, they should

Fnever be bent or folded.

2. Inserting Diskettes

a. Computer should be "off" when inserting diskettes into disk drive.

b. Open disk drive door by lifting up.

73
APPLE lie Instructions (Continued)

c. Remove diskette from storage envelope.

d. Insert diskette with label up and notch on lower left side.

e. Slide diskette into drive. If it does not slide in smoothly,

do not force it. Simply pull it out and try again.

f. Close disk drive door.

g. Caution: Never insert or remove diskettes when the red disk

drive Tight is on.

3. Removing Diskettes

a. Turn computer off.

b. Open disk drive door.

c. Remove diskette, holding it by the label.

d. Insert diskette in storage envelope. .. .9 I

-V

74. ..

i

-' .:,N . :.

, .'.1

* 7 4 . " *

2.. 7 .. . :::.::-

." .' '.".'" " " " ;" '- '. " v -.-'.', -.' "" .,,.. . . .. .. -. . . -. - .. . • .. .-~ .-. -. ... .. - '*' :
Diskette and Audio Tape Labeling

Note: There are ten sets of both the diskettes and audio tapes. They are

Ta'led and organized as shown below. :

Diskettes

(1) Red Labels - Diskette #1

Intro to Program NZ

Intro and Practice for Module I

(2) Blue Labels - Diskette #2

Intro and Practice for Modules 2 and 3 -

(3) Yellow Label: - Diskette #3

Intro and Practice for Modules 4 and 5

(4) Green Labels - Diskette #4

Intro and Practice for Modules 6 and 7

Audio Tapes

(1) Audio Tape #1

Side 1 - Intro to Program, Intro to Module 1

Side 2 - Practice for Module 1

(2) Audio Tape #2

Side 1 - Intro to Module 2

Side 2 -.Practice for Module 2

(3) Audio Tape #3

Side 1 - Intro to Module 3

Side 2 - Practice for Module 3

(4) Audio Tape #4 ,"

Side I - Intro to Module 4

Side 2 - Practice for Module 4

(5) Audio Tape #5

Side 1 - Intro to Module 5

Side 2 - Practice for Module 5

(6) Audio Tape #6

(Page 75) iiiii

Side 1 - Intro to Module 6

Side 2 - Practice for Module 6

(7) Audio Tape #7

Side 1 - Intro to Module 7

Side 2 - Practice for PlQdu 7. ..... _ .__ 2 . _. ._ . . .: . .. :.,. =_ _ _ _. .. .. .. , r" ';:. ....• ';":'-.-;?:?..;..--;..?. " % ' , """ ',.'1...i.-:......---'....-:.-i.i'-.-.-:...-..:.'.-.. '.'.-/ " "-.--- '-2.2
SIUDl MTIVA=INAL SKL TRAINNG

PURSE C' POGRAM

This seven modle program is designed to give students personalized

training in those strategies and skills that can help than learn to

take control of their lives. The program teaches students how to

define what is important to them, to identify career interests and

goats, to set short and long term goals, to manage stress, to caununicate effectively with others, and to solve problems. In short, the

program helps students to motivate themselves by being good selfmanagers.

P~JGRAM FaOR

The main content of the motivational skill training program is presented in self-contained, easy to read printed modules. Each module

has been formatted to include a statement of the rationale or purpose

of each module, a statement of cbjectives, embedded questions and

practice exercises, an opportunity for students to chose a level of

effort for learning each module, key words and an accampanying glossary,

and personally relevant case histories and examples. A posttest follos

each module and allows students the opportunity to test themselves over

key concepts and skills.

The program and each module are designed to be introduced by an instructor or by a caputer-assisted instruction (CAI) program. Following this

introduction, students read through each module on their an. When they

finish each module, students then take the module posttest and partici- - pate in either an instructor-led mall group discussion/practice session

or they receive a CAI practice/simulated group experience.

BEE S CF PROGRAM 4

The motivational skill training program has been shown to improve students'

test scores and to reduce test failure rates. The Army is interested in - further exploring the benefits of this type of skill training as well as

in exploring the enhancements possible through the use of CAl.

PURPOSE CF EACH MO.DU'

=rOI= ICN: Eplains the concepts of personal responsibility and

positive self-control; introduces students to the basic strategies

for taking ccntrol of their lives as well as the purpose and organization of the whole program.

SELF K0RULEIGE: Helps students explore their values and beliefs, define

what is important to then, and resolve values conflicts.

CAREER EXPORATION: Helps students explore their career interests and

goals and to relate these to their military specialty.

76 . o.7 
GOAL SU! C: Helps students learn a systematic approach to setting

goals and- pursuing them; helps them learn to make decisians.

STRESS MRO : Helps students understand what stress is and teaches

them a variety of do, think, and say strategies for managing stress V6

in their lives.

--I'-VE CMMTICATICN: Helps students define their typical c-ruzuCaticn style and helps them learn how to use effective "I" messages

and active listening skills.

PROBLEM SOLVING: Helps students learn to apply a systematic approach

to identifying and solving probleas; provides a sumrnary to the

Program-.L

77.

i" -- :

".'.
•C

STUDENT HANDOUTS - "

Module 1 - Introduction

No Handouts

Module 2 - Self Knowledge

Contract Form (for CAI Practice)

Module 3 -Career Exploration

Handout #1 - Decision Making Process (for CAI Practice)

Handout #2 - Evaluating Risks and Costs (for CAI Practice)

Module 4 - Goal Setting

Game Instructions (for CAI Introduction)

Module 5 - Stress Management

Handout #1 - Mistaken Beliefs (for CAI Practice)

Handout #2 - Worksheet (for CAI Practice)

Module 6 - Effective Communication

Worksheet (for CAI Practice) .4:::"

Module 7 -Problem Solving

Handout #1 - Defining the Problem (for CAI Practice)

Handout #2 - Generating and Evaluating Alternatives (for CAI

Practice)

78

...-:..

4,'.

:-a..~

1:::::

78" -A
I~.. ~*

3,.

.3'.

3.-

1

Appendix C: .' A.

Student Assignment Sheet I;I

I

,

'3- -

79

' *'~' 3. * 3,. *3*****...................... 3.-. 3.. * ...- *\
.Condi ;ion: Class #:

B-CAI Intro & Prac STUDENT ASSIGNMENT SHEET

C-CAI Intro/Instr Prac

D-lnstr Intro/CAI Prac

E-Instr Intro & Prac

EXPERIMENTAL CONTROL

NAME SSN RANK SEX NAME SSN RANK SEX

Reular Army .

1. 2.

2. 3.

3. 4.

4. 5.

5. 6.

6. 7.

7. 8.

E. 9.

9. 10.

National Guard 11.

1. 12. "-C2. 13.

3. 14.

4. 15.

Reserve 16.

1. 17. ..:...1

2. 18.

19.

20.

21.

22.

23.

24.

,3 ASSIGNMENT RULES: Please assign 2 females to the experimental group if possible. Try

to achieve the mix of regular Army, National Guard, and Reserve shown above. If this

isn't possible, try to balance between experimental and control groups. Indicate the

component for students assigned to the control condition. For class sizes smaller

than 35 students, use the following rules:

Total Class Size # Assioned to Experimental # Assigned to Control

IF: 21 or more 15 students Remainder of Class

10-20 10 students Remainder, if any

9 or less All None

80 . - - - , .-
Appendix D:

Descriptive Statistics

for Noncategorical Variables

by Experimental Group

Vz
The following information is offerred to aid in interpretation of the data.

For the following variables, hgh scores are desirable. ASVAB General, ASVAB

Electrical, ASVAB Clerical, Critical Thinking Pre- and Posttests, Self-Efficacy

Pre- and Posttest, and Instructor Rating. "

For the folowing set of variables, low scores reflect desirable attributes

or results: Pre- and Posttest measures of adaptation, frustration, overload, v.- d' .. '

deprivation, nutrition, self-perception, type A behavior, and anxiety; test failures;

time to complete tasks; and Progress Index.

82
. r -p

Appendix D

Descriptive Statistics for Noncategorical %%%

Variables by Group

Correlation with

Variable Group N _ SD Progress Index

ASVAB General CC 144 105.9 11.76

CAI 36 105.0 9.52

CAll 21 107.5 13.01

ICAI 34 10.6 13.61

II 31 107.9 9.51

Total 266 106.5 11.58 .204**

ASVAB Electrical CC 144 108.0 9.36

CAI 36 106.6 10.35

CAB! 21 109.2 11.29

ICAI 34 110.9 12.59

II 31 110.5 9.12

Total 266 108.6 10.10 .321.**

ASVAB Clerical CC 105 102.2 12.38

CAI 21 101.8 9.55

CAlI 13 103.6 17.80

ICAI 9 104.3 14.51 11 11 101.6 I11.13__-

Total 159 102.4 12.47 .197**

Critical Thinking CC 159 2.75 1.08

Pretest CAI 36 3.14 1.29

CAl 44 2.86 1.17

ICAI 40 2.85 1.31

II 43 3.00 1.13

Total 322 2.86 1.15 .108

(continued)

83
Descriptive Statistics for Noncategorical

Variables by Group (continued)

Correlation with S..

Variable Group A N S_D Progress Index

Self-Efficacy Pretest CC 172 93.4 11.69

CAI 36 91.1 13.56

CAll 45 94.7 11.69

ICAI 39 94.4 12.94

If 46 95.5 12.41

Total 338 93.7 12.13 .107

Adaptation Pretest CC 173 214.7 107.7

CAI 36 220.2 129.0

:2 CAll 46 215.2 101.4

ICAI 40 217.7 128.0 __ 46 179.1 96.6

Total 341 210.9 110.5 .003

Frustration Pretest CC 172 24.59 4.14

CAI 36 24.17 3.84

CAll 45 24.76 3.69

ICAI 40 24.15 4.15

If 46 22.91 4.54

Total 339 24.29 4.13 .034

Overload Pretest CC 173 19.45 3.65

CAI 36 20.83 4.03

CAII 45 20.44 3.91

ICAI 40 19.68 4.19

11 46 18.98 3.92

Total 340 19.69 3.85 .156*

I*

"r

(continued)

84

* ~*~* ~ -. ** -~-~- -* *p.. --
Descriptive Statistics for Noncategorical

Variables by Group (continued)

Correlation with

Variable Group .N X D Progress Index

Anxiety Pretest CC 168 22.24 5.92

CAI 35 23.26 5.71

CAll 44 21.45 5.27

ICAI 39 22.82 5.57

II 46 21.11 6.61

Total 332 22.16 5.8 .055

Critical Thinking Posttest CC 169 3.04 1.13

CAI 46 2.97 1.11

CAI[ 45 3.22 1.02

ICAI 39 3.46 1.14

11 46 3.20 1.05

Total 345 3.12 1.10. 0.55

Self-Efficacy Posttest CC 175 93.8 11.47

CAI 46 92.5 13.56

CAll 46 94.0 11.75

ICAI 40 93.3 12.53

II 46 94.9 12.23

Total 353 93.8 11.96 .065

Adaptation Posttest CC 174 201.3 109.9

CAI 46 245.5 117.0

CA1l 46 222.7 103.8

ICAI 40 236.1 140.4

II 46 223.1 178.9

Total 352 216.7 117.0 .061

(continued)

85

' ' . ' 1 . .-
. 6. 

Descriptive Statistics for Noncategorical

Variables by Group (continued>-. " -.

Correlation with

Variable Group N _ SD Progress Index

CAI 36 23.19 5.13

CAll 45 23.87 5.55

ICAI 40 24.43 4.18

II 46 22.85 4.54

Total 339 23.78 4.69 .004 _.-"

Nutrition Pretest CC 172 15.19 2.28

CAI 36 16.33 3.12

CARI 44 15.07 2.62

ICAI 40 14.95 1.83

1_ 46 15.41 2.08

Total 338 15.30 2.37 .006

Self -Perception Pretest CC 172 18.60 4.02

CAI 36 19.31 5.21

CAn 45 17.64 4.06

ICAI 39 18.56 4.01

II 46 17.26 4.88

Total 338 18.36 4.30 .143*

Type A Behavior Pretest CC 171 23.76 4.60

CAI 36 24.17 4.55

CA!! 45 23.09 4.88

ICAI 40 23.15 4.73

11 46 23.26 4.05

Total 338 23.57 4.56 .064

i ,

(continued)

86

:::::

.9. .:.9.9 .
Descriptive Statistics for Noncategorical

Variables by Group (continued)

Correlation with

Variable Group N X_ SD Progress Index .,

Frustration Posttest CC 175 23.98 4.82

CAI 46 25.28 4.18

CAll 46 24.91 4.10

ICAI 40 23.93 4.46

1I 46 24.63 5.17

Total 353 24.35 4.66 .019

Overload Posttest CC 175 18.79 3.56

CAI 46 20.50 4.34

CAll 46 19.52 4.46

ICAI 40 19.50 3.87

1_ 46 18.74 4.57

Total 353 19.18 3.99 .068

Deprivation Posttest CC 174 23.48 4.62

CAI 46 23.35 4.54

CAD 45 23.09 4.63

ICAI 40 23.58 4.43

I 46 22.24 5.08

Total 351 23.26 4.64 .039

Nutrition Posttest CC 174 15.03 2.41

CAI 46 14.96 2.16

CA! 46 14.96 2.67

ICAI 40 14.75 1.79

1_ 46 15.07 2.26

Total 352 14.98 2.32 .116 -"

(continued)

8'.7k

: ... # , .... " . "r . . ... . ",,. .• . ..... . . .. •. ., ., , .. - -, .. ,., . ,. , % ,..-.-.-,-.. ,- ,- % ,. ,, .. ,.............S
Descriptive Statistics for Noncategorical

Variables by Group (continued)

Correlation with

Variable Group SN SD Progress Index

Failures on Tasks 1-16 CC 175 1.22 1.23

CAI 46 1.43 1.24

CAU 46 .93 1.25

ICAI 40 1.18 1.30

n_ 46 .98 1.22

Total 353 1.18 1.24 .311**

Time to Complete CC 175 111.67 22.18

Tasks 1-16 CAI 47 110.06 20.59

CAll 46 106.96 24.56

ICAI 40 110.40 27.31

II 46 103.54 22.39

Total 354 109.65 23.00 .706**

Failures on Tasks 17-26 CC 175 1.65 1.43

CAI 47 1.81 1.78

ICAI 40 2.05 2.04

U 46 1.41 1.18 .

Total 354 1.64 1.51 .325"*

Time to Complete CC 175 213.92 38.56

Tasks 17-26 CAI 47 222.72 37.49

CAIR 46 211.46 37.39

ICAI 40 214.93 40.50

II 46 201.35 37.44

Total 354 213.23 38.5-4 .870* -

(continued)

88

V. **-W * *, S .. - .. . . :. 
Descriptive Statistics for Noncategorlcal

Variables by Group (continued)

Correlation with

Variable Group N _ SD Progress Index

Total Test Failures CC 175 2.37 1.98

CAI 46 3.17 2.31

CAII 46 2.26 2.08

ICAI 40 3.23 2.63

_ 46 2.39 1.___

Total 333 2.31 2.10 .318**

Progress Index CC 173 .913 .122

CAI #7 933 .116

CAll #6 .902 .120

ICA! 39 .915 .124 - 4..

I . 46 .$68 .129

Total 353 .909 .123 1.00

8. -..-. $%.'+

.12 " .

89