Journal of Organizational Behavior Management, 31:55–94, 2011
Copyright © Taylor & Francis Group, LLC
ISSN: 0160-8061 print/1540-8604 online
DOI: 10.1080/01608061.2010.541821

DISCUSSION ARTICLE

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive Computer-Based
Instruction: A Review of Studies Published
Between 1995 and 2007
DOUGLAS A. JOHNSON and SOPHIE RUBIN
Operant-Tech Consulting, and Western Michigan University, Kalamazoo, Michigan, USA

Computer-based instruction (CBI) has been growing rapidly asa training tool in organizational settings, but close attention
to behavioral factors has often been neglected. CBI represents
a promising instructional advancement over current training
methods. This review article summarizes 12 years of comparative research in interactive computer-based instruction relevant toemployee training techniques. The results demonstrate that CBI isan effective and viable training technique, and several areas inneed of further examination are detailed.
KEYWORDS computer-based instruction, interactive, training

Training has always played an important role for business and industry.
This importance is demonstrated by employer expenditures, with investments that reach $55 billion in formal training programs and $180 billion in
informal on-the-job training yearly (Schultz & Schultz, 2006). As job procedures change due to technological innovations, the need for effective
training is becoming increasingly important. Business owners and managers
are in need of cost-efficient and high-quality training methods to keep up
with the demand for skilled labor.

The authors would like to thank Dr. Alyce Dickinson, Dr. Mark Dixon, and theanonymous reviewers for their helpful suggestions while reviewing this manuscript.
Address correspondence to Douglas A. Johnson, Operant-Tech Consulting, P.O. Box
20415, Kalamazoo, MI 49019, USA. E-mail: djohnson@operant-tech.com
55

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

56

D. A. Johnson and S. Rubin

Organizational behavior management (OBM) does not appear to be
keeping up with this demand. A glimpse at some of the most popular OBM
books indicates that chapters on training / instruction or any other coverageof the topic are absent (Braksick, 2000; Daniels & Daniels, 2004). An informal review of articles published in the Journal of Organizational Behavior
Management during a five-year period reveals a similar trend. Of the 65 articles (excluding book reviews) published between 2003 and 2007, only four
(Hickman & Geller, 2003; Pampino, Wilder, & Binder, 2005; Rohlman et al.,
2004; Sasson & Austin, 2004) appear to cover training in any depth beyond
task clarification. Furthermore, only two (Pampino et al.; Rohlman et al.)appear to involve training solutions more in-depth than “training as usual”(i.e., simply giving trainees a manual or lecture). Thus, out of 65 articles,
only about 3% cover sophisticated behavioral training methodologies.
This area represents an important opportunity for OBM practitioners todeliver a product that is highly requested by the business community, which
can then be possibly followed by practitioners selling more behavioral solutions for other business problems and needs (that may be less requested but
just as needed). Considering the importance of training methods to managers, the topic appears underrepresented and therefore more research iscalled for. Indeed, the topic rarely receives any attention beyond a very
superficial coverage relating to task clarification. Perhaps this is because
instruction and training are often viewed as just antecedents to behavior,
and OBM aims to utilize the more powerful variable of consequences toinfluence behavior. From a molecular point of view, instruction and training
do not need to be just antecedent events but can also include behaviors andconsequences, as exemplified by Direct Instruction, Personalized System ofInstruction (PSI), and Precision Teaching (Fox, 2004; Maloney, 1998; Slocum,2004). However, most who argue that training is just an antecedent areprobably taking a molar view. That is, they are taking a more broad viewand see training overall (including all the relevant training antecedents,
behaviors, and consequences) as an antecedent to on-the-job performance,
which should be followed by workplace consequences. While it is also truethat training is likely to be ineffective and have no long-term impact without being followed up with on-the-job consequences, that is no reason toequate training as being unimportant. Regardless of whether you are taking
a molecular view or molar view, training remains a very important part of theworkplace process, and behavior analysts would do well not to be dismissiveof it. Given that behavior analysts often consider themselves experts on thelearning process, it is surprising that there is a paucity of research on training
and instruction within OBM. While there are exceptions such as Accelerated
Learning and Performance-Based Instruction (Brethower & Smalley, 1998;
Bruce, 2004; Daniels, 2000), the topic still remains underrepresented.
Despite the lack of research on training and instruction, the applied
world has not forgotten about the topic. For leaders of business and industry,

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI57

an increasingly common solution to meet the needs of training is computerbased instruction (CBI). Oblinger and Rush (2003) state that the corporate
e-learning market was expected to exceed $23 billion by 2004, demonstrating a 68.8% annual growth since 1999. Part of the popularity of CBI may be
due to its flexibility. There is no need to wait until the number of trainees
grows large enough to fill a classroom or wait until a suitable trainer canbe located. CBI can easily accommodate geographically diverse locations
and appears to reduce the amount of time dedicated to training (Schultz &Schultz, 2006). Despite its popularity, caution should be urged, as popularity does not equate with effectiveness. When designing CBI, a number ofvariables need to be considered.
First of all, instruction should enforce active engagement, a criticalvariable echoed by many others (Markle, 1990; Molenda & Russell, 2006;
Skinner, 1954, 1958). Instruction should be learner paced, proceeding only
when the learner is ready, a feature also implemented by other effective
behavioral teaching approaches such as PSI (Fox, 2004). Further, effective
instruction should include mastery criteria to ensure a learner demonstrates
mastery of material prior to being allowed to progress to subsequent material, much like Direct Instruction, PSI, or an effective tutor (Slocum, 2004).
Effective instruction should also include immediate feedback to strengthen
desired responses and correct undesired responses.
For the purposes of the article, training and education will be treated assimilar activities, subsumed under the more generic label of instruction. The
critical attribute of instruction (as well as education and training) is that it
is aimed at producing learning (Molenda & Russell, 2006). Computer-based
instruction can solve many of the problems of traditional instructional methods. For example, lectures and workshops are typical instructional methods,
but are problematic because they are instructor paced, and group instruction
tends to be geared toward the average learner. Therefore, advanced learners
are left bored and anxious to move on, while slower learners are confusedand left behind. Some instructional approaches such as Direct Instruction
attempt to overcome this problem by using placement tests to form homogeneous groups in an attempt to ensure that everyone in the group is anaverage learner (Slocum, 2004). The feasibility of adapting this for the business world of adult learners is questionable. Learners are more likely tohave very diverse repertoires, making the formation of homogeneous groups
much more difficult. Furthermore, multiple groups suggest multiple trainers,
which may not be practical. Computer-based instruction can avoid this difficulty of group instruction by individualizing the instructional pace for eachtrainee. As long as computers are available, large-scale implementation withlarge numbers of trainees is possible, yet training can still remain learner
paced.
Textbooks and training manuals are also typical instructional methods. Such textual materials have the advantage of being learner paced inDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

58

D. A. Johnson and S. Rubinthat trainees don’t need to turn a page until they are ready. However,they have a problem in that they can’t enforce active responding. Textual
materials can include study objectives, review questions, frequent response
requests, and other similar attempts at interaction, but trainees can easily bypass such requests with no immediate consequence. No matter how
carefully hidden an answer is, there is always the probability that all but
the most self-disciplined learners will peek at answers prior to making a
response (Markle, 1990). Furthermore, even if a response is made, textual
material offers no programmed differential consequences. CBI programs
can be designed so that progression in the instructional material is prevented until learners make an appropriate response. Thus, CBI can enforce
interactions.
One-on-one trainers come closest to being the most viable effective training solution. Unlike group instruction, the pace can be tailored to the individual student. Unlike books, active responding canbe enforced. Unfortunately, one-on-one trainers can be cost prohibitiveand impractical. Computer-based instruction is much like a one-on-one
trainer, except many of the associated costs are avoided. One of thebiggest advantages of CBI is the possibility of enforced and economicalinteractions.
Many current CBI programs claim to be interactive, but a closer look atthem can reveal problems with their definition of interactive. Interactive
sometimes only means that the learner has control over the pace andsequencing of materials. This level of interaction is no more advanced than atextbook, given that textbooks also allow learners to turn pages when readyand to skip around between chapters. In general, CBI programs have failed
to revolutionize instruction because most have been designed to replicate
traditional instruction, thus they often produce the same results (Engelmann,
1992; Skinner, 1963). Instruction should be designed to promote responses
that are more meaningful than those required by a simple digital textbook(Markle, 1990). Interactions should be demonstrative, requiring the learnerto show they understand a given point before proceeding to new material.
The possibility of allowing economical, enforced, and demonstrative interactions is the one unique offering of CBI, one that distinguishes it fromother instructional alternatives. For brevity’s sake, CBI that is economical,
enforced, and demonstrative in its interactions will simply be referred to as“interactive CBI” for the remainder of the article. The main purpose of thisreview is to examine if interactive CBI truly is more effective than other training formats and if so, what are the most effective forms of interactive CBI.
The best practices described by this article should aid managers, consultants,
and training developers to select and create the most effective training solutions. Furthermore, by discovering what is currently known about interactive
CBI, it is hoped the stage will be set for investigating what we don’t know
in future studies.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI59

Although much of CBI research focuses on inferred behaviors/
processes and variables outside the control of a programmer or instructional designer (e.g., schema formation, self-directedness), this article will
review only interactive CBI research investigating variables that could becontrolled by an instructional designer (e.g., feedback inclusion, response
requirements). Part of the reason is to operationally define concepts thatwill allow for replication, but the main concern relates to the pragmatics ofimproving instruction. For example, discovering that CBI is more effectivewith self-directed learners tells instructional designers, computer programmers, and training developers little about how they should alter CBI. Instead,such individuals are simply left to hope that the recipients of their products
just happen to be self-directed learners.
Much of CBI research is also problematic in terms of the measures
employed. Social validation is often used as the sole measure of success.
While learner preference is certainly an important consideration, it should
not be the only measure used to evaluate success. Again, popularity does
not necessarily equate with effectiveness. Given that organizations are concerned with how training will improve organizational performance, thisarticle will only survey those CBI studies that included an objective measureof performance.
Many of the research designs used are problematic as well. Typicaldesigns include the one-group pretest-posttest design and the untreatedcontrol group design (Shadish, Cook, & Campbell, 2002). The one-grouppretest-posttest design involves assessing learner knowledge with a pretest,
administering a CBI program, and then assessing learner knowledge again
with a posttest. Effectiveness of the CBI program is evaluated by comparing
the pretest scores (no instruction condition) with the posttest scores (having
received instruction condition). The untreated control group design involves
two groups. The first group receives CBI and the second group receives no
instruction. Both groups are then given a test to assess their knowledge.
Similar to the previous design, effectiveness of the CBI program is evaluated by comparing the control group scores (no instruction condition) with
the CBI group scores (having received instruction condition). Ultimately,
the outcomes of these designs only tell us if CBI is better than nothing at
all, a less than awe-inspiring standard. These designs tell us little about therelative effectiveness of CBI compared to other instructional approaches,
which does not help those in charge of training choose the most effective
solutions. Thus, this review will only examine comparative research involving interactive CBI. The comparative evaluations will be between interactive
CBI and another type of instruction such as an alternative interactive CBI,noninteractive CBI, or non-CBI instruction.
To increase the likelihood of this study’s outcomes being relevant to
the managers, supervisors, and training developers, only studies involving
employees or employee analogs (normal, adult learners) will be considered.

60

D. A. Johnson and S. Rubin

Part of the reason for excluding CBI studies with children is that differencesin verbal sophistication might imply differential effects of experimental variables. For similar reasons, CBI studies where the subject matter involves aforeign language will also be excluded.

METHOD

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Search Procedures
The Psychological Information (PsycINFO) database was used to identify
peer-reviewed journal articles on interactive computer-based training/teaching between the years of 1995 and 2007. The following subject phrases
were used to identify articles: computer assisted instruction, computer training, teaching machines, and computer assisted diagnosis. The following
keywords were also used to identify additional articles: Web-based training, Web based training, Web-based instruction, Web based instruction,
computer-based training, computer based training, computer-based instruction, computer based instruction, computer-aided training, computer aided
training, computer-aided instruction, computer aided instruction, computerassisted training, computer assisted training, computer-assisted instruction,
computer assisted instruction, computer training, multimedia instruction,
multimedia training, Internet instruction, Internet training, and computer
instruction. These searches yielded a total of 3,970 articles.

Initial Inclusion
The abstracts of the 3,970 articles were read in an attempt to exclude
nonrelevant articles. The first and second authors independently reviewed
100% of the abstracts. Articles were excluded on the following grounds:
participants were selected from “abnormal” populations, such as DSM-IV
diagnosed clients; participants were nonadults (under 18 years of age);participants were developmentally disabled; instructional content involved
language skills acquisition, such as computer assisted language learning(CALL); articles that were reviews of books or book chapters; or articles
that were theoretical in nature with no empirical data. Articles were also
excluded if they only studied variables beyond the control of an instructional designer, such as self-directedness of learners, locus of control, etc.
Articles were included if they involved comparative investigations of interactive computer-based training/teaching with normal, adult learners. The
empirical comparisons could be between interactive CBI and another formof instruction, including alternative interactive CBI methods, noninteractive
CBI, or instruction that wasn’t computer-based. A CBI program would beconsidered interactive if it required the learner to demonstrate understanding of the material during the learning process (i.e., answer questions whileDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI61

learning the material prior to final performance outcome measures). A CBIprogram would be considered noninteractive if learners were not required
to demonstrate understanding of the material during the learning process(e.g., only viewed a simulation, only clicked a next button, etc). To be
included, articles had to include objective measures of performance outcomes. Finally, the articles had to investigate variables that could potentiallybe controlled by an instructional designer, such as the pace of the program,type of feedback, etc. Interobserver agreement (IOA) was calculated for theabstracts reviewed by both authors (agreements divided by agreements plus
disagreements). There was a 90.1% agreement between authors for initial
inclusion. Articles were only excluded if both the first and second authors
independently agreed they should be excluded, so that all disagreements
resulted in the article being included for a more in-depth analysis than areading of the abstract (such inclusion of disagreements did not mean anarticle would be included for the final coding and analysis at this stage). At
this stage of inclusion there was a bias toward including articles, and thiswas done intentionally because it is often very difficult to evaluate an article
on the basis of the abstract alone. As a result, 3,059 articles were excluded
and 911 articles were retained.

Final Inclusion
The remaining 911 articles were read in detail to determine if they met theinclusion criteria. The first author independently evaluated 100% of the articles to determine their appropriateness for inclusion. The second author
independently evaluated a random selection of the articles (n = 410) to
obtain IOA. There were no disagreements at this stage of inclusion, resulting in IOA reaching 100%. The final count of articles to be included inthis review was 71 articles. Some of these articles involved multiple experiments, which were separated out to facilitate analysis. To clarify which
specific comparison is being discussed, c1, c2, or c3 (comparison 1, comparison 2, or comparison 3, respectively) will be added after the year citation(e.g., Atkinson, 2002c1, 2002c2; Bodemer, Kazmerski, & Torgerson, 2004c1,
2004c2; Howard-Jones & Martin, 2002c1, 2002c2; Miller & Malott, 1997c1,
1997c2; Moreno & Valdez, 2005c1, 2005c2, 2005c3; Rohlman et al., 2004c1,
2004c2, 2004c3). Thus, a total of 79 experimental comparisons were selected
for coding and analysis.

Coding Categories and Definitions
GOAL
Interactive CBI for each article was classified as (a) education, (b) training,or (c) life skills. Articles were considered as for “education” if CBI was

62

D. A. Johnson and S. Rubin

done as part of an educational curriculum within a higher education setting.
The classification of “training” was used if CBI was designed to facilitate
performance in currently held jobs. The classification of “life skills” wasused for instructional purposes that didn’t fit either the education or trainingclassifications (e.g., parenting skills, exercise skills, etc).
USAGE

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Interactive CBI was classified as either (a) primary or (b) supplemental,
depending on whether the CBI was designed to stand alone as the main
form of instruction or be used in conjunction with another instructional
format.
PACE
The interactive CBI was classified as either (a) machine paced or (b) student
paced. CBI was considered “machine” paced if the learner had no control
over the speed at which material was presented (with the exception of interactions). For example, if a learner watched an automated sequence of slides
in which the program only paused for the learner to make a response, thenthis was considered machine paced. CBI was considered “student” paced
if the instructional material progressed only when a learner allowed (e.g.,learner clicks a “next” button).
MASTERY CRITERIAArticles were also classified according to the presence or absence of mastery criteria. An article was classified as utilizing mastery criteria only if thearticle detailed a system in which learners could not progress to later partsof instructional material prior to demonstrating understanding on previous
material by meeting some predetermined criteria. Further, the program had
to be designed in some fashion to prevent learners from successfully passing an interaction on their first attempt by blindly selecting answer choices.
Otherwise, an article was designated as not including mastery criteria.
RESPONSE INTERACTION
Articles were classified as (a) compose, (b) select, (c) mixed, (d), various,or (e) unknown based on the type of response learners were required tomake by the interactive CBI program. “Compose” was defined as an interaction type in which a large variety of potential responses could be given,such as typing a response or using a drag-and-drop interaction. “Select”
was defined as an interaction type in which learners choose answers froma list of predetermined answer choices (i.e., multiple choice). The “mixed”

Effectiveness of Interactive CBI63

classification was used for CBI programs that utilized compose and selectformats within the same instructional program. The “various” classification
was used when an article used multiple programs, some of which used compose and some of which used select (e.g., comparison study of composing
versus selecting an answer). Articles were marked as “unknown” when there
were insufficient details to classify the response interaction.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

FEEDBACK TYPE
Articles were also classified based on the type of feedback used in theinteractive CBI: (a) contingent-specific, (b) contingent-nonspecific, (c) noncontingent, (d) none, (e) various, or (f) unknown. “Contingent-specific” wasapplied to programs in which feedback specifies why learner’s responses
were correct or incorrect. “Contingent-nonspecific” was used when the CBIprogram simply stated whether or not the learner’s response was right orwrong, with no explanation. The “noncontingent” classification was used
for CBI programs that simply stated the correct answer, with no evaluationof the learner’s response. “None” was used when feedback was not presented for learner’s individual responses during the CBI. “Various” was used
when an article used multiple programs that differed in the types of feedback utilized. “Unknown” was applied when there were insufficient details
to classify the type of feedback used.
SUPPLEMENTAL INCENTIVES
Articles were classified according to the type of external incentive used:(a) performance dependent, (b) performance independent, (c) none, or (d)various. For “performance dependent,” rewards or punishers were provided
by a source external of the CBI, but the quantity of which depended on
how well a learner performed during the interactive CBI program (e.g.,losing 5 points for every incorrect answer, receiving $20 for achieving 90%
correct or better). “Performance independent” was applied when rewards orpunishers were provided by a source external of the CBI and the quantity
did not depend on how well the learner performed (e.g., receiving extra
credit for participation, receiving $50 for simply completing CBI). When
there were no external incentives specified, the label of “none” was applied.
“Various” was reserved for studies in which some participants received one
type of external incentives and other participants received a different typeof external incentives (or lack thereof).
OUTCOMES
Of the experimental comparisons being made, the instructional approachthat proved superior was also assessed. Statistical comparisons of differences

64

D. A. Johnson and S. Rubin

between instructional approaches were judged as significant at an alphalevel of 0.05 or less.

Coding IOARESULTS
Comparisons Between Interactive CBI and Another Instructional
FormatThe Appendix summarizes the results of the coding classifications. Overall
the results favored the use of interactive CBI alone or as a supplement.
In the 42 comparisons between instructional formats, 27 demonstrated
improvements through the use of interactive CBI (64.3%). Thirteen comparisons found no difference or mixed results (31.0%). Only two comparisonsfound another instructional format superior to interactive CBI (4.8%). Thus,
interactive CBI was found to be at least as good as, if not better, than instructional alternatives 95.2% of the time. See Figure 1 for a summary of thesecomparisons.
30
Number of Comparisons

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Interobserver agreement was calculated based on the classifications of two
independent coders. Overall IOA was calculated for the categories of goal,
usage, pace, mastery criteria, response interaction, feedback type, supplemental incentives, and outcomes. IOA was calculated using the number ofagreements divided by the number of agreements plus disagreements. The
resulting IOA score was 95.6%. The disagreements were then analyzed anddiscussed between coders until an agreement was reached.

25
20
15
10
5
0
Favoring interactive
CBI

No differences / mixed Favoring instructional
resultsformat besides
interactive CBIFIGURE 1 Summary of comparisons between interactive CBI and other formats.

Effectiveness of Interactive CBI65

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Interactive CBI Compared With Noninteractive CBIThere were twenty comparisons that examined interactive CBI and noninteractive CBI (Bodemer, Ploetzner, Feuerlein, & Spada, 2004c1; Bodemer
et al., 2004c2; Cook, Gelula, Dupras, & Schwartz, 2007; Darabi, Nelson, &Palanki, 2007; Davis, Bostow, & Heimisson, 2007; Dornisch & Sperling, 2006;
Eckerman et al., 2002; Evans & Gibbons, 2007; Howard-Jones & Martin,
2002c1; Howard-Jones & Martin, 2002c2; Katayama & Crooks, 2003; Kerwin,
2006; Kritch & Bostow, 1998; Martin, Klein, & Sullivan, 2007; Miller & Malott,
1997c1; Miller & Malott, 1997c2; Moreno & Valdez, 2005c1; Rohlman et al.,2004c2; Tudor, 1995; Van Rooij, 2007). Fourteen of the twenty comparisonsfavored interactive CBI over noninteractive CBI. Two comparisons (HowardJones & Martin, 2002c1; Kerwin, 2006) favored noninteractive CBI overinteractive CBI. One comparison (Moreno & Valdez, 2005c1) found mixed
results. Three comparisons (Bodemer et al., 2004c1; Dornisch & Sperling,
2006; Rohlman et al., 2004c2) found no differences.

Interactive CBI Compared With Textual Presentation
There were six comparisons that examined interactive CBI and textual
instruction such as textbooks, manuals, and workbooks (Aberson, Berger,
Healy, & Romero, 2003; González & Birch, 2000; Luyben, Hipworth, &Pappas, 2003; Morris, 2001; Pelayo-Alvarez, Albert-Ros, Gil-Latorre, &Gutierrez-Sigler, 2000; Rohlman et al., 2004c1). Three comparisons foundinteractive CBI to be superior. Two studies found no differences between
training conditions (González & Birch; Morris). One study found mixed
results (Rohlman et al., 2004c1). In that study, interactive CBI was foundto be superior to a manual with text-based quizzes and a manual without
text-based quizzes. However, no differences were found between interactive
CBI and a manual with both text-based quizzes and answers.

Textual/Interactive CBI Hybrid Compared With Textual
Instruction Alone
There was one study that examined textual instruction with interactive CBIcompared with textual instruction alone (Sholomskas & Carroll, 2006), whichfound superior performance with the CBI supplement.

Interactive CBI Compared With Video Presentation
Only one study compared interactive CBI to a video presentation (Segal,
Chen, Gordon, Kacir, & Gylys, 2003). The study found no differences
between teaching methods.

66

D. A. Johnson and S. Rubin

Interactive CBI Compared With Group Oral Presentation
There were 10 comparisons that examined interactive CBI and lecture style
instruction (Blasko, Kazmerski, & Torgerson, 2004; Connolly, MacArthur,Stansfield, & McLellan, 2007; Fisher, Deshler, & Schumaker, 1999; Fitzgerald,
1995; Jenny & Fai, 2001; R. H. Maki & W. S. Maki, 2003; R. H. Maki,
W. S. Maki, Patterson, & Whittaker, 2000; Orey, Zhao, Fan, & Keenan, 1998;
Parchman, Ellis, Christinaz, & Vogel, 2000; Shute, Gawlick-Grendell, Young,& Burnham, 1996). Eight of the ten comparisons demonstrated superior
results for interactive CBI. Fisher et al. and Shute et al. were the exceptions,finding no differences.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Lecture/Interactive CBI Hybrid Compared With Lecture Alone
Four studies involved using interactive CBI to supplement lecture andcompared such a hybrid with lecture alone (Batchelder & Rachal, 2000;
Croninger, Tumiel, & Sowa, 1995; Flora & Logan, 1996; Hahne, Benndorf,Frey, & Herzig, 2005). Two of the four comparisons found that adding
an interactive CBI component improved the lecture format. Batchelder &Rachal and Hahne et al. found no differences between the supplemented
and nonsupplemented lecture formats.

Comparisons of Different Elements Within Interactive CBIPACINGOf the 79 experimental comparisons included in this review, 75 utilized
learner pacing (94.9%). The remaining 4 utilized computer controlled pacing (5.1%). None of the studies directly compared the benefits or costs ofimplementing learner versus computer controlled pacing.
SEQUENCES
There were three comparisons of scripted instructional sequences versus
learner controlled sequences (Bell & Kozlowski, 2002; Bodemer et al.,2004c2; Henry, 1995). Two of the comparisons found favorable results whenusing scripted instructional sequences and one comparison (Henry) foundno differences. A study by Green, Eppler, Ironsmith, & Wuensch (2007) compared linear and branching formats. In linear formats, all users are exposedto the same instructional sequences. In branching formats, users who makeerrors are exposed to additional supplemental material designed to remediate errors. The study found performance superior under the branching
format. J. Reisslein, Atkinson, Seeling, & M. Reisslein (2006) comparedsequences in which users are first exposed to worked out examples followedEffectiveness of Interactive CBI67

by problems, sequences in which users are first exposed to problems followed by worked out examples, and sequences utilizing the fading of
hints. No differences were found between instructional sequences. Lusk
and Atkinson (2007) examined whether performance would be best when
instructional sequences displayed only one step at a time with sequences
displaying all steps simultaneously. Performance was superior when only
one step was displayed at a time.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

PRACTICE
Three studies examined the effects of additional practice items(Schnackenberg & Sullivan, 2000; Shute & Gawlick, 1995; Shute, Gawlick, &Gluck, 1998). While the amounts of practice differed, two of the three studies found that additional practice improved performance. The third study(Shute & Gawlick) found no effects by adding additional practice items. One
study looked at the effects of the density of interactions (Whittam, Dwyer,& Leeming, 2004). This differed from amount of practice because the total
number of questions remained consistent across conditions (3 questions after
every 5 instructional frames, 6 questions after every 10 instructional frames,or 12 questions after every 25 instructional frames). Density of interactions
was irrelevant to performance outcomes. Fox and Ghezzi (2003) examinedthe effects of fluency training with normal practice. No performance differences were found. However, Brothen and Wambach (2004) compared timed
quizzes with untimed quizzes and found that imposing a time limit improved
performance.

MODALITY OF NARRATIVE/PROMPTS
Seven comparisons examined the types of modality by which narration andprompts are delivered (Atkinson, 2002c1, 2002c2; Atkinson, Mayer, & Merrill,
2005; Kalyuga, Chandler, & Sweller, 1999, 2000; Lusk & Atkinson, 2007; S. M.
Truman & P. J. Truman, 2006). Four comparisons evaluated narration andprompts delivered via auditory or visual means (Atkinson, 2002c1, 2002c2;
Kalyuga et al., 1999; S. M. Truman & P. J. Truman, 2006). All four favored
audio narration and prompts. Kalyuga et al. (1999) also included a dual
audio/visual condition, but audio only was still superior. Atkinson et al.
examined whether instructions should be delivered using a human voice
or machine voice and found improved performance when utilizing humanvoices. Kalyuga et al. looked specifically at the modality of descriptions for
diagrams. They found no differences between audio text, visual text, andaudio/visual text used to describe on-screen diagrams. Lusk & Atkinsoncompared animated prompts, static prompts, and no prompts and foundsuperior performance for animated prompts.

68

D. A. Johnson and S. Rubin

GRAPHICS/TEXT

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Three comparisons examined the effects of integrating graphics and text orusing a split approach (Bodemer et al., 2004c1, 2004c2; Moreno & Valdez,
2005c1). All three found that integrating graphics and text improved performance. Johari (2003) compared CBI with graphs to CBI without graphs
and found the inclusion of visuals improved performances. Parchman et al.
(2000) found that the inclusion of multimedia elements improved performance over their exclusion. Prestera, Clariana, and Peck (2005) lookedat whether matching screen colors of teaching and testing frames would
improve retention over mismatches. No differences were found between
conditions.
RULES VERSUS LEARNING BY EXAMPLES
Two studies compared CBI utilizing definitional learning (detailed rules) versus exemplar learning (examples/nonexamples). One study found exemplar
training to be superior (Fox & Ghezzi, 2003), whereas the other study foundno differences (Hopkins, 2002).
RELATIONSHIP TRAINING
One study examined the effect of requiring users to learn concepts throughthe following methods: by training users to classfiy concepts, training users
how concepts are related, a mixture of classification and relationship training, or no training (Fox & Sullivan, 2007). Performance was higher whenconcept learning utilized classification training only. Another study compared three different methods for presenting information: task information,
cognitive information, and functional validity information (Gattie & Bisantz,
2006). Task information involved detailing how information presented isrelated to the training criteria. Cognitive information involves detailing which
cognitive strategies (i.e., rules) should be used. Functional validity information involves emphasizing the relationships between training criteria andrules being used. The outcome of the study found that the task information
format would produce the highest results. A study by Munyofu et al. (2007)
compared the presentations of animations alone, animation and text mixed
together with a focus on the current topic, and animation and text mixed
together with a focus on the relationships between topics. Performance
was found to be higher when the instruction focused on the relationships
between topics.
POSTFEEDBACK DELAYS
One study examined inclusion and exclusion of postfeedback delays (Kelly& Crosbie, 1997). Postfeedback delays involve an arrangement where theEffectiveness of Interactive CBI69

user’s answer and relevant feedback are displayed following a response,and users cannot progress to subsequent material for a set period of time.
Postfeedback delays were shown to be beneficial for learning.
TIMED CBIOne study (Moreno & Valdez, 2005c2) looked at imposing a time limit forcompletion of CBI in its entirety (not just practice components) versus notime limits. The study found no differences between conditions.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

OPEN BOOK/CLOSED BOOK INTERACTIONS
One study looked at the effect of allowing supplemental materials during
training interactions (not testing interactions) versus not allowing such material (Rohlman et al., 2004c3). The study found that an open book format
produced superior results during testing sessions.
RESPONSE REQUIREMENTS
Five comparisons examined the effects of requiring overt responses orrequesting covert responses (Bodemer et al., 2004c1, 2004c2; Miller &Malott, 1997c1, 1997c2; Tudor, 1995). Four comparisons found that an overt
response requirement resulted in improved performance. One comparison(Bodemer et al., 2004c1) found no differences. Three studies compared theeffects of interaction formats that require users to compose a response versus formats that require users to select a response (Clariana, 2003, 2004;
Katayama, Shambaugh, & Doctor, 2005). All three studies favored requiring
users to compose a response (fill-in-the-blank) rather than select a response(multiple choice). One study compared the effects of requiring users to compose original note summaries versus requiring users to copy and paste text tocreate notes while using large and small graphical organizers (Crooks, White,& Barnard, 2007). This study found no significant main effects between conditions, although a significant interaction was found. One study examinedthe effect of required notes to be composed versus making note composition optional (Armel & Shrock, 1996) and found performance was higher
when note taking was required.
MULTIPLE PATHWAYS VERSUS SINGULAR PATH TO CORRECT ANSWER
One study (Yadrick, Regian, Connolly-Gomez, & Robertson-Schule, 1997)
examined the differential effects of allowing users different methods of producing a correct answer versus forcing users to duplicate the trained method
exactly. Allowing users to employ any method was found to produce thebest results.

70

D. A. Johnson and S. Rubin

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

EFFECTS OF FEEDBACK
Out of the 79 comparisons, 25 (31.6%) utilized contingent and nonspecificfeedback, 18 (22.8%) utilized contingent and specific feedback, 6 (7.6%)
utilized noncontingent feedback, and 9 (11.4%) did not utilize feedback.
Among the remaining comparisons, 19 lacked sufficient details to identify the type of feedback used and 2 (Green et al., 2007; Morrison, Ross,
Gopalakrishnan, & Casey, 1995) used a mixture of feedback types. Only
two studies made direct comparisons of feedback approaches (Gibbons,
Robertson, & Thompson, 2001; Morrison et al., 1995). Gibbons et al. examined the effects of providing expert feedback with the effects of providing
learner feedback. Comparative feedback involved evaluative comparisons
between answers learners selected with answers that should have been
selected. Learner feedback involved objective summaries of the selections
that learners had made. The study found favorable results for providing
evaluative feedback (expert commentary) over providing objective feedback(summary of learner performance). The study also compared the effects ofsingle step feedback (what should be learned from a particular step) withlinked feedback (what should be learned from a particular step as well as
how the step relates to other steps). No differences were found in learnerperformance.
Morrison et al. (1995) looked at four different feedback formats: knowledge of correct response (KCR), delayed knowledge of correct response(delayed KCR), answer until correct (AUC), and no feedback. KCR involved
learners being informed of the correct answer following a response, regardless of the accuracy of the response (noncontingent feedback). The delayed
KCR format was identical to KCR, except that feedback was not delivered
immediately after the response. With the AUC format, learners would be
informed of the answer only after an accurate response (contingent feedback). Both KCR and delayed KCR were found to be superior to AUC
and no feedback. However, no differences were found between KCR anddelayed KCR.
MASTERY LEARNING
Out of the 79 comparisons, 12 (15.2%) included some form of mastery
learning. Two of those studies made direct comparisons of mastery learning (Montazemi & Wang, 1995; Moreno & Valdez, 2005c3). Montazemi andWang compared the inclusion and exclusion of mastery learning criteria
and found that the inclusion of mastery improved learning. Moreno andValdez (2005c3) looked at two different mastery learning formats: one inwhich learners had to repeat multiple previous steps following an error with
one in which learners had to repeat a single previous step following anerror. Repetition of multiple previous steps produced better performance
than repetition of a single previous step.

Effectiveness of Interactive CBI71

INCENTIVES

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Out of the 79 comparisons, 21 (26.6%) involved the use of external incentives (rewards and punishers delivered by a source other than the computerassisted instruction). In four of those studies, the incentives were delivered
contingent upon specific performance. Seventeen studies involved incentives delivered for just completion of the program and were otherwise independent of specific performance. Two studies (Morrison et al., 1995; Munson& Crosbie, 1998) directly compared the use of performance-dependentand performance-independent incentives. Both studies found that making
incentives dependent upon specific performance during learning improved
performance.

DISCUSSION
Consistent with previous reviews (J. A. Kulik, 1994; C.-L. C. Kulik &J. A. Kulik, 1991), the overall results found that interactive CBI improved
performance. A number of best practices can be recommended based on
the findings of this review. Foremost, interactive CBI is an effective andrecommended method for delivery of training materials, one that is very
likely to produce superior learning outcomes over other approaches suchas noninteractive CBI, textbook instruction, and lecture instruction. Even
if one does not wish to implement a training approach dependent on CBIalone, supplementing current instructional methods with interactive CBI also
appears to improve learner performance. Supervisors and managers wishing
to maximize their training practices should give strong consideration to CBI.
Furthermore, such supervisors and managers would also do well to carefully
look at the content of the CBI solutions already in use or under consideration for use, making sure that these programs are truly interactive and notjust electronic page turners.
Furthermore, the results of this article also suggest many ways in which
the antecedents to behavior should be arranged in a CBI environment,
which are important to consider for training developers when designing theirown training solutions. In contrast to theories such as constructivism that
advocate learner controlled instructional sequences (Alberto & Troutman,
2003; Carnine, 1992; Hirsch, 1996), the conclusions here support approaches
that advocate scripted instructional sequences. Utilizing a high number ofpractice items appears to be beneficial. When deciding on the modality
for the instructional narration, using auditory delivery is recommended.
Integrating visuals and graphics into CBI appears to have beneficial outcomes. In regards to the type of behavior to be evoked, the results suggestthat responding should be kept at the overt level, as opposed to covert
responding. Furthermore, learner responding in the form of composing
a response (fill-in-the-blank) is recommended over selecting a responseDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

72

D. A. Johnson and S. Rubin

formats (multiple choice). If incentives are used, improved performance will
be produced if incentives are contingent upon specific performance during
learning, not just simply completing the program.
Some of the studies examined warrant further discussion. For instance,
Segal et al. (2003) appears to stand in contrast to the other studies in thata video presentation was found to be just as effective as interactive CBI.
However, there were methodological concerns that limit the conclusions tobe drawn from this study. Namely, the video showed feedback for all possible answers, both correct and incorrect solutions. The CBI program only
showed feedback for solutions the learner selected. As such, the learners in
the video condition were exposed to considerably more details and instruction. Batchelder and Rachal (2000) is another study with unusual results.
In that study, no differences were found between lecture alone and lecture
supplemented with CBI. However, it is important to note that there were no
gains in learning from pre- to posttest in either condition. The participants inthe study were prison inmates, and as noted by the authors, the participants
appeared more interested in an excuse to be out of their cells rather thanlearning any of the material. Shute et al. (1996) is another unusual exception,
demonstrating no differences between a lecture format and interactive CBI.
This study contains a noteworthy confound: learners in the lecture condition
were told that there would be a posttest and all learners completed the curriculum, whereas learners in the CBI condition were not informed that there
would be a posttest and many of them did not complete the curriculum prior
to the posttest. Perhaps if the methodological concerns above had been controlled for, there may have been even further evidence of interactive CBI’s
effectiveness.
Only two studies found an alternative instructional method to be superior to interactive CBI (noninteractive conditions in both Howard-Jones &Martin, 2002c1 and Kerwin, 2006). In the Howard-Jones & Martin study, theauthors noted that the interactions were designed to cover only some of therelevant material. Thus, it is possible that the interactions reduced attention
to other relevant material for which no interactions were created, whereas
in the noninteractive condition all of the material was given equal attention,
ultimately resulting in a depressive effect on the test scores of the interactive
condition. When the authors conducted a follow-up experiment (HowardJones & Martin, 2002c2) in which all of the relevant material was involved in
the interactions, interactive CBI was found to be superior to noninteractive
CBI. The reasons for the atypical results found in Kerwin (2006) are lessclear.
There are a number of limitations to this review article worth noting.
The studies often varied greatly in terms of research design and experimental rigor, yet these differences are not reflected in the data set. Only a small
number of studies involved training settings, potentially limiting the generalization of results. Another potential limitation is the number of studies

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI73

that were excluded due to the exclusion criteria. Unfortunately, there area relatively small number of studies that meet such criteria. Although thenumber could be easily increased by including studies that were simply
demonstrations (i.e., made no comparisons between formats or conditions),
this would not aid in discovering best practices. Similarly, this number could
also be greatly increased by including perceived gains in learning (self-reportmeasures only such as “I feel like I learned more with this program”), but
this may not necessarily tell us about real gains in learning, which is ofparamount importance to organizations. Another shortcoming of this study
is that PsycINFO was the only database used to identify articles. There may
have been other articles that would have met the inclusionary criteria, but
are not included in PsycINFO and thus not included in this review. Future
reviewers may wish to broaden the scope to databases besides PsycINFO.
Another weakness of this review is that there was little attempt to control
for differences in experimental rigor between studies, with all studies being
given equal importance. Future reviews of this type would benefit from anattempt to classify the results according to relative experimental strength.
Despite the advantages of CBI cited throughout this article, it should be
noted that CBI is not a cure-all solution free of disadvantages. For example, CBI often requires a greater up-front investment for the development oftraining materials (Kruse & Keil, 2000). Certain affective, motor, and interpersonal skills may not be well suited for CBI (Heinich, Molenda, & Russell,1993). Computer programming also has not yet reached a point where thecorrectness of complex learner answers can be easily evaluated (Chase,1985). Some have tried to address this concern by utilizing a hybrid of CBIand human evaluators, where the majority of instruction is handled by CBI,
but humans are still in charge of evaluating learner responses (see CAPSI for
an example: Pear & Crone-Todd, 1999; Pear & Martin, 2004). Individuals in
a CBI learning environment may experience a reduction in both social andcultural interactions (Heinich et al., 1993; Kruse & Keil, 2000). Ultimately,
computers are not a magic fix for training, but require careful attention to
design and learner behavior, which is often lacking in current CBI programs(Engelmann, 1992; Skinner, 1984; Tudor & Bostow, 1991). Interactive CBI
can only reach its true potential through careful research and thoughtful
implementation.
On the basis of this review, there are several possible directions for
OBM researchers to pursue when investigating best training practices. Only
13 of the comparisons involved training situations. Although education andtraining are highly similar activities and all of the articles were selected with
relevance to workplace training in mind (i.e., only studies with normal adults
were included), it would be beneficial to the field if more studies of this typewere conducted in workplace situations.
Besides investigating the usage of interactive CBI in various training
situations, more studies on how altering the various aspects of CBI canDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

74

D. A. Johnson and S. Rubin

improve training outcomes are warranted. For example, none of the studies
compared computer programs where pacing was completely controlled by
the learner versus pacing controlled by the computer. In regards to interactions, adding further practice items appears to be beneficial. However, theupper limit to such additions is unknown. In fact, discussions over lean programming would suggest that the addition of too many practice items might
actually be harmful (Markle, 1990; Rummler, 1965). Further, whether suchpractice in a CBI format should be timed or untimed has not been established. Another area in need of more research is whether CBI works best via
learning from definitions or examples/nonexamples. Although one study
found that postfeedback delays improved performance (Kelly & Crosbie,
1997), more research is needed to better establish and expand upon thisarea. For example, when do such delays become so short that they lose theireffectiveness or so long as to produce detrimental learner frustration? CBIcomparisons of different types of feedback, direct comparisons of masteryand nonmastery learning, and performance dependent versus performance
independent incentives are also lacking.
Ultimately there is much to discover about the ways to improve organizational training and the best practices of interactive computer-based
training. Fortunately, due to behavior analysis’ emphasis on carefully examining the environmental variables that influence behavior, behavior analysts
are well suited to the task of pursuing such research.

REFERENCES
Aberson, C. L., Berger, D. E., Healy, M. R., & Romero, V. L. (2003). Evaluationof an interactive tutorial for teaching hypothesis testing concepts. Teaching of
Psychology, 30, 75–78.
Alberto, P. A., & Troutman, A. C. (2003). Applied behavior analysis for teachers (6thed.). Upper Saddle River, NJ: Pearson.
Armel, D., & Shrock, S. A. (1996). The effects of required and optional computerbased note taking on achievement and instructional completion time. Journalof Educational Computing Research, 14, 329–344.
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94, 416–427.
Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency
in multimedia learning: Examining the impact of an animated agent’s voice.
Contemporary Educational Psychology, 30, 117–139.
Batchelder, J. S., & Rachal, J. R. (2000). Efficacy of a computer-assisted instruction
program in a prison setting: An experimental study. Adult Education Quarterly,
50, 120–133.
Bell, B. S., & Kozlowski, S. W. (2002). Adaptive guidance: Enhancing selfregulation, knowledge, and performance in technology-based training.
Personnel Psychology, 55, 267–306.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI75

Binder, C., & Watkins, C. L. (1990). Precision teaching and direct instruction: Measurably superior instructional technology in schools. Performance
Improvement Quarterly, 3, 74–96.
Blasko, D. G., Kazmerski, V. A., & Torgerson, C. N. (2004). COR V2: Teaching observational research with multimedia courseware. Behavior Research Methods,
Instruments, & Computers, 36, 250–255.
Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration of information during learning with dynamic and interactive visualisations.
Learning and Instruction, 14, 325–341.
Braksick, L. W. (2000). Unlock behavior, unleash profits. New York, NY: McGrawHill.
Brethower, D., & Smalley, K. (1998). Performance-based instruction: Linking
training to business results. San Francisco, CA: Jossey-Bass/Pfeiffer.
Brothen, T., & Wambach, C. (2004). The value of time limits on internet quizzes.
Teaching of Psychology, 31, 62–64.
Bruce, G. S. (2004). Instructional design made easy. Atlanta, GA: Aubrey Daniels
International.
Carnine, D. (1992). Expanding the notion of teachers’ rights: Access to tools thatwork. Journal of Applied Behavior Analysis, 25, 13–19.
Chase, P. N. (1985). Designing courseware: Prompts from behavioral instruction. The
Behavior Analyst, 8, 65–76.
Clariana, R. B. (2003). The effectiveness of constructed-response and multiple-choicestudy tasks in computer aided learning. Journal of Educational Computing
Research, 28, 395–406.
Clariana, R. B. (2004). An interaction of screen colour and lesson task in CAL. British
Journal of Educational Technology, 35, 35–43.
Connolly, T. M., MacArthur, E., Stansfield, M., & McLellan, E. (2007). A quasiexperimental study of three online learning courses in computing. Computers& Education, 49, 345–359.
Cook, D. A., Gelula, M. H., Dupras, D. M., & Schwartz, A. (2007). Instructional
methods and cognitive and learning styles in web-based learning: Report oftwo randomised trial. Medical Education, 41, 897–905.
Croninger, W. R., Tumiel, J. P., & Sowa, T. (1995). The effects of hypercard tutorial
on student learning of biomechanics. The American Journal of Occupational
Therapy, 49, 456–460.
Crooks, S. M., White, D. R., & Barnard, L. (2007). Factors influencing the effectiveness of note taking on computer-based graphic organizers. Journal of
Educational Computing Research, 37, 369–391.
Daniels, A. C. (2000). Bringing out the best in people: How to apply the astonishingpower of positive reinforcement. New York, NY: McGraw-Hill.
Daniels, A. C., & Daniels, J. E. (2004). Performance management: Changing
behavior that drives organizational effectiveness. Atlanta, GA: Aubrey Daniels
International.
Darabi, A. A., Nelson, D. W., & Palanki, S. (2007). Acquisition of troubleshooting
skills in a computer simulation: Worked example vs. conventional problem
solving instructional strategies. Computers in Human Behavior, 23, 1809–1819.

Downloaded By: [Western Michigan University] At: 15:32 13 March 201176

D. A. Johnson and S. Rubin

Davis, D. R., Bostow, D. E., & Heimisson, G. T. (2007). Strengthening scientificverbal behavior: An experimental comparison of progressively prompted andunprompted programmed instruction and prose tutorials. Journal of Applied
Behavior Analysis, 40, 179–184.
Dornisch, M. M., & Sperling, R. A. (2006). Facilitating learning from technologyenhanced text: Effects of prompted elaborative interrogation. The Journal of
Educational Research, 99, 156–165.
Eckerman, D. A., Lundeen, C. A., Steele, A., Fercho, H. L., Ammerman, T. A., &Anger, W. K. (2002). Interactive training versus reading to teach respiratory
protection. Journal of Occupational Health Psychology, 7, 313–323.
Engelmann, S. (1992). War against the schools’ academic child abuse. Portland, OR:
Halcyon House.
Evans, C., & Gibbons, N. J. (2007). The interactivity effect in multimedia learning.
Computers & Education, 49, 1147–1160.
Fisher, J. B., Deshler, D. D., & Schumaker, J. B. (1999). The effects of an interactive multimedia program on teachers’ understanding and implementation of an
inclusive practice. Learning Disability Quarterly, 22, 127–142.
Fitzgerald, G. E. (1995). The effects of an interactive videodisc training programin classroom observation skills used as a teaching tool and as a learning tool.
Computers in Human Behavior, 11, 467–479.
Flora, S. R., & Logan, R. E. (1996). Using computerized study guides to increase
performance on general psychology examinations: An experimental analysis.
Psychological Reports, 79, 235–241.
Fox, E. J. (2004). The personalized system of instruction: A flexible and effective
approach to mastery learning. In D. J. Moran & R. W. Malott (Eds.), Evidencebased educational methods (pp. 201–221). San Diego, CA: Elsevier.
Fox, E. J., & Ghezzi, P. M. (2003). Effects of computer-based fluency training on
concept formation. Journal of Behavioral Education, 12, 1–21.
Fox, E. J., & Sullivan, H. J. (2007). Comparing strategies for teaching abstract concepts in an online tutorial. Journal of Educational Computing Research, 37,
307–330.
Gattie, G. J., & Bisantz, A. M. (2006). The effects of integrated cognitive feedback components and task conditions on training in a dental diagnosis task.
International Journal of Industrial Ergonomics, 36, 485–497.
Gibbons, A. S., Robertson, D. J., Duffin, J. R., & Thompson, B. (2001). Effects
of administering feedback following extended problem solving. Journal of
Educational Computing Research, 25, 417–426.
González, G. M., & Birch, M. C. (2000). Evaluating the instructional efficacy
of computer-mediated interactive multimedia: Comparing three elementary
statistics tutorial modules. Journal of Educational Computing Research, 22,
411–436.
Green, R. S., Eppler, M. A., Ironsmith, M., & Wuensch, K. L. (2007). Review question formats and web design usability in computer-assisted instruction. British
Journal of Educational Technology, 38, 679–686.
Hahne, A. K., Benndorf, R., Frey, P., & Herzig, S. (2005). Attitude towards computerbased learning: Determinants as revealed by a controlled interventional study.
Medical Education, 39, 935–943.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI77

Heinich, R., Molenda, M., & Russell, J. D. (1993). Instructional media and the newtechnologies of instruction (4th ed.). New York, NY: Macmillan.
Henry, M. J. (1995). Remedial math students’ navigation patterns through hypermedia software. Computers in Human Behavior, 11, 481–493.
Hickman, J. S., & Geller, E. S. (2003). Self-management to increase safe driving among short-haul truck drivers. Journal of Organizational Behavior
Management, 23, 1–20.
Hirsch, E. D., Jr. (1996). The schools we need and why we don’t have them. New
York, NY: Doubleday.
Hopkins, M. T. (2002). The effects of computer-based expository and discovery
methods of instruction on aural recognition of music concepts. Journal of
Research in Music Education, 50, 131–144.
Howard-Jones, P. A., & Martin, R. J. (2002). The effect of questioning on concept
learning within a hypertext system. Journal of Computer Assisted Learning, 18,
10–20.
Jenny, N. Y. Y., & Fai, T. S. (2001). Evaluating the effectiveness of an interactive
multimedia computer-based patient education program in cardiac rehabilitation.
The Occupational Therapy Journal of Research, 21, 260–275.
Johari, A. (2003). Effects of inductive multimedia programs in mediating word
problem translation misconceptions. Journal of Instructional Psychology, 30,
47–68.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention andredundancy in multimedia instruction. Applied Cognitive Psychology, 13,
351–371.
Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating experience into
the design of multimedia instruction. Journal of Educational Psychology, 92,
126–136.
Katayama, A. D., & Crooks, S. M. (2003). Online notes: Differential effects ofstudying complete or partial graphically organized notes. The Journal of
Experimental Education, 71, 293–312.
Katayama, A. D., Shambaugh, R. N., & Doctor, T. (2005). Promoting knowledgetransfer with electronic note taking. Teaching of Psychology, 32, 129–131.
Kelly, G., & Crosbie, J. (1997). Immediate and delayed effects of imposed postfeedback delays in computerized programmed instruction. The Psychological
Record, 47, 687–698.
Kerwin, M. E. (2006). Evaluation of a computer-based instructional package about
eating disorders. Computers in Human Behavior, 22, 1059–1066.
Kritch, K. M., & Bostow, D. E. (1998). Degree of constructed-response interaction incomputer-based programmed instruction. Journal of Applied Behavior Analysis,
31, 387–398.
Kruse, K., & Keil, J. (2000). Technology-based training: The art and science of design,development, and delivery. San Francisco, CA: Jossey-Bass/Pfeiffer.
Kulik, J. A. (1994). Meta-analysis studies of findings on computer-based instruction.
In E. L. Baker and H. F. Harold, Jr. (Eds.), Technology assessment in educationand training (pp. 9–33). Hillsdale, NJ: Erlbaum.
Kulik, C.-L. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An
updated analysis. Computers in Human Behavior, 7, 75–94.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

78

D. A. Johnson and S. Rubin

Lusk, M. M., & Atkinson, R. K. (2007). Animated pedagogical agents: Does
their degree of embodiment impact learning from static or animated worked
examples? Applied Cognitive Psychology, 21, 747–764.
Luyben, P. D., Hipworth, K., & Pappas, T. (2003). Effects of CAI on the academic
performance and attitudes of college students. Teaching of Psychology, 30,
154–158.
Maki, R. H., & Maki, W. S. (2003). Prediction of learning and satisfaction in webbased and lecture courses. Journal of Educational Computing Research, 28,
197–219.
Maki, R. H., Maki, W. S., Patterson, M., & Whittaker, P. D. (2000). Evaluationof a web-based introductory psychology course: I. Learning and satisfaction
in on-line versus lecture courses. Behavior Research Methods, Instruments, &Computers, 32, 230–239.
Maloney, M. (1998). Teach your children well: A solution to some of North America’s
educational problems. Cambridge, MA: Cambridge Center for Behavioral
Studies.
Markle, S. M. (1990). Designs for instructional designers. Champaign, IL: Stipes.
Martin, F., Klein, J. D., & Sullivan, H. (2007). The impact of instructional elements
in computer-based instruction. British Journal of Educational Technology, 38,
623–636.
Miller, M. L., & Malott, R. W. (1997). The importance of overt responding inprogrammed instruction even with added incentives for learning. Journal of
Behavior Education, 7, 497–503.
Molenda, M., & Russell, J. D. (2006). Instruction as an intervention. In J. A.
Pershing (Ed.), Handbook of human performance technology (pp. 335–369).
San Francisco, CA: Pfeiffer.
Montazemi, A. R., & Wang, F. (1995). On the effectiveness of a neural network
for adaptive external pacing. Journal of Artificial Intelligence in Education, 6,
379–404.
Moreno, R., & Valdez, A. (2005). Cognitive load and learning effects of having
students organize pictures and words in multimedia environments: The roleof student interactivity and feedback. Educational Technology Research andDevelopment, 53, 35–45.
Morris, E. (2001). The design and evaluation of Link: A computer-based learning
system for correlation. British Journal of Educational Technology, 32, 39–52.
Morrison, G. R., Ross, S. M., Gopalakrishnan, M., & Casey, J. (1995). The effects
of feedback and incentives on achievement in computer-based instruction.
Contemporary Educational Psychology, 20, 32–50.
Munson, K. J., & Crosbie, J. (1998). Effects of response cost in computerized
programmed instruction. The Psychological Record, 48, 233–250.
Munyofu, M., Swain, W. J., Ausman, B. D., Lin, H., Kidwai, K., & Dwyer, F.
(2007). The effects of different chunking strategies in complementing animatedinstruction. Learning, Media and Technology, 32, 407–419.
Oblinger, D. G., & Rush, S. C. (2003). The involvement of corporations in distanceeducation. In M. G. Moore and W. G. Anderson (Eds.), Handbook of distanceeducation (pp. 587–598). Mahwah, NJ: Erlbaum.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Effectiveness of Interactive CBI

79

Orey, M., Zhao, R., Fan, H.-L., & Keenan, R. (1998). Summative evaluation of the
SINCGARS tutor. Computers in Human Behavior, 14, 579–595.
Pampino, R. N., Jr., Wilder, D. A., & Binder, C. (2005). The use of functional assessment and frequency building procedures to increase product knowledge anddata entry skills among foremen in a construction organization. Journal of
Organizational Behavior Management, 25, 1–36.
Parchman, S. W., Ellis, J. A., Christinaz, D., & Vogel, M. (2000). An evaluation of
three computer-based instructional strategies in basic electricity and electronics
training. Military Psychology, 12, 73–87.
Pear, J. J., & Crone-Todd, D. E. (1999). Personalized system of instruction incyberspace. Journal of Applied Behavior Analysis, 32, 205–209.
Pear, J. J., & Martin, T. L. (2004). Making the most of PSI with computer technology.
In D. J. Moran and R. W. Malott (Eds.), Evidence-based educational methods(pp. 223–243). San Diego, CA: Elsevier.
Pelayo-Alvarez, M., Albert-Ros, X., Gil-Latorre, F., & Gutierrez-Sigler, D. (2000).
Feasibility analysis of a personalized training plan for learning researchmethodology. Medical Education, 34, 139–145.
Prestera, G. E., Clariana, R., & Peck, A. (2005). Memory-context effects of screen
color in multiple-choice and fill-in tests. Journal of Educational Multimedia
and Hypermedia, 14, 415–436.
Reisslein, J., Atkinson, R. K., Seeling, P., & Reisslein, M. (2006). Encountering theexpertise reversal effect with a computer-based environment on electrical circuit
analysis. Learning and Instruction, 16, 92–103.
Rohlman, D. S., Eckerman, D. A., Ammerman, T. A., Fercho, H. L., Lundeen, C. A.,
Blomquist, C., & Anger, W. K. (2004). Quizzing and feedback in computerbased and book-based training for workplace safety and health. Journal of
Organizational Behavior Management, 24, 1–26.
Rummler, G. A. (1965). The economics of lean programming. National Society for
Programmed Instruction Journal, 4, 8–10.
Sasson, J. R., & Austin, J. (2004). The effects of training, feedback, and participant
involvement in behavioral safety observations on office ergonomic behavior.
Journal of Organizational Behavior Management, 24, 1–30.
Schnackenberg, H. L., & Sullivan, H. J. (2000). Learner control over full andlean computer-based instruction under differing ability levels. Educational
Technology Research and Development, 48, 19–35.
Schultz, D., & Schultz, S. E. (2006). Psychology & work today. Upper Saddle River,
NJ: Pearson.
Segal, D., Chen, P. Y., Gordon, D. A., Kacir, C. D., & Gylys, J. (2003). Development
and evaluation of a parenting intervention program: Integration of scientific andpractical approaches. International Journal of Human-Computer Interaction,
15, 453–467.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasiexperimental designs for generalized causal inference. Boston, MA: Houghton
Mifflin.
Sholomskas, D. E., & Carroll, K. M. (2006). One small step for manuals: Computerassisted training in twelve-step facilitation. Journal of Studies on Alcohol, 67,
939–945.

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

80

D. A. Johnson and S. Rubin

Shute, V. J., & Gawlick, L. A., (1995). Practice effects on skill acquisition, learning
outcome, retention, and sensitivity to relearning. Human Factors, 37, 781–803.
Shute, V. J., Gawlick, L. A., & Gluck, K. A. (1998). Effects of practice and learnercontrol on short- and long-term gain and efficiency. Human Factors, 40,
296–310.
Shute, V. J., Gawlick-Grendell, L. A., Young, R. K., & Burnham, C. A. (1996).
An experiential system for learning probability: Stat Lady description andevaluation. Instructional Science, 24, 25–46.
Skinner, B. F. (1954). The science of learning and the art of teaching. Harvard
Educational Review, 24, 86–97.
Skinner, B. F. (1958). Teaching machines. Science, 128, 969–977.
Skinner, B. F. (1963). Reflections on a decade of teaching machines. Teachers College
Record, 65, 168–177.
Skinner, B. F. (1984). A matter of consequences. Washington Square, NY: NYU Press.
Slocum, T. A. (2004). Direct instruction: The big ideas. In D. J. Moran & R. W.
Malott (Eds.), Evidence-based educational methods (pp. 81–94). San Diego, CA:Elsevier.
Truman, S. M., & Truman, P. J. (2006). An investigation of the situated learnability
effects of single- and dual-modal systems in education: A report of musicoriented learning environment and science computer-assisted teaching studies.
British Journal of Educational Technology, 37, 131–142.
Tudor, R. M. (1995). Isolating the effects of active responding in computer-based
instruction. Journal of Applied Behavior Analysis, 28, 343–344.
Tudor, R. M., & Bostow, D. E. (1991). Computer-programmed instruction: The relation of required interaction to practical application. Journal of Applied Behavior
Analysis, 24, 361–368.
Van Rooij, S. W. (2007). WebMail versus WebApp: Comparing problem-based learning methods in a business research methods course. Journal of Interactive
Learning Research, 18, 555–569.
Whittam, K. P., Dwyer, W. O., & Leeming, F. C. (2004). Effects of quiz distribution on web-based instruction in an industrial training environment. Journal of
Educational Computing Research, 30, 57–68.
Yadrick, R. M., Regian, J. W., Connolly-Gomez, C., & Robertson-Schule, L. (1997).
Dyadic vs. individual practice with exploratory and directive mathematics
tutors. Journal of Educational Computing Research, 17, 165–186.

81

Education

Education

Atkinsonet al. (2005)

Batchelder &Rachal(2000)

Education

Atkinson(2002c1)

Education

Education

Armel &
Shrock(1996)Atkinson(2002c2)

Education

Goal

Aberson et al.
(2003)

Authors

Student

Student

Student

Student

Student

Pace

Supplemental Student

Primary

Primary

Primary

Primary

Primary

Usage

No

No

No

No

No

No

Unknown

ComposeComposeComposeCompose

SelectMastery Response

ContingentSpecific

None

None

None

None

ContingentSpecificFeedback

APPENDIX Summary of Studies Involving Interactive Computer-Based Instruction

CBI tutorial andlab workbook

Independent
variables

Summary of theresults(Continued)

Performancehigher under
CBI tutorial
Performance
Required note
Performance
Independent taking, optional
higher undernote taking, and required note
no note taking
taking
Performance
None
Audio agent
higher under
prompts, text
audio agent; noagent prompts,
differences
and nobetweenprompts;
animated agent, animated andstatic agent, and static agent, butboth improved
no agent
performanceover no agent
None
Audio agent
Performance
prompts, audio
higher under
only prompts,
audio agent
and text only
prompts
prompts
None
Human voice
Performance
prompts andhigher undermachine voice
human voiceprompts
prompts
None
CBI/lecture
No differences
format andbetweenlecture only
conditions
formatNone

Incentives

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

82

Primary

Primary

Bodemer et al. Education(2004c1)

Bodemer et al. Education(2004c2)

Primary

Education

Blasko et al.
(2004)

Primary

Usage

Training

Goal

Bell &
Kozlowski(2002)

Authors

APPENDIX (Continued)

Student

Student

Student

Machine

Pace

No

No

No

No

SelectSelectCompose

SelectMastery Response

None

Unknown

Unknown

ContingentSpecificFeedback

Independent
variables

Summary of theresults

Adaptive guidance Performance(machine
higher undercontrolled) andadaptivelearner control
guidance
None
CBI format andPerformance
lecture format
higher under
CBI format
Performance
Integrated
Performance
Independent graphics / text
higher underand split
integrated
graphics / text;
graphics / text;
Overt response
no differences
request andbetween overt
covert response and covert
requestresponserequests
Performance
Performance
Integrated
higher under
Independent graphics / text
integrated
with overt
graphics / text
responsewith overt
request,response
integrated
requests;
graphics / text
performancewith covert
response
higher underrequest, andscripted
split graphics
sequences
text with covert
responserequest;scriptedsequences andself-discovery
sequences

None

Incentives

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

83

Education

Education

Crooks et al.
(2007)

Education

Cook et al.
(2007)

Croninger
et al. (1995)

Education
Student

Student

Primary

Student

Supplemental Student

Primary

Primary

Supplemental Student

Clariana (2004) Education

Connolly et al.
(2007)

No

Supplemental Student

No

No

No

No

No

No

Supplemental Student

Education
Brothen &Wambach(2004)
Clariana (2003) Education

Compose

SelectMixed

Unknown

Various

Various

SelectContingentPerformance
Nonspecific
Dependent(Continued)

Performance
Timed andhigher under
untimedtimed quizzes
quizzesContingentPerformance
Compose andPerformance
NonspecificIndependent select response
higher whenrequests
learners mustcomposeresponses
ContingentNone
Compose andPerformance
Nonspecificselect response
higher whenrequests
learners mustcomposeresponses
Unknown
None
CBI format andPerformance
lecture format
higher under
CBI format
Unknown
None
Interactive CBI
No difference
andbetweennoninteractive
conditions
CBI
Performance
ContingentNone
CBI / lecture
higher under
Nonspecificformat and
CBI / lecture
lecture only
formatformatNone
None
Composed
No main effects
summary notes
betweenand copied
conditions;verbatim notes;
summary notes
small and large
more effective
graphical
when usingorganizers
large graphicalorganizers andverbatim notes
more effective
when using
small graphicalorganizers

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

84

Education

Training

Eckermanet al. (2002)

Education

Davis et al.
(2007)

Dornisch &Sperling(2006)

Education

Goal

Darabi et al.
(2007)

Authors

APPENDIX (Continued)

Primary

Primary

Primary

Primary

Usage

Student

Student

Student

Student

Pace

Yes

No

No

No

SelectComposeComposeUnknown

Mastery Response
None

Incentives

Independent
variables

Summary of theresults

Interactive
Performance
problemhigher undersolving,
interactive
noninteractive
problemprocess worked solving
examples, andnoninteractive
product worked
examplesContingentNone
Progressive
Performance
Nonspecificprompting,higher under
traditional
interactive
interactive, andconditions; nononinteractive
differences
CBI
betweeninteractive types
None
Performance
Suggested
No differences
Independent interactive CBI
betweenandformats
noninteractive
CBI
Performance
ContingentPerformance
Interactive CBI,
higher under
NonspecificIndependent viewing
interactive CBIinteractive CBI,noninteractive
CBI, andtraining manual

Unknown

Feedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

85

Education

Fox & Ghezzi(2003)

Education

Education

Flora & Logan(1996)

Gattie &Bisantz(2006)

Education

Fitzgerald(1995)

Education

Training

Fisher et al.
(1999)

Fox & Sullivan(2007)

Education

Evans &
Gibbons(2007)

Student

Student

Student

Primary

Primary

Primary

Student

Student

Student

Supplemental Student

Primary

Primary

Primary

No

No

Yes

No

No

No

No

SelectSelectSelectSelectCompose

SelectSelectNone

None

Interactive CBIandnoninteractive
CBI
CBI format andworkshopformat
CBI format andlecture format

Performancehigher underinteractive CBI(Continued)

No differences
betweenconditions
Unknown
None
Performancehigher under
CBI format
ContingentPerformance
CBI / lecture
Performance
Nonspecific
Dependent
format andhigher underlecture only
CBI / lectureformatformatContingentPerformance
Exemplar training Performance
NonspecificIndependent and definitional
higher underlearning;
exemplarfluency andtraining; nopractice
differences
betweenfluency andpractice
Performance
ContingentPerformance
Classification
higher with
Specific
Independent training,classification
multipletraining
relationstraining, both,and none
ContingentPerformance
Task information, Performance
Specific
Independent cognitive
higher with task
information,
informationand functional
validity
information

ContingentSpecific

ContingentSpecificDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

86

Primary

Primary

Primary

Education

Hopkins (2002) Education

Howard-Jones& Martin(2002c1)

Education

Student

Student

Student

Pace

Student

Student

Student

Supplemental Student

Primary

Henry (1995)

Education

Green et al.
(2007)

Primary

Education

Education

González &Birch (2000)

Primary

Usage

Hahne et al.
(2005)

Education

Goal

Gibbons et al.
(2001)

Authors

APPENDIX (Continued)

No

No

No

No

No

No

No

SelectMixed

Unknown

SelectComposeComposeComposeMastery Response

Incentives

Independent
variables

Summary of theresults

Noncontingent Performance
Comparative
Performance
Independent (expert)
higher under
feedback andcomparative
objective
feedback; no(learner)differences
feedback; single between single
step feedback
step and linked
and linked step
step feedback
feedback
Noncontingent Performance
Textbook, basic No differences
Independent CBI, multimedia between
CBI, and notraining
training
methods
VariousNone
Branching and
Performancelinear format
higher underbranchingformatNo differences
Unknown
None
CBI / lecture
betweenformat andinstructional
lecture only
formatsformatUnknown
None
Low learner
No differences
control andbetweenhigh learnerconditions
control
Unknown
None
Expository
No differences(detailed rules)
betweenand discovery
conditions(examples /nonexamples)
Performance
Noncontingent None
Interactive CBIhigher underandnoninteractive
noninteractive
CBI
CBI

Feedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

87

Primary

Primary

Primary

Education

Training

Training

Education

Johari (2003)

Kalyuga et al.
(1999)

Kalyuga et al.
(2000)

Katayama &Crooks
(2003)

Katayama et al. Education(2005)

Primary

Primary

PrimaryLife Skills

Jenny & Fai(2001)

Primary

Education

Howard-Jones& Martin(2002c2)

Student

Student

Student

MachineStudent

Student

Machine

No

No

No

No

No

No

No

Various

Compose

SelectSelectMixed

SelectCompose

None

None

Interactive CBIandnoninteractive
CBI
CBI format andlecture format

Performancehigher underinteractive CBI(Continued)

Performancehigher under
CBI format
Performance
CBI with graphs Performance
ContingentSpecificIndependent and CBIhigher underwithout graphs
CBI with graphs
Unknown
None
Audio narration, Performancetextualhigher undernarration, andaudio narration
audio / textual
mixturenarration
No differences
ContingentNone
Diagrams with
Nonspecificaudio, diagrams between
conditions
with text,
diagrams withaudio / textual
mixture, anddiagrams only
Performance
None
None
Interactive CBIhigher underandinteractive CBInoninteractive
CBI
None
None
Compose andPerformanceselect response
higher underrequests
composeresponserequests

Unknown

Unknown

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

88

Education

Education

Kritch andBostow(1998)

Luyben et al.
(2003)

Education

Kerwin (2006)

Education

Education

Kelly &
Crosbie(1997)

Lusk &Atkinson(2007)

Goal

Authors

APPENDIX (Continued)

Primary

Primary

Primary

Primary

Primary

Usage

Student

Student

Student

Student

Student

Pace

Yes

No

No

No

Yes

Mixed

ComposeComposeMixed

ComposeMastery Response

Incentives

Independent
variables

Summary of theresults

Noncontingent Performance
Postfeedback
Performance
Independent delays and nohigher under
postfeedback
postfeedback
delaysdelaysUnknown
None
Fully interactive Performance
CBI, partiallyhigher underinteractive CBI,noninteractive
andCBInoninteractive
CBI
ContingentNone
High interactivity, Performance
Nonspecificlowhigher underinteractivity,
highyoked (sameinteractivity
time as high),
and nointeractivity
Unknown
Performance
Animated agent
Performance
Independent prompts, static
higher underagent prompts,
animated agent
and noprompts;
prompts;
performance
displaying one
higher whenstep at a time
displayed one
and displaying
step at a time
all steps
simultaneously
ContingentNone
CBI and textbook Performance
Nonspecifichigher under
CBI

Feedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

89

Education

Education

Maki et al.
(2000)

Martin et al.
(2007)

Miller & Malott Education(1997c1)

Education

Maki & Maki
(2003)

Primary

Primary

Primary

Primary

Student

Student

Student

Student

No

No

Yes

Yes

Compose

SelectSelectSelectPerformance
Dependent

NoneContingentNone
Nonspecific

ContingentNone
Nonspecific

ContingentSpecificUnknown

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

(Continued)

Performancehigher under
CBI format
CBI and lecture
Performance
formathigher under
CBI format
Full CBI program Performance(information +higher under
objectives +interactive CBIpractice with(conditions
feedback +includingexamples +practice withreview), CBI
feedback)
program
without
objectives, CBI
program
without
examples, CBI
program
without
practice, CBI
program
without review,
and CBI
program with
only
information
Performance
Overt response
higher underrequest andcovert response overt response
requestrequest

CBI format andlecture format

90

Primary

Moreno &Valdez(2005c1)

Primary

Education

Education

Morris (2001)

Primary

Primary

Education

Moreno &Valdez(2005c2)Moreno &Valdez(2005c3)

Education

Primary

Montazemi &Education
Wang (1995)

Usage

Primary

Goal

Miller & Malott Education(1997c2)

Authors

APPENDIX (Continued)

Student

Student

Student

MachineStudent

Student

Pace

No

No

No

No

Yes

No

SelectComposeComposeCompose

SelectComposeMastery Response

Incentives

ContingentSpecific

None

ContingentNone
Nonspecific

ContingentNone
Nonspecific

ContingentNone
Nonspecific

ContingentNone
Nonspecific

ContingentPerformance
Nonspecific
Dependent

Feedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Summary of theresults

Overt response
Performancerequest andhigher undercovert response overt response
requestrequestMastery learning Performancehigher underwith timed
mastery
review after
learning witherrors and notimed review
masterylearning
Integrated
Performancegraphics / text,
higher undergraphics alone,
integrated
text alone;graphics / text;
interactive CBImixed results inandregards tointeractivity
noninteractive
CBI
Timed andNo differences
untimed CBI
betweenconditions
Reset of multiple Performancesteps after error
higher underand reset ofreset of multiple
single step after steps after error
error
CBI, textbook,
No differences
and no training
betweentraining
methods

Independent
variables

91

Education

Education

Munson &Crosbie(1998)

Munyofu et al.
(2007)

Primary

Primary

Primary

Parchman et al. Training(2000)

Pelayo-Alvarez Education
et al. (2000)

Primary

Primary

Primary

Training

Orey et al.
(1998)

Education

Morrison et al.
(1995)

Student

Student

Student

Student

Student

Student

No

No

Yes

Yes

No

No

SelectSelectSelectSelectCompose

SelectVarious

None

Unknown

None

ContingentNone
Nonspecific

ContingentSpecificNoncontingent None

ContingentVarious
NonspecificVarious

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

(Continued)

Performancehigher under
KCR anddelayed KCR(no difference
between KCR
types);
performancehigher undercontingent
incentives
Contingent
Performance
incentives andhigher undernoncontingent
contingent
incentives
incentives
Animation only, Performancetext / animation higher when
focused only on text / animation
current topic,
focused on
text / animation relationships
focused on
between topics
relationships
between topics
CBI format andPerformance
lecture format
higher under
CBI format
Multimedia CBI, Performancebasic CBI,
higher undergame-based
multimedia CBICBI, and lectureformat
CBI and training Performance
manualhigher under
CBI
KCR, delayed
KCR, AUC, no
feedback, andno questions;contingent
incentives andnoncontingent
incentives

92

Training

Education

Reisslein et al.
(2006)

Rohlman et al.
(2004c1)

Education

Goal

Prestera et al.
(2005)

Authors

APPENDIX (Continued)

Primary

Primary

Primary

Usage

Student

Student

Student

Pace

Yes

No

No

SelectComposeMixed

Mastery Response

Incentives

ContingentNone
NonspecificNoncontingent None

ContingentNone
NonspecificFeedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Summary of theresults

Screen colors
No differences
match during
between
instruction /conditions
testing andscreen colors
mismatching
during
instruction /testing
Showing worked No differencesout examples
betweenfollowed byconditions
problems,
showingproblems
followed byworked out
examples, andfading ofprompts
CBI, training
Performance
manual withhigher underquiz / answers,
CBI andtraining manual
training manualwith quiz only,with quiz /training manualanswers, nowith no quizdifferences
between CBIand training
manual withquiz / answers

Independent
variables

93

Training

Rohlman et al.
(2004c3)

Primary

Primary

Training

Training

Education

Sholomskas &Carroll(2006)

Shute &
Gawlick(1995)
Shute et al.
(1998)

Student

Student

Student

Student

Student

Student

Supplemental Student

Life Skills

Primary

Primary

Primary

Primary

Segal et al.
(2003)

Schnackenberg Education& Sullivan(2000)

Training

Rohlman et al.
(2004c2)

No

No

No

No

No

Yes

Yes

SelectCompose

Mixed

SelectMixed

SelectSelect(Continued)

Interactive CBI,No differences
partialbetweeninteractive CBI,
conditions
andnoninteractive
CBI
ContingentNone
Open book
Performance was
Nonspecificformat during
higher underinteractions and open bookclosed bookformatformat during
interactions
ContingentNone
More practice
Performance
Specific
items and lesshigher underpractice items;
more practicelearner decides
items; noif more practicedifferences if
needed andlearner orcomputercomputer
decides if moredecides if morepractice needed practice needed
ContingentPerformance
CBI and video
No differences
Specific
Independent tapebetweenconditions
Performance
Unknown
None
CBI / manual
higher underformat and
CBI / manual
manual only
formatformatUnknown
Performance
More practice
No differences
Independent items and lessbetweenpractice items
conditions
ContingentPerformance20, 14, 5–20
Performance
Specific
Independent (learnerhigher under
decided), 11,more (20)and 5 practicepractice items
items

ContingentNone
NonspecificDownloaded By: [Western Michigan University] At: 15:32 13 March 2011

94

Education

Training

Education

Whittam et al.
(2004)

Yadrick et al.
(1997)

Education

Tudor (1995)

Van Rooij(2007)

Primary

Education

Truman &Truman(2006)

Student

Student

Student

Student

Pace

Primary

Student

Supplemental Student

Primary

Primary

Primary

Usage

Education

Goal

Shute et al.
(1996)

Authors

APPENDIX (Continued)

No

No

No

No

No

Yes

Compose

SelectSelectCompose

SelectMixed

Mastery Response

None

None

Incentives

ContingentSpecific

ContingentSpecific

ContingentSpecific

None

None

None

ContingentNone
NonspecificUnknown

ContingentSpecificFeedback

Downloaded By: [Western Michigan University] At: 15:32 13 March 2011

Summary of theresults

No differences
CBI format,
betweenlecture format,
instructionaland noformats
instruction
Text only prompts Performancehigher underand voice-only
voice-only
prompts
prompts
Overt response
Performancerequest andhigher undercovert response overt response
requestrequestInteractive CBIPerformanceandhigher under
noninteractive
interactive CBICBIHigh, medium,
No differences
and low quiz
betweendensityconditions
Multiple possible Performancecorrect
higher undersolutions andmultipleexact
possible correct
duplication ofsolutions
instructions

Independent
variables