Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UKCorrespondence: djohnson@operant-tech.com

Slide 1

Going Beyond SimpleInteractions:
How to Get the Most Out of Your
Computer-Based Instruction

Douglas A. Johnson

Slide 2

CBI is clearly popular and growingSlide 3A long lost solution to typical training problems:
Teaching Machines

Justified?
Jump on bandwagon or be skeptical
Using PowerPoint that different orsame old training methods dressed up
in fancy multimedia clothing
A bunch of bells and whistles signifying
nothingFirst, a small history lesson
Teaching machines are the 1950sequivalent of CBIDesigning to overcome the shortcomingof traditional instruction
What’s wrong with our usual way of
training
Lecture: Too fast for some, too slow forothers, learners off-task, attempts tofacilitate interaction (mass responses)
just results in copying
Books: No interaction, you don’t know ifpeople are right until after training
Videotapes: Again, no interaction
One-on-one tutor: Better, but notnecessarily cost efficient and tutors
may not sequence material the best
Despite being clunky things, teachingmachines still fixed all of theseproblems!

Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UK
Correspondence: djohnson@operant-tech.com

Slide 4

Okay, if they were so good,where are they now?

Slide 5

CBI: A new hope

Forgotten Lessons from Teaching Machines






Obstacles
People were uncomfortable withcomputers and learning from themLess relevant in today’s highly
computerized world
IBM: Produced many machines,
wonderful results, decided to back out
at last moment, no contract
Rheem: All the models they produced
were defective
Eventually teaching machine advocates
got frustrated, quit, and just developed
textbooks. Today we’re not dependent
on the whims of manufacturers todevelop computer based instruction

So, what can we learn from theteaching machine movement, given weno longer face the obstacles they
faced?
Criteria for a good teaching machine /computer based instructional program
Continual Activity (interactive; makethem do something to ensure they’repaying attention and learning correctly)
Carefully tested and retested
sequences
Use small steps to eliminate discomfort
from being wrong
Write out responses rather than selectfrom multiple choice (recall vs.
recognition)
Learner Paced
Immediate feedback and reward
Mastery learning (prove you
understand concept A before you’reallowed to see concept B)

Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UK
Correspondence: djohnson@operant-tech.com

Slide 6

Slide 7The CBI that’s out there:
Good or Bad?

Efficient Enforced Demonstrative
Interactions: More than just clicking “next”









Slide 8Let’s see what the research says






Induce continual activity, but something
still seems missing
Claim to be interactive, multimedia
Sounds nice, but what do they usuallymean by interactive?
o User control over words andpictures that are presentedo Advancing material isn’t thatrevolutionary

For remainder of talk, when I say
interactive, this is what I’m referring toDifferent from simple user interactions
Books can’t, lecture can’t, videotapes
can’t, one-on-one tutors can’t
This is THE thing that’s truly special
about CBI. The thing that addresses all
those faults I mentioned earlier.
Unfortunately, it’s also the piece that is
usually missing in most computer
training programs

Assuming that teaching machine
criteria hold for CBI, but let’s confirm itand see what else is being looked at
Establish whether interactive CBI iseffective compared to other forms ofinstruction
Do the criteria for teaching machines
hold for to CBI?
What other variables are being
investigated?
Best practices
Future directions

Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UK
Correspondence: djohnson@operant-tech.com

Slide 9What’s wrong with most research out there?

Slide10Sorting through the research
PsycINFO (1995-2005)
Subject phrases: Computer AssistedInstruction, Computer Training, Teaching Machines,
Computer Assisted Diagnosis

Keywords: web-based training, web basedtraining, web-based instruction, web basedinstruction, computer-based training, computer basedtraining, computer-based instruction, computer basedinstruction, computer-aided training, computer aidedtraining, computer-aided instruction, computer aidedinstruction, computer-assisted training, computerassisted training, computer-assisted instruction,
computer assisted instruction, computer training,
multimedia instruction, multimedia training, internetinstruction, internet training, computer instruction

Blah, blah...Basically, looked
through a LOT of studies fora ten year period

Failure to utilize uniquely (noninteractive; might as well use a trainingmanual)
Excessive emphasis on antecedents
and inferred behaviors/processes
Too much reliance on social validation(“did you like it?”)o Lack of objective performanceoutcomes (did it work ornot???)Designs that don’t evaluate relative
effectiveness of CBIo One-group pretest-posttest
designs (math-test, teach,math-test)o Untreated control group design(two groups, one taught math,one not)o All you showed was better thannothing at all...big deal
Waded the thousands of weak studies
to present you with the few good
Focused on studies where variables
could be controlled by instructional
designero Pace of program, inclusion/type
feedback, etc
o Not self-directedness, locus ofcontrol, etc

Lots and lots of articles; some goodand many bad

Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UKCorrespondence: djohnson@operant-tech.com

Slide
11

CBI vs. Other InstructionCBI versus other instruction

57 used student-pacing
None compared student-pacing tomachine pacing
3 assessed composing vs. selecting
All favored composing
5 direct assessments of overt vs. covert
4 favored overt, 1 found no difference
15 used mastery learning, but only 1compared mastery learning CBI to no
mastery learning CBIFavored mastery learning

Out of 61 experimental comparisons

24 compared interactive CBI toanother type of instruction (noninteractive CBI (8),textbook/manual (5), lecture (7),human tutors (3), videotapes(1))
15 favored interactive CBI
7 found no differences
1 favored non-interactive CBI
1 favored videotape
0 favored textbook/manual
0 favored lectures
0 favored human tutors

Slide
12

Selected Findings
Out of 61 experimental comparisonsComposing vs. selecting
Overt vs. covert
Mastery learning vs.
non-mastery






Slide
13

Best Training Practices
Use interactive CBIEnforce overt responding
Make users compose responses
rather than select, if feasible
More interactions (practice) are better
than lessArrange it so that trial-and-errorresponding and racing are either
punished or preventedRacing: in an effort to complete an
instructional program as fast aspossible, learners often respond soquickly that mistakes are madeGoing Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UK
Correspondence: djohnson@operant-tech.com

Slide
14

Example of what can go wrong

Slide
15

Punish or Prevent Racing Induced Guessing

Mastery criteria
Early researchers built punishment
contingencies into their teachingmachines. Repeating sets on whicherrors were made is aversive because
it delays completion (reinforcer of being
finished is postponed).
Postfeedback delays
One study comparing postfeedback
delays and no postfeedback delays
favored postfeedback
Compose rather than selectExternal incentives
Specific performance dependent vs.
independent
2 studies compared dependent andindependent
Both favored dependent

Going Beyond Simple Interactions: How to Get the Most Out of Your Computer-Based Instruction
DOUGLAS A. JOHNSON
Western Michigan University
May 10th, 2007
Behavioural Management Techniques Leadership Conference, Warrington, Cheshire, UK
Correspondence: djohnson@operant-tech.com

Slide16

What do we still need to find out?

How to best prevent / punish of trialand-error responding and racing
Density of interactions? Frequently,infrequently, only at end of unit?
Machine vs. student-paced?
Errorless vs. error management?
o Small steps: reinforcing ortedious?
Imposing time limits?
How much practice should be used?

Test these things out yourselvesSlide17Questions?