The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 1

Slide 2

•BSA has a very long history(Brethower, 1972, 1982; Maley &Harshbarger, 1974; Morasky, 1982)•Developed a very elaborate andimpressive series of maps, charts, andother tools•Whether improves selection ofinterventions and adds value toorganizations remains open to scientificinquiry

Slide 3•Many definitions can be found•Common themes include large scale
perspectives with many interacting
partsThe Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 4

•Critical attributes when determining if
BSA or other behavior change approach•Implied that analysis is done prior tointervention

Slide 5

•For example•Boy's behavior analyzed in terms ofbehavioral consequences and token
economy selected as result•Not a neurological intervention eventhough multiple neurons, blood vessels,and tissues involved•Inappropriate to label the intervention
as neurologicalSlide 6

•Similarly•behaviors of employees analyzed interms at a single level and task
clarification, goal setting, and feedback
selected as result•Not BSA-driven intervention•inappropriate to consider the resultsof the intervention as supporting
behavioral systems analysis, even ifsystems factors were considered anddiscussed following the completion ofthe intervention.
•An after-the-fact analysis that
speculates about the role of systems
variables does not retroactively cause
an intervention to be informed by BSA

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 7

•Why do we bother with cumbersome
processSlide 8•It gives us pretty special outcomes•It helps us separate out
demonstrations of fact from simple
assertions of fact

Slide 9•Of particular interest is the ability tounderstand and establish causality

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 10

•Client testimonials, case studies, andactive usage•Important•But do not prove causality

Slide 11•Large amounts of time and money•Where the field currently is•Where we believe it should beExperimentation is the only way we know ourconfidence about causality isn’t misplaced

Slide 12•Why JOBM? Long history, BSA popularamong the JOBM audience•All issues of JOBM from 1977 thru
2010, volumes 1-30, were reviewed toidentify articles that included content
related specifically to BSA

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 13systems analysis, systems thinking, processing system, receiving
system, process maps, relationship map, total performancesystem, performance matrix, super system, organizational scan,
behavior engineering model, human performance system•Search terms•First level: All issues•Second level: 105 articles•Third level: 30 articles

Slides
14-16

Reviewed
30 BSA articles
for classification

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slides
17-21

Placed into one of five categories

Placed into one of five categories

Placed into one of five categories

Placed into one of five categories

Placed into one of five categories•No empirical data involving behavioral
systems analysis•Minimal empirical data involving
behavioral systems analysis•Experimental evidence involving
behavioral systems analysis approachcompared against a no-intervention
control condition•Experimental evidence involving
behavioral systems analysis approachcompared against an alternativeapproach condition•Experimental evidence involving one
behavioral systems analysis approachcompared against another behavioral
systems approach

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 22

BSA publication trends over time

•Publication trends

16141210Total
8

Experimental BSA data
Minimal BSA data

6

No BSA data420

1977-1980

Slide 23

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

Percentages for different
types of BSA articles
100•Percentages of article types•Note about Sasson, Alvero, & Austin(2006)

9080
7060
50
40
30
20
100
No BSA data

Slide 24

Minimal BSA data

Contrast with nointervention

Contrast with other Contrast between
intervention
BSA interventions•Search terms•Other journals•BSA definition

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, CO
Correspondence: djohnson@operant-tech.com

Slide 25

•Even acknowledging the possible
shortcomings•Lots of discussion about BSA

Slide 26

•We have plenty of articles simply
talking about BSA and some withminimal data
•It allows for demonstration of thephenomenon•Does not show cause and effect•Minimal data is not enoughSlide 27

•Lack of experimentation•Virtually none of our data allows forsatisfactory verification or replication

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 28

•Logic and sensibility are not enough•Behavioral systems analysis has great
potential•The plurality of anecdotes and casestudies does not prove causalrelationships

Slide 29

•As scientists, this is tremendous
opportunity•"Understand that I am trying to be
descriptive, not accusatory.“•At least three basic lines ofexperimental research that should be
conducted and are currently being
neglected•Namely studies comparing BSA
approaches to no interventions,alternative non-BSA approaches, anddifferent types of BSA approaches

There is a lot ofexperimentation
we could do

Slide 30

How do interventions driven by BSA
differ from other approaches?

•Does BSA lead to measurably superior
results?
•Does BSA lead to measurably superiorresults in comparison to traditional PMalone?
•Are visual representations of tools
necessary or will text-based
counterparts work just as well?
•Which systems tools are essential andwhich are excess?

The Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 31

How many levels ofanalysis are necessary?

Slide 32

Such research will bequite complex anddifficult to conduct•How many levels of analysis arenecessary?
•How much measurable gain is
obtained with each level of analysis?
•Does the mission statement need tobe considered and communicated to alllevels?
•Empirical evidence for suboptimization principle?
•Does BSA need to be done prior to anintervention, concurrently, or either?
•Is it better to implement an immediatechange in absent of systems analysis or
wait until analysis is complete andimplement delayed change?

•Outside of an experimental
framework, BSA is quite complex in andof itself•No doubt that such good experimental
or quasi-experimental research will bedifficult to conduct•Ability to place strong confidence inrecommending the use of BSA should
merit the extra effort and difficultyThe Science Behind Behavioral Systems Analysis:
A Review of the Journal of Organizational Behavior Management
DOUGLAS A. JOHNSON, Sarah E. Casella, and Sin Chien Lee
Western Michigan University
May 28th, 2011
Association for Behavior Analysis International 37th Annual Convention, Denver, COCorrespondence: djohnson@operant-tech.com

Slide 33

This difficult research is necessary to meetour goals of description, prediction,
understanding, and control•Mission statement of behavioral
sciences•We’re missing the piece of
demonstrating control•Serious disconnect between thescientific mission and scientificprocesses of behavioral systems
analysis•Many are aware of this disconnect•Presentation is not meant to solve theproblem, but rather suggest some waysof addressing the problems•Bring spotlight on this concern•Spur on more experimental work inthis field

Slide 34

Slide 35

THE END
Contact:
djohnson@operant-tech.com